MIT 201 – STATISTICAL COMPUTING I

TUTORIAL 2: Data Importing and Exporting in R

1 Libraries in R
Libraries, also known as packages, in R are collections of functions, data, and documentation
that extend the functionality of base R. Libraries provide additional capabilities and tools for
data manipulation, analysis, visualization, and more. In R, you can install and load libraries to
access their functions and resources.
1.1 Installing Libraries
Before using a library, you need to install it on your system. This is typically done once using
the install.packages() function, specifying the name of the library you want to install.
Example:
# Installing a library
install.packages("ggplot2")
1.2 Loading Libraries
Once a library is installed, you can load it into your R session using
the library() or require() function. Loading a library makes its functions and resources
available for use.
Example:
# Loading a library
library(ggplot2)
1.3 Commonly Used Libraries
R has a vast ecosystem of libraries covering various domains. Here are some commonly used
libraries:
• ggplot2: A powerful library for data visualization, offering a wide range of high-quality
plotting capabilities.
• dplyr: A library for data manipulation and transformation, providing efficient and
intuitive functions for filtering, summarizing, and transforming data.
• tidyr: A library for data tidying, used for reshaping and restructuring data into a more
organized format.
• readr: A library for reading and writing structured data in various formats, such as CSV,
Excel, and more.
• caret: A library for machine learning and predictive modeling, offering tools for data
preprocessing, model training, and evaluation.
• stringr: A library for string manipulation, providing functions for pattern matching,
string extraction, and text manipulation.
• lubridate: A library for working with dates and times, offering convenient functions for
parsing, manipulating, and formatting date-time data.
• magrittr: A library for enhancing the readability and expressiveness of R code by
enabling the use of the pipe operator %>%.
• purrr: A library that enhances functional programming in R, providing a consistent and
intuitive syntax for working with lists and vectors.
• rmarkdown: A library for creating dynamic and reproducible reports and documents using
R Markdown, allowing integration of code, text, and visualizations.
1.4 Using Library Functions
Once a library is loaded, you can use its functions by calling them directly. Library functions
are typically preceded by the library name or package name, followed by the function name.
Example:
# Using a function from the ggplot2 library
ggplot(data = mydata, aes(x = Age, y = Salary)) + geom_point()
Libraries in R extend the capabilities of base R, providing additional functionality and tools
for various tasks. By leveraging the power of libraries, you can enhance your data analysis,
visualization, and programming workflows in R.

2 tidyverse
The tidyverse is a collection of R packages designed to enhance data manipulation, analysis,
and visualization. It follows the principles of tidy data and provides a consistent and intuitive
grammar for working with data. The tidyverse packages are designed to work together
seamlessly, allowing you to build efficient and readable data workflows. Let's explore the
tidyverse in more detail:
2.1 Core Packages
The tidyverse consists of several core packages that form the foundation of the ecosystem.
These packages include:
• ggplot2: A powerful package for data visualization, offering a layered grammar of
graphics that enables the creation of highly customizable and publication-quality plots.
• dplyr: A package for data manipulation and transformation, providing a set of intuitive
functions for filtering rows, selecting columns, mutating data, and summarizing data.
• tidyr: A package for data tidying, used to reshape and restructure data into a more
organized format, such as converting data from wide to long format or vice versa.
• readr: A package for reading and writing structured data in various formats, such as
CSV, Excel, and more. It provides fast and efficient functions for importing data into R.
• purrr: A package that enhances functional programming in R, providing a consistent and
intuitive syntax for working with lists and vectors.
2.2 Loading the tidyverse
To use the tidyverse packages, you can load them all at once using
the library(tidyverse) command. This will load the core packages and their dependencies
into your R session.
Example:
# Loading the tidyverse
library(tidyverse)
2.3 Tidy Data Principles
The tidyverse follows the principles of tidy data, which emphasize a standardized format for
data representation. Tidy data has the following characteristics:
• Each variable has its own column.
• Each observation has its own row.
• Each value has its own cell.
By adhering to these principles, the tidyverse simplifies data manipulation and analysis
workflows.
2.4 Tidyverse Workflow
The tidyverse promotes a consistent and intuitive workflow for working with data. It typically
involves a series of steps, such as importing data, tidying data, transforming and summarizing
data, and visualizing the results using ggplot2.
Example:
# Example tidyverse workflow
library(tidyverse)

# Step 1: Importing data

data <- read_csv("data.csv")
# Step 2: Tidying data
tidy_data <- data %>%
gather(key = "Variable", value = "Value", -ID)

# Step 3: Transforming and summarizing data

summary_data <- tidy_data %>%
group_by(Variable) %>%
summarize(Average = mean(Value))

# Step 4: Visualization using ggplot2

ggplot(summary_data, aes(x = Variable, y = Average)) +
geom_bar(stat = "identity")
2.5 Additional tidyverse Packages
In addition to the core packages, the tidyverse ecosystem includes several other packages that
provide additional functionality for specific tasks. Some commonly used additional packages
include:
• stringr: A package for string manipulation, providing functions for pattern matching,
string extraction, and text manipulation.
• lubridate: A package for working with dates and times, offering convenient functions for
parsing, manipulating, and formatting date-time data.
• forcats: A package for working with categorical variables, providing functions for
manipulating and reordering factor levels.
• magrittr: A package that enhances the readability and expressiveness of R code by
enabling the use of the pipe operator %>%.
The tidyverse packages work harmoniously together, allowing you to build data analysis
pipelines that are efficient, readable, and reproducible.

3 Importing data from various file formats

R provides several packages and functions for importing data from various file formats.
Whether you need to import data from CSV, Excel, JSON, or other formats, R offers flexible
and efficient methods to handle different data sources. Let's explore some common file
formats and how to import data from them:
3.1 CSV Files
CSV (Comma-Separated Values) files are one of the most commonly used file formats for
tabular data. R provides the read.csv() function to import data from CSV files.
Example:
# Importing data from a CSV file
data <- read.csv("data.csv")
3.2 Excel Files
R offers multiple packages for importing data from Excel files, including readxl, openxlsx,
and xlsx. These packages provide functions such as read_excel() to read data from Excel
files.
Example (using readxl package):
# Installing and loading the readxl package
install.packages("readxl")
library(readxl)

# Importing data from an Excel file

data <- read_excel("data.xlsx", sheet = "Sheet1")
3.3 JSON Files
JSON (JavaScript Object Notation) files are commonly used for storing structured data. R
provides the jsonlite package, which includes functions like fromJSON() to import data from
JSON files.
Example:
# Installing and loading the jsonlite package
install.packages("jsonlite")
library(jsonlite)

# Importing data from a JSON file

data <- fromJSON("data.json")
3.4 XML Files
XML (eXtensible Markup Language) files are used for storing structured data.
The XML package in R provides functions like xmlParse() and xmlToDataFrame() to import
data from XML files.
Example (using XML package):
# Installing and loading the XML package
install.packages("XML")
library(XML)

# Importing data from an XML file

doc <- xmlParse("data.xml")
data <- xmlToDataFrame(doc)
3.5 Other File Formats
R supports many other file formats, such as SQLite databases, SAS files, SPSS files, and
more. Specific packages are available for each file format. For example,
the DBI and RSQLite packages can be used to import data from SQLite databases.
Example (importing from SQLite database using RSQLite package):
# Installing and loading the RSQLite package
install.packages("RSQLite")
library(RSQLite)

# Importing data from a SQLite database

con <- dbConnect(RSQLite::SQLite(), "database.db")
data <- dbGetQuery(con, "SELECT * FROM table")
dbDisconnect(con)
These are just a few examples of how to import data from various file formats in R. R
provides a wide range of packages and functions to handle different file formats, allowing
you to efficiently load and work with diverse data sources.
I hope these notes provide a clear understanding of importing data from various file formats
in R. If you have any further questions or need specific examples with any particular file
format, feel free to ask!

4 Importing data from the internet

R provides several packages and functions that allow you to directly import data from the
internet. Whether you need to retrieve data from a website, an API, or a remote server, R
offers flexible and efficient methods to handle web-based data sources. Let's explore some
common methods for importing data from the internet:
4.1 URL-based Data
You can directly import data from a URL using functions like read.csv() or read.table().
These functions accept a URL as the file path argument and retrieve the data from that URL.
Example:
# Importing data from a URL
url <- "https://example.com/data.csv"
data <- read.csv(url)
4.2 Web Scraping
Web scraping is the process of extracting data from websites. R provides several packages,
such as rvest and xml2, that facilitate web scraping. These packages allow you to navigate
and extract data from HTML or XML documents.
Example (using rvest package for web scraping):
# Installing and loading the rvest package
install.packages("rvest")
library(rvest)

# Web scraping example

url <- "https://example.com"
page <- read_html(url)
data <- page %>%
html_nodes("table") %>%
html_table()
4.3 APIs (Application Programming Interfaces)
Many websites and online services provide APIs to access and retrieve data
programmatically. R provides packages like httr and jsonlite that make it easy to interact
with APIs and import data into R.
Example (using httr and jsonlite packages to retrieve data from an API):
# Installing and loading the httr and jsonlite packages
install.packages("httr")
install.packages("jsonlite")
library(httr)
library(jsonlite)

# API request example

url <- "https://api.example.com/data"
response <- GET(url)
data <- fromJSON(content(response, "text"))
4.4 FTP (File Transfer Protocol)
If the data is available on an FTP server, you can use the RCurl or curl packages to import
data from the server. These packages provide functions to establish an FTP connection and
retrieve the data.
Example (using RCurl package to retrieve data from an FTP server):
# Installing and loading the RCurl package
install.packages("RCurl")
library(RCurl)

# FTP example
url <- "ftp://example.com/data.csv"
data <- getURL(url)
4.5 Other Protocols and Libraries
R supports additional protocols and libraries for importing data from the internet. For
example, the curl package allows you to make HTTP requests and retrieve data.
The XML and jsonlite packages can handle XML and JSON data retrieved from web
services.
Example (using curl package to retrieve data using HTTP):
# Installing and loading the curl package
install.packages("curl")
library(curl)
# HTTP request example
url <- "https://api.example.com/data"
response <- curl_fetch_memory(url)
data <- fromJSON(response$content)
These are just a few examples of how to import data from the internet into R. R provides a
wide range of packages and functions to handle web-based data sources, allowing you to
efficiently retrieve and work with data from websites, APIs, FTP servers, and more.

5 Web scrapping coronavirus data from Worldometer (Activity)

5.1 Introduction
The website "https://www.worldometers.info/coronavirus/ ↗" provides up-to-date and
comprehensive statistics on the coronavirus pandemic. It offers a wide range of information,
including the number of confirmed cases, deaths, recoveries, and active cases across different
countries. The website also provides graphs, trends, and other relevant data related to
COVID-19.
5.2 Web Scraping and Importing Data into R
5.2.1 Step 1: Install and Load Required Packages
To begin, make sure you have the necessary packages installed in R. We'll be using the
"rvest" package for web scraping and the "readr" package for data import.
install.packages("rvest")
install.packages("readr")
library(rvest)
library(readr)
5.2.2 Step 2: Fetch the Web Page
We'll start by fetching the web page using the read_html() function from the "rvest"
package.
url <- "https://www.worldometers.info/coronavirus/"
page <- read_html(url)
5.2.3 Step 3: Inspect the Web Page Structure
To scrape the table data, we need to identify the HTML structure of the table on the web
page. We can use the browser's developer tools or the html_nodes() function in R to inspect
the HTML elements.
# Inspect the HTML structure of the table
html_nodes(page, "table")
5.2.4 Step 4: Extract Table Data
Once we've identified the HTML structure of the table, we can use the html_table() function
to extract the table data into a data frame.
# Extract the table data
table_data <- page %>%
html_nodes("table") %>%
html_table(fill = TRUE)
5.2.5 Step 5: Clean and Manipulate the Data
The extracted table data might require some cleaning and manipulation to make it suitable for
analysis. You can use various functions from the "dplyr" package to perform data cleaning
tasks such as removing unnecessary columns, renaming columns, and converting data types.
# Convert the list to a data frame
df <- as.data.frame(table_data[[1]])

# Perform data cleaning and manipulation as needed

# For example, if you want to remove unnecessary columns
df <- df[, c("Country,Other", "TotalCases", "TotalDeaths", "TotalRecovered", "Acti
veCases")]

# Rename the columns for better readability

colnames(df) <- c("Country", "Total Cases", "Total Deaths", "Total Recovered", "Ac
tive Cases")
5.2.6 Step 6: Save the Data as a CSV File
Finally, you can save the extracted and cleaned data as a CSV file for further analysis or
visualization.
# Save the data as a CSV file
write_csv(df, "coronavirus_data.csv")
That's it! You have successfully scraped the table data from the
"https://www.worldometers.info/coronavirus/ ↗" website and imported it into R. You can
now use the extracted data for your desired analysis or visualization tasks.
Note: Web scraping should be done responsibly and in accordance with the website's terms of
service. Make sure to check if the website allows scraping and consider any applicable legal
and ethical considerations.

6 Importing CSV data into R from the web (Activity)

The data from the URL "https://www.stats.govt.nz/assets/Uploads/Annual-enterprise-
survey/Annual-enterprise-survey-2021-financial-year-provisional/Download-data/annual-
enterprise-survey-2021-financial-year-provisional-csv.csv ↗" represents the Annual
Enterprise Survey (AES) for the financial year 2021 in New Zealand. The Annual Enterprise
Survey is conducted by Statistics New Zealand and provides valuable insights into the
financial performance and characteristics of businesses in the country.
The dataset contains a wide range of information related to New Zealand businesses,
including their financial data, industry classifications, employment statistics, and more. It
offers a comprehensive view of the economic landscape and allows analysts, researchers, and
policymakers to assess the performance and trends in various sectors.
The data is structured as a CSV (Comma-Separated Values) file, which is a commonly used
file format for tabular data. Each row in the dataset represents a unique business entity, while
each column represents a specific attribute or variable associated with the businesses.
By importing and analyzing this data, researchers and analysts can gain insights into the
financial health, productivity, and characteristics of businesses in New Zealand. They can
explore trends, compare industries, identify patterns, and make data-driven decisions to
support economic growth and development.
It's important to note that the specific variables and attributes present in the dataset may vary,
and it's necessary to explore the data further to understand its structure and content.
To import data from the given URL into R:
6.1 Step 1: Install and Load Required Packages
To begin, make sure you have the necessary packages installed in R. We'll be using the
"readr" package for data import.
install.packages("readr")
library(readr)
6.2 Step 2: Set the URL
Set the url variable to the given URL that contains the CSV file we want to import.
url <- "https://www.stats.govt.nz/assets/Uploads/Annual-enterprise-survey/Annual-e
nterprise-survey-2021-financial-year-provisional/Download-data/annual-enterprise-s
urvey-2021-financial-year-provisional-csv.csv"
6.3 Step 3: Import the Data from the URL
Use the read_csv() function from the "readr" package to read the CSV data from the
specified URL. This function automatically detects the delimiter and parses the data into a
data frame.
data <- read_csv(url)
6.4 Step 4: Explore the Imported Data
After importing the data, it's a good practice to explore and understand its structure. Here are
a few functions that can help with that:
• head(data): View the first few rows of the imported data.
• dim(data): Check the dimensions of the data (number of rows and columns).
• names(data): View the column names.
• str(data): Display the structure of the data, including variable types.
For example:
head(data)
dim(data)
names(data)
str(data)
6.5 Step 5: Data Manipulation and Analysis
Once you have imported the data, you can perform various data manipulation and analysis
tasks based on your requirements. Here are a few common operations you can perform:
• Subsetting: Select specific rows or columns from the data.
• Filtering: Filter the data based on specific conditions.
• Aggregation: Calculate summary statistics or aggregate data by groups.
• Visualization: Create plots and visualizations to gain insights from the data.
For example, let's say you want to calculate the mean value for a specific variable in the
dataset. You can use the mean() function along with the $ operator to access the variable:
mean_value <- mean(data$variable_name)
6.6 Step 6: Save the Data
If you want to save the imported data as a local file on your computer, you can use
the write_csv() function from the "readr" package. This function allows you to save the data
as a CSV file.
write_csv(data, "output.csv")
In the above example, the imported data will be saved as "output.csv" in the current working
directory.
That's it! You have successfully imported the data from the given URL into R. You can now
explore, manipulate, and analyze the data further based on your specific requirements.
Note: Make sure you have a stable internet connection to access the URL and that the URL is
valid and accessible. Additionally, the read_csv() function assumes that the CSV file is
formatted correctly with appropriate headers and consistent data types. If the CSV file has
any special formatting or if you encounter any issues, you may need to adjust the code
accordingly.

7 Previewing data in R
Previewing data is an essential step in data analysis as it provides an initial understanding of
the dataset's structure and content. In R, there are several methods to preview data effectively.
The head() function allows you to quickly examine the first few rows of a dataset, giving you
a glimpse of the data's structure. Additionally, the View() function provides an interactive
spreadsheet-like view in RStudio, enabling you to explore the dataset more conveniently.
The str() function summarizes the structure of the data, displaying variable types and example
values. Furthermore, the summary() function provides summary statistics for each variable,
offering insights into the distribution and characteristics of the data. By utilizing these
previewing techniques in R, analysts can gain an initial understanding of the data, identify any
potential issues, and begin the process of data exploration and analysis.

7.1 Using the head() function

The head() function allows you to quickly preview the first few rows of a dataset in R. By
default, it displays the first 6 rows of the data. This function is useful for getting a glimpse of
the data and understanding its structure.
# Preview the first 6 rows of the dataset
head(dataset)
```

7.2 Specifying the number of rows to display

You can specify the number of rows to display using the n argument of the head() function.
This is helpful when you want to preview a specific number of rows, such as the first 10
rows.
# Preview the first 10 rows of the dataset
head(dataset, n = 10)
```

7.3 Using the View() function

The View() function opens a separate window in RStudio where you can interactively explore
the dataset. It provides a spreadsheet-like view where you can scroll through the data, sort
columns, and search for specific values. This is useful for examining larger datasets in a more
convenient manner.
# Open a separate window to view the dataset
View(dataset)
```

7.4 Using the str() function

The str() function provides a concise summary of the structure of the dataset. It displays the
data type and the first few values of each variable, allowing you to understand the overall
structure and content of the data.
# Display the structure of the dataset
str(dataset)
```

7.5 Previewing specific columns

If you want to preview specific columns of the dataset, you can use the indexing operator ($)
along with the head() function. This allows you to focus on specific variables of interest.
# Preview the first 10 rows of a specific column
head(dataset$column_name, n = 10)
```

7.6 Using the summary() function

The summary() function provides summary statistics for each variable in the dataset. It gives
you a quick overview of the central tendency, dispersion, and distribution of the data.
# Display summary statistics of the dataset
summary(dataset)
```
7.7 Customizing the preview
You can customize the preview of the data by combining different functions. For example,
you can use the select() function from the "dplyr" package to choose specific columns and
then apply the head() function to preview those selected columns.
library(dplyr)

# Preview the first 10 rows of specific columns

head(select(dataset, column1, column2), n = 10)
```
These techniques allow you to quickly get an overview of your data, understand its structure,
and make informed decisions about further data manipulation and analysis.

8 Exporting data from R to different file formats

8.1 Exporting to CSV
To export data from R to a CSV (Comma-Separated Values) file, you can use
the write.csv() or write.csv2() functions. These functions allow you to save your data
frame as a CSV file, which can be easily opened and read by other software applications.
# Export data to a CSV file
write.csv(data, file = "output.csv", row.names = FALSE)
```

The `file` parameter specifies the name and path of the output file, and th
e `row.names` parameter determines whether row names should be included in
the exported file.

8.2 Exporting to Excel

To export data from R to an Excel file, you can use the openxlsx package. This package
provides functions to create and write data to Excel files.
# Install and load the openxlsx package
install.packages("openxlsx")
library(openxlsx)

# Export data to an Excel file

write.xlsx(data, file = "output.xlsx")
```

The `write.xlsx()` function writes the data frame to an Excel file specifie
d by the `file` parameter.

8.3 Exporting to other file formats

R provides functions to export data to various other file formats, such as TXT, JSON, XML,
and more. The specific functions and packages required depend on the desired file format.
# Export data to a TXT file
write.table(data, file = "output.txt", sep = "\t", row.names = FALSE)

# Export data to a JSON file

jsonlite::write_json(data, file = "output.json")

# Export data to an XML file

xml2::write_xml(data, file = "output.xml")
```

The `write.table()` function can be used to export data to a TXT file, whil
e other packages like `jsonlite` and `xml2` provide functions specific to J
SON and XML file formats, respectively.
8.4 Customizing export options
Depending on the file format, you can customize various export options. For example, you
can specify the delimiter in a CSV file using the sep parameter, or include additional
parameters to control the output format.
# Export data to a CSV file with a semicolon delimiter
write.csv(data, file = "output.csv", row.names = FALSE, sep = ";")

# Export data to a JSON file with additional options

jsonlite::write_json(data, file = "output.json", pretty = TRUE)
```

By adjusting these export options, you can tailor the output to meet your s
pecific requirements.

8.5 Saving plots and visualizations

In addition to exporting data, you can also save plots and visualizations generated in R to
various file formats using functions like png(), pdf(), or svg(). These functions allow you to
save the graphical output as image files or vector graphics.
# Save a plot as a PNG image
png("plot.png")
plot(data)
dev.off()

# Save a plot as a PDF file

pdf("plot.pdf")
plot(data)
dev.off()
```

The `png()` and `pdf()` functions specify the output file, and the `dev.off
()` function closes the device and saves the plot.
By utilizing these export methods in R, you can save your data and visualizations to different
file formats, making them accessible and shareable with others or for further analysis in
external applications.

9 Customizing the working directory in R

The working directory in R refers to the folder or directory on your computer where R will
read and write files by default. It is the default location for R to search for files, load data, and
save outputs.
9.1 Getting the Current Working Directory
To check the current working directory in R, you can use the getwd() function. It returns the
current working directory as a character string.
# Get the current working directory
getwd()
```

9.2 Setting the Working Directory

You can set a new working directory in R using the setwd() function. This function allows
you to specify the path to the desired directory.
# Set a new working directory
setwd("path/to/new/directory")
```
 Replace "path/to/new/directory" with the actual path to the directory you w
ant to set as the working directory. It's important to note that you need t
o use the appropriate file path syntax based on your operating system (e.g.
, backslashes `\` for Windows, forward slashes `/` for macOS and Linux).

9.3 Relative and Absolute Paths

When specifying the path to a directory, you can use either a relative or an absolute path. A
relative path is defined relative to the current working directory, while an absolute path
specifies the full path from the root directory.
# Set a relative path as the working directory
setwd("data")

# Set an absolute path as the working directory

setwd("/Users/username/Documents/data")
```

In the first example, the "data" folder is located within the current worki
ng directory. In the second example, the absolute path specifies the full p
ath to the "data" folder.

9.4 Choosing the Working Directory in RStudio

In RStudio, you can choose the working directory interactively using the "Session" menu. Go
to "Session" -> "Set Working Directory" -> "Choose Directory" and navigate to the desired
folder.
9.5 Using RStudio Project
RStudio projects provide a convenient way to manage working directories and associated
files. By creating an RStudio project, the working directory is automatically set to the project
directory, and you can easily switch between multiple projects.
To create a new RStudio project, go to "File" -> "New Project" -> "New Directory" or
"Existing Directory". Follow the prompts to set up your project, and RStudio will handle the
working directory for you.
9.6 Directory Navigation Functions
R provides several functions for navigating directories and working with file paths. Some
commonly used functions include list.dirs() to list directories within a
directory, list.files() to list files in a directory, and file.path() to construct platform-
independent file paths.
# List directories within a directory
list.dirs("path/to/directory")

# List files in a directory

list.files("path/to/directory")

# Construct a file path

file.path("path", "to", "file")
```

These functions are useful for exploring and manipulating files and directo
ries within your working directory.
Customizing the working directory allows you to conveniently manage and access files in R.
By setting the appropriate working directory, you can easily read and write files, load data,
and save outputs, enhancing your workflow and efficiency in data analysis and project
management.