Common tasks

Prerequisites: Install the nycflights13 and tidyverse packages

library(tidyverse)
library(nycflights13)

Data from nycflights13

The data comes from the US Bureau of Transportation Statistics, and is documented in ?flights

The tidyverse

The tidyverse is an opinionated collection of R packages designed for data science. All packages share an underlying design philosophy, grammar, and data structures.

Definition of tidy data: (see paper by Hadley Wickham)

• Each variable is a column

• Each observation is a row

• Each type of observational unit is a table

Core tidyverse packages

stringr	working with strings
forcats	working with factors

dplyr basics

Filtering rows

filter(flights, month == 1, day == 1)

Comparisons

jan1 <- filter(flights, month == 1, day == 1)

Logical operators

Flights in November OR December

The following code finds all flights that departed in November OR December:

filter(flights, month == 11 | month == 12)

A useful short-hand is x %in% y. This will select every row where x is one of the values in y. We could use it to rewrite the code above:

nov_dec <- filter(flights, month %in% c(11, 12))

Arrange rows with arrange()

arrange() works similarly to filter() except that instead of selecting rows, it changes their order.

It takes a data frame and a set of column names to order by.

arrange(flights, year, month, day)

Use desc() to re-order by a column in descending order:

arrange(flights, desc(arr_delay))

Select columns with select()

select() allows you to rapidly zoom in on a useful subset using the names of the variables.

# Select columns by name
select(flights, year, month, day)

There are a number of helper functions you can use within select():

starts_with("abc"): matches names that begin with "abc".

ends_with("xyz"): matches names that end with "xyz".

contains("ijk"): matches names that contain "ijk".

Add new variables with mutate()

To add new columns that are functions of existing columns. That is the job of mutate(). mutate() always adds new columns at the end of your dataset.

# create a smaller dataset with less columns
flights_sml <- select(flights, 
  year:day, 
  ends_with("delay"), 
  distance, 
  air_time
)

Example: using mutate()

mutate(flights_sml,
  gain = arr_delay - dep_delay,
  speed = distance / air_time * 60
)

mutate(flights_sml,
  gain = arr_delay - dep_delay,
  hours = air_time / 60,
  gain_per_hour = gain / hours
)

Grouped summaries with summarise()

summarise() collapses a data frame to a single row:

summarise(flights, delay = mean(dep_delay, na.rm = TRUE))

## # A tibble: 1 × 1
##   delay
##   <dbl>
## 1  12.6

The summarise() function is useful when we pair it with group_by().

This way, the analysis can be done for individual groups.

The pipe

Example

The pipe sends the output of the LHS function to the first argument of the RHS function.

# pipe example
sum(1:8) %>%
  sqrt() %>% 
  log()

## [1] 1.791759

# the log of the square root
# of the sum of the elements 
# of the vector [1 ... 8]
log(sqrt(sum(1:8)))

## [1] 1.791759

Example for data exploration

Imagine that we want to explore the relationship between the distance and average delay for each location.

There are three steps to prepare this data:

1. Group flights by destination.

2. Summarize to compute distance, average delay, and number of flights.

3. Filter to remove noisy points and Honolulu airport, which is almost twice as far away as the next closest airport.

Power of the pipe %>% operator

# create dataframe of "delays"
delays <- flights %>% 
  group_by(dest) %>% 
  summarize(
    count = n(),
    dist = mean(distance, na.rm = TRUE),
    delay = mean(arr_delay, na.rm = TRUE)
  ) %>% 
  filter(count > 20, dest != "HNL")

Transformations

flights %>% 
  group_by(year, month, day) %>% 
  summarize(mean = mean(dep_delay, na.rm = TRUE))