G is for group_by

For the letter G, I'd like to introduce a very useful function: group_by. This function lets you group data by one or more variables. By itself, it may not seem very useful, but it's great when you start manipulating and summarizing data. That's because many of the functions applied to data after you used group_by are done groupwise. First, let's demonstrate the effect group_by has on a filter. I'll load my reading dataset and group it by the Fiction flag (so 1 means the book was fiction and 0 means it was non-fiction). What was the longest book I read in each of those categories?

library(tidyverse)

## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --

## <U+2713> ggplot2 3.2.1     <U+2713> purrr   0.3.3
## <U+2713> tibble  2.1.3     <U+2713> dplyr   0.8.3
## <U+2713> tidyr   1.0.0     <U+2713> stringr 1.4.0
## <U+2713> readr   1.3.1     <U+2713> forcats 0.4.0

## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()

reads2019 <- read_csv("~/Downloads/Blogging A to Z/SaraReads2019_allrated.csv", col_names = TRUE)

## Parsed with column specification:
## cols(
##   Title = col_character(),
##   Pages = col_double(),
##   date_started = col_character(),
##   date_read = col_character(),
##   Book.ID = col_double(),
##   Author = col_character(),
##   AdditionalAuthors = col_character(),
##   AverageRating = col_double(),
##   OriginalPublicationYear = col_double(),
##   read_time = col_double(),
##   MyRating = col_double(),
##   Gender = col_double(),
##   Fiction = col_double(),
##   Childrens = col_double(),
##   Fantasy = col_double(),
##   SciFi = col_double(),
##   Mystery = col_double(),
##   SelfHelp = col_double()
## )

reads2019 %>%
  group_by(Fiction) %>%
  filter(Pages == max(Pages)) %>%
  select(Title, Pages, MyRating, Fiction)

## # A tibble: 2 x 4
## # Groups:   Fiction [2]
##   Title           Pages MyRating Fiction
##   <chr>           <dbl>    <dbl>   <dbl>
## 1 How Music Works   345        5       0
## 2 It               1156        4       1

It was my longest book overall, and therefore the longest fiction book. The longest non-fiction book I read was How Music Works, an amazing exploration of music history both generally and personally for the author, David Byrne (from Talking Heads). I picked the book up at an airport bookstore and absolutely loved it.

I can also group by multiple variables, and use that to summarize my data. Let's see what happens when I use two of my genre variables for grouping, then generate a count of the number of books in each group.

reads2019 %>%
  group_by(Childrens, Fantasy) %>%
  summarise(count = n())

## # A tibble: 4 x 3
## # Groups:   Childrens [2]
##   Childrens Fantasy count
##       <dbl>   <dbl> <int>
## 1         0       0    49
## 2         0       1    21
## 3         1       0     1
## 4         1       1    16

Since these genres aren't mutually exclusive, grouping by 2 variables gave me 4 groups: 49 of the books I read last year were neither children's fiction nor fantasy, 21 were fantasy not written for children, 1 was children's fiction but not fantasy, and 16 were children's fantasy. Let's update the code to have it also give me the longest book from each of those categories.

reads2019 %>%
  arrange(desc(Pages)) %>%
  group_by(Childrens, Fantasy) %>%
  summarise(count = n(),
            title = first(Title),
            Pages = first(Pages))

## # A tibble: 4 x 5
## # Groups:   Childrens [2]
##   Childrens Fantasy count title                             Pages
##       <dbl>   <dbl> <int> <chr>                             <dbl>
## 1         0       0    49 The Robber Bride                    528
## 2         0       1    21 It                                 1156
## 3         1       0     1 The Queen's Corgi: On Purpose       181
## 4         1       1    16 The Patchwork Girl of Oz (Oz, #7)   346

Of course It will end up here, since it's the longest book I read in 2019. It's also the longest fantasy not written for children (although it is written about children). The Robber Bride was the longest non-fantasy, non-children's book. The Queen's Corgi was the one non-fantasy children's book I read, and The Pathwork Girl of Oz was the longest children's fantasy.

See you tomorrow when we talk about reading in different file types with the haven package!

F is for filter

For the letter F - filters! Filters are incredibly useful, especially when combined with the main pipe %>%. I frequently use filters along with ggplot functions, to chart a specific subgroup or remove missing cases or outliers. As one example, I could use a filter to chart only fiction books from my reading dataset.

library(tidyverse)

## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --

## <U+2713> ggplot2 3.2.1     <U+2713> purrr   0.3.3
## <U+2713> tibble  2.1.3     <U+2713> dplyr   0.8.3
## <U+2713> tidyr   1.0.0     <U+2713> stringr 1.4.0
## <U+2713> readr   1.3.1     <U+2713> forcats 0.4.0

## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()

reads2019 <- read_csv("~/Downloads/Blogging A to Z/SarasReads2019_allrated.csv", col_names = TRUE)

## Parsed with column specification:
## cols(
##   Title = col_character(),
##   Pages = col_double(),
##   date_started = col_character(),
##   date_read = col_character(),
##   Book.ID = col_double(),
##   Author = col_character(),
##   AdditionalAuthors = col_character(),
##   AverageRating = col_double(),
##   OriginalPublicationYear = col_double(),
##   read_time = col_double(),
##   MyRating = col_double(),
##   Gender = col_double(),
##   Fiction = col_double(),
##   Childrens = col_double(),
##   Fantasy = col_double(),
##   SciFi = col_double(),
##   Mystery = col_double(),
##   SelfHelp = col_double()
## )

reads2019 %>%
  filter(Fiction == 1) %>%
  ggplot(aes(Pages)) +
  geom_histogram() +
  scale_y_continuous(breaks = seq(0,16,1)) +
  scale_x_continuous(breaks = seq(0,1200,100)) +
  ylab("Frequency") +
  theme_classic()

## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

I could also use filters to create a new dataset - perhaps one of my top books I read during 2019.

library(magrittr)

## 
## Attaching package: 'magrittr'

## The following object is masked from 'package:purrr':
## 
##     set_names

## The following object is masked from 'package:tidyr':
## 
##     extract

top_books <- reads2019 %>%
  filter(MyRating == 5)

top_books %$%
  list(Title)

## [[1]]
##  [1] "1Q84"                                                       
##  [2] "Alas, Babylon"                                              
##  [3] "Elevation"                                                  
##  [4] "Guards! Guards! (Discworld, #8; City Watch #1)"             
##  [5] "How Music Works"                                            
##  [6] "Lords and Ladies (Discworld, #14; Witches #4)"              
##  [7] "Moving Pictures (Discworld, #10; Industrial Revolution, #1)"
##  [8] "Redshirts"                                                  
##  [9] "Swarm Theory"                                               
## [10] "The Android's Dream (The Android's Dream #1)"               
## [11] "The Dutch House"                                            
## [12] "The Emerald City of Oz (Oz #6)"                             
## [13] "The End of Mr. Y"                                           
## [14] "The Human Division (Old Man's War, #5)"                     
## [15] "The Last Colony (Old Man's War, #3)"                        
## [16] "The Long Utopia (The Long Earth #4)"                        
## [17] "The Marvelous Land of Oz (Oz, #2)"                          
## [18] "The Miraculous Journey of Edward Tulane"                    
## [19] "The Night Circus"                                           
## [20] "The Patchwork Girl of Oz (Oz, #7)"                          
## [21] "The Patron Saint of Liars"                                  
## [22] "The Wonderful Wizard of Oz (Oz, #1)"                        
## [23] "The Year of the Flood (MaddAddam, #2)"                      
## [24] "Witches Abroad (Discworld, #12; Witches #3)"                
## [25] "Wyrd Sisters (Discworld, #6; Witches #2)"

Or I could create one of the 10 longest books I read:

long_books <- reads2019 %>%
  arrange(desc(Pages)) %>%
  filter(between(row_number(), 1, 10)) %>%
  select(Title, Pages)

library(expss)

## 
## Use 'expss_output_viewer()' to display tables in the RStudio Viewer.
##  To return to the console output, use 'expss_output_default()'.

## 
## Attaching package: 'expss'

## The following objects are masked from 'package:magrittr':
## 
##     and, equals, or

## The following objects are masked from 'package:stringr':
## 
##     fixed, regex

## The following objects are masked from 'package:dplyr':
## 
##     between, compute, contains, first, last, na_if, recode, vars

## The following objects are masked from 'package:purrr':
## 
##     keep, modify, modify_if, transpose

## The following objects are masked from 'package:tidyr':
## 
##     contains, nest

## The following object is masked from 'package:ggplot2':
## 
##     vars

as.etable(long_books, rownames_as_row_labels = FALSE)

Title	Pages
It	1156
1Q84	925
Insomnia	890
The Institute	576
The Robber Bride	528
Life of Pi	460
Cell	449
Cujo	432
The Human Division (Old Man's War, #5)	431
The Year of the Flood (MaddAddam, #2)	431

I can also filter on multiple criteria, with logical operators. To filter on two things, I'd combine them with &. In this example, I'll select the books that took me longer than a week to read and that were at least 400 pages long.

reads2019 %>%
  filter(read_time > 7 & Pages >= 400) %>%
  select(Title, Pages, Author, read_time)

## # A tibble: 2 x 4
##   Title                             Pages Author           read_time
##   <chr>                             <dbl> <chr>                <dbl>
## 1 The Long War (The Long Earth, #2)   419 Pratchett, Terry         8
## 2 The Robber Bride                    528 Atwood, Margaret         9

Lastly, let's filter with "or", so we select cases that meet one of the two criteria. We create or with | . The first criteria is read time less than 1 day (meaning I started and finished the book in the same day). The second criteria are my long reads/long books criteria from above. Since there's two parts to this side of the |, I enclose them in parentheses so the statement is evaluated together across the data:

reads2019 %>%
  filter(read_time < 1 | (read_time > 7 & Pages >= 400)) %>%
  select(Title, Pages, Author, read_time)

## # A tibble: 4 x 4
##   Title                                             Pages Author       read_time
##   <chr>                                             <dbl> <chr>            <dbl>
## 1 Empath: A Complete Guide for Developing Your Gif…   104 Dyer, Judy           0
## 2 The Long War (The Long Earth, #2)                   419 Pratchett, …         8
## 3 The Robber Bride                                    528 Atwood, Mar…         9
## 4 When We Were Orphans                                320 Ishiguro, K…         0

You can read more about logical and arithmetic operators that can be used with filter here.

Tomorrow, we'll discuss the group_by function!

E is for Exposition Pipe

For the letter E, I want to talk about a set of operators provided by tidyverse (specifically the magrittr package) that makes for much prettier, easier-to-read code: pipes. The main pipe %>% pushes the object to the left of it forward into functions on the right, so that instead of coding f(x), it would be x %>% f(). This lets you chain together multiple functions to apply to a single object.

One of these pipes is known as the exposition pipe, and it looks like this: %$%. For many base R functions, there is no data attribute (i.e., data = ...), and so when you specify one or more variables, you have to also specify the data frame (which would be formatted as data$variable). For instance, to run a correlation between two variables, you'd need to write the code as cor(data$variable1, data$variable2).

The exposition pipe exposes the columns of a data frame to the function, so you only need to specify the data frame once. This cleans up your code, while also allowing these functions to be embedded in a larger string of data wrangling and manipulation. Here's how I can use the exposition code in action (but first, I've created a new version of the dataset that has a single Rating variable, which we created previously with the coalesce function, so you'll want to download and use that file going forward):

library(tidyverse)

## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --

## <U+2713> ggplot2 3.2.1     <U+2713> purrr   0.3.3
## <U+2713> tibble  2.1.3     <U+2713> dplyr   0.8.3
## <U+2713> tidyr   1.0.0     <U+2713> stringr 1.4.0
## <U+2713> readr   1.3.1     <U+2713> forcats 0.4.0

## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()

library(magrittr)

## 
## Attaching package: 'magrittr'

## The following object is masked from 'package:purrr':
## 
##     set_names

## The following object is masked from 'package:tidyr':
## 
##     extract

reads2019 <- read_csv("~/Downloads/Blogging A to Z/SarasReads2019_allrated.csv", col_names = TRUE)

## Parsed with column specification:
## cols(
##   Title = col_character(),
##   Pages = col_double(),
##   date_started = col_character(),
##   date_read = col_character(),
##   Book.ID = col_double(),
##   Author = col_character(),
##   AdditionalAuthors = col_character(),
##   AverageRating = col_double(),
##   OriginalPublicationYear = col_double(),
##   read_time = col_double(),
##   MyRating = col_double(),
##   Gender = col_double(),
##   Fiction = col_double(),
##   Childrens = col_double(),
##   Fantasy = col_double(),
##   SciFi = col_double(),
##   Mystery = col_double(),
##   SelfHelp = col_double()
## )

reads2019 %$%
  cor(MyRating, read_time)

## [1] -0.03762191

This is the correlation between my rating of the book and the number of days it took to read it. It's a negative correlation, meaning the more I liked the book, the faster I got through it, which makes sense, but it's also very weak. But what if I wanted to create a variable and then perform a correlation with that new variable? I can do that by combining the main pipe and the exposition pipe:

reads2019 %>%
  mutate(DifRating = MyRating - AverageRating) %$%
  cor(DifRating, read_time)

## [1] 0.007745212

The variable I created is the difference between my rating and the average rating from all Goodreads users. The correlation is basically 0, so there seems to be no relationship between how much more highly I rate a book (compared to others) and how quickly I finish it.

Tomorrow, we'll talk about filters!

D is for dummy_cols

For the letter D, I'm going to talk about the dummy_cols functions, which isn't actually part of the tidyverse, but hey: my posts, my rules. This function is incredibly useful for creating dummy variables, which are used in a variety of ways, including multiple regression with categorical variables. When conducting linear regression, the assumption is that both the predictor and outcome variables are numeric. To include categorical variables, you need to convert them to numeric variables. If they aren't strictly continuous, then you instead create dummy variables to represent the different categories. If I had three levels on a categorical variable, I'd need 2 dummy variables: one to delineate category 1 from the other 2, and another to delineate category 2 (with the third category being represented by 0s on the other two variables).

There are, of course, other uses for dummy variables. For instance, at work, I was examining unique users of our testing system by time of day. Our system creates a row for every action by the user, with a time stamp. If I simply generated counts of these rows during spans of time, I would get a count of actions per hour by users (clicks, highlights, etc.), rather than individual users logged in during a given hour. So I created dummy codes by hour of day, then aggregated by unique user identifier. This was how I could generate accurate counts of how many users were online during a given hour.

To apply this procedure to the reading dataset, I used the dummy_cols function to create dummy variables (or flags) for genre. I created a long-form dataset of the top genres for each title, which you can download here. For simplicity, this file only contains Book.ID, title, and genre (with a separate entry for each genre, so some books have a single row, for one genre, and others have multiple rows, to reflect multiple genres).

library(tidyverse)

## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --

## <U+2713> ggplot2 3.2.1     <U+2713> purrr   0.3.3
## <U+2713> tibble  2.1.3     <U+2713> dplyr   0.8.3
## <U+2713> tidyr   1.0.0     <U+2713> stringr 1.4.0
## <U+2713> readr   1.3.1     <U+2713> forcats 0.4.0

## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()

longreads2019 <- read_csv("~/Downloads/Blogging A to Z/reads2019_long.csv")

## Parsed with column specification:
## cols(
##   Book.ID = col_double(),
##   Title = col_character(),
##   genre = col_character()
## )

I can use the dummy_cols functions to create the genres flags, that I can aggregate down and merge into the reads2019 file (I've created a version without genre flags, available here). For this function, you'll need the fastDummies package (so add install.packages("fastDummies") before the rest of the code). Also, since the number of dummy code variables typically are equal to the number of categories minus 1, the function automatically removes the first dummy variable from the final file. Since I'm using these as flags rather than dummy variables, I want to overide that default, which I do with remove_first_dummy = FALSE.

library(fastDummies)

genres <- longreads2019 %>%
  dummy_cols(select_columns = "genre", remove_first_dummy = FALSE)

genres <- genres %>%
  group_by(Book.ID) %>%
  summarise(Fiction = max(genre_Fiction),
            Childrens = max(genre_Childrens),
            Fantasy = max(genre_Fantasy),
            SciFi = max(genre_SciFi),
            Mystery = max(genre_Mystery),
            SelfHelp = max(genre_SelfHelp))

reads2019 <- read_csv("~/Downloads/Blogging A to Z/ReadsNoGenre.csv",
                      col_names = TRUE)

## Parsed with column specification:
## cols(
##   Title = col_character(),
##   Pages = col_double(),
##   date_started = col_character(),
##   date_read = col_character(),
##   Book.ID = col_double(),
##   Author = col_character(),
##   AdditionalAuthors = col_character(),
##   AverageRating = col_double(),
##   OriginalPublicationYear = col_double(),
##   read_time = col_double(),
##   MyRating = col_double(),
##   Gender = col_double(),
##   NewRating = col_double(),
##   FinalRating = col_double()
## )

reads2019 <- reads2019 %>%
  left_join(genres, by = "Book.ID")

I know I've sprinkled in other tidyverse functions in these posts, such as group_by and summarise. Don't worry! I'll post more about those functions in this series - stay tuned!

C is for coalesce

For the letter C, we'll talk about the coalesce function. If you're familiar with SQL, you may have seen this function before. It combines two or more variables into a single column, and is a way to deal with missing data. When you give it a list of variables, it selects the first non-missing value it finds. Because of that, order of entry of those variables is important.

I recently used this function at work to get last performance on an item. That is, diplomates in our program take spaced assessments, with repetition of items (same content, but slightly different item wording) that they answered incorrectly. If they got the initial item correct, they may never see a related item again (although over time they will), but if they were incorrect, they may get related items once or more times to see if they're learning. So we have some people who will have a missing value for the repeated item and others who will have correct/incorrect. The coalesce function allowed me to get their final performance on item content by telling it to first check the repeated item column and then, if that's missing, to look at the initial item column.

How can we use the coalesce function for the reading dataset? I noticed that for 17 books in the dataset, I forgot to supply a rating, so they're showing up as 0. Let's select those titles, and I can figure out how I'd like to rate them.

library(tidyverse)

## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --

## <U+2713> ggplot2 3.2.1     <U+2713> purrr   0.3.3
## <U+2713> tibble  2.1.3     <U+2713> dplyr   0.8.3
## <U+2713> tidyr   1.0.0     <U+2713> stringr 1.4.0
## <U+2713> readr   1.3.1     <U+2713> forcats 0.4.0

## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()

reads2019 <- read_csv("~/Downloads/Blogging A to Z/SarasReads2019.csv", col_names = TRUE)

## Parsed with column specification:
## cols(
##   Title = col_character(),
##   Pages = col_double(),
##   date_started = col_character(),
##   date_read = col_character(),
##   Book.ID = col_double(),
##   Author = col_character(),
##   AdditionalAuthors = col_character(),
##   AverageRating = col_double(),
##   OriginalPublicationYear = col_double(),
##   read_time = col_double(),
##   MyRating = col_double(),
##   Gender = col_double(),
##   Fiction = col_double(),
##   Childrens = col_double(),
##   Fantasy = col_double(),
##   SciFi = col_double(),
##   Mystery = col_double(),
##   SelfHelp = col_double()
## )

unrated <- reads2019 %>%
  filter(MyRating == 0)

list(unrated$Title)

## [[1]]
##  [1] "Bird Box (Bird Box, #1)"                            
##  [2] "Elevation"                                          
##  [3] "Glinda of Oz (Oz, #14)"                             
##  [4] "It"                                                 
##  [5] "Just Evil (Evil Secrets Trilogy, #1)"               
##  [6] "Oryx and Crake (MaddAddam, #1)"                     
##  [7] "Precarious Pasta (Cozy Corgi Mysteries, #14)"       
##  [8] "Redshirts"                                          
##  [9] "Rinkitink in Oz (Oz, #10)"                          
## [10] "Summerdale"                                         
## [11] "The 5 Love Languages: The Secret to Love That Lasts"
## [12] "The Long Mars (The Long Earth, #3)"                 
## [13] "The Magic of Oz (Oz, #13)"                          
## [14] "The Marvelous Land of Oz (Oz, #2)"                  
## [15] "The Scarecrow of Oz (Oz, #9)"                       
## [16] "The Tin Woodman of Oz (Oz, #12)"                    
## [17] "Tik-Tok of Oz (Oz, #8)"

So now I want to generate a variable with my new ratings, and merge it into my reads dataset.

unrated$NewRating <- c(3, 5, 4, 4, 2, 4, 4, 5, 4, 3, 3, 4, 4, 5, 3, 3, 4)

unrated <- unrated %>%
  select(Book.ID, NewRating)

reads2019 <- reads2019 %>%
  left_join(unrated, by = "Book.ID")

Finally, we'll create our final ratings variable by coalescing the two ratings variables. I want it to first look at the new rating column, so it can capture those updates. For rows missing that variable (because I already rated them), it will look next at the original rating variable.

reads2019 <- reads2019 %>%
  mutate(FinalRating = coalesce(NewRating, MyRating))

table(reads2019$FinalRating)

## 
##  2  3  4  5 
##  2  9 51 25

No more 0s!

B is for bind_rows

Moving on to the letter B, today we'll talk about merging datasets that contain the same variables but add new cases. This is easily done with bind_rows. Let's say I realized I forgot to log some of the books I read last year, and I wanted to merge those in to my existing dataset. I selected a handful of books from my to-read list, generated some read time and rating data, and saved the results in a csv file (which you can find here). Now I want to load my existing dataset and the new one:

library(tidyverse)

## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --

## <U+2713> ggplot2 3.2.1     <U+2713> purrr   0.3.3
## <U+2713> tibble  2.1.3     <U+2713> dplyr   0.8.3
## <U+2713> tidyr   1.0.0     <U+2713> stringr 1.4.0
## <U+2713> readr   1.3.1     <U+2713> forcats 0.4.0

## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()

reads2019 <- read_csv("~/Downloads/Blogging A to Z/SarasReads2019.csv", col_names = TRUE)

## Parsed with column specification:
## cols(
##   Title = col_character(),
##   Pages = col_double(),
##   date_started = col_character(),
##   date_read = col_character(),
##   Book.ID = col_double(),
##   Author = col_character(),
##   AdditionalAuthors = col_character(),
##   AverageRating = col_double(),
##   OriginalPublicationYear = col_double(),
##   read_time = col_double(),
##   MyRating = col_double(),
##   Gender = col_double(),
##   Fiction = col_double(),
##   Childrens = col_double(),
##   Fantasy = col_double(),
##   SciFi = col_double(),
##   Mystery = col_double(),
##   SelfHelp = col_double()
## )

addreads <- read_csv("~/Downloads/Blogging A to Z/SarasAdds.csv")

## Parsed with column specification:
## cols(
##   Title = col_character(),
##   Pages = col_double(),
##   date_started = col_character(),
##   date_read = col_character(),
##   Book.ID = col_double(),
##   Author = col_character(),
##   AdditionalAuthors = col_character(),
##   AverageRating = col_double(),
##   OriginalPublicationYear = col_double(),
##   read_time = col_double(),
##   MyRating = col_double(),
##   Gender = col_double(),
##   Fiction = col_double(),
##   Childrens = col_double(),
##   Fantasy = col_double(),
##   SciFi = col_double(),
##   Mystery = col_double(),
##   SelfHelp = col_double()
## )

Now we just bind the two datasets together:

reads2019 <- reads2019 %>%
  bind_rows(addreads)

Did these additions change the ordering by page length?

reads2019 <- reads2019 %>%
  arrange(desc(Pages), Author)

head(reads2019)

## # A tibble: 6 x 18
##   Title Pages date_started date_read Book.ID Author AdditionalAutho…
##   <chr> <dbl> <chr>        <chr>       <dbl> <chr>  <chr>           
## 1 The …  1216 6/12/2019    6/18/2019  3.30e1 Tolki… <NA>            
## 2 The …  1181 6/12/2019    6/17/2019  1.86e7 Atwoo… <NA>            
## 3 It     1156 8/14/2019    8/21/2019  2.79e7 King,… <NA>            
## 4 1Q84    925 9/3/2019     9/10/2019  1.04e7 Murak… Jay Rubin, Phil…
## 5 Inso…   890 8/10/2019    8/13/2019  1.06e4 King,… Bettina Blanch …
## 6 The …   592 8/18/2019    8/23/2019  1.16e4 King,… <NA>            
## # … with 11 more variables: AverageRating <dbl>, OriginalPublicationYear <dbl>,
## #   read_time <dbl>, MyRating <dbl>, Gender <dbl>, Fiction <dbl>,
## #   Childrens <dbl>, Fantasy <dbl>, SciFi <dbl>, Mystery <dbl>, SelfHelp <dbl>

It did! The longest book is now The Lord of the Rings, at 1216 pages, and number two is The MaddAddam Trilogy, 1181 pages.

This is a pretty easy trick. Later on in this series, we'll talk about combining datasets that share cases but add new variables - joins - which is one of the times the tidy data mindset becomes very important.

A is for arrange

The arrange function allows you to sort a dataset by one or more variable, either ascending or descending. This function is especially helpful if you plan on aggregating your data with summarize (which, we'll get to later), so you can select specific rows in that command.

It's similar to the Excel complex sort, where the order of entry determines which variable is sorted first, second, ..., last. For this example, I'll use data I put together about my reading for 2019 (which you can download here, or use your own datafile of choice). First up, as with all of these posts going forward, we'll want to load tidyverse. (If you haven't installed it yet, just add install.packages("tidyverse") before the rest of the code.)

library(tidyverse)

## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --

## <U+2713> ggplot2 3.2.1     <U+2713> purrr   0.3.3
## <U+2713> tibble  2.1.3     <U+2713> dplyr   0.8.3
## <U+2713> tidyr   1.0.0     <U+2713> stringr 1.4.0
## <U+2713> readr   1.3.1     <U+2713> forcats 0.4.0

## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()

reads2019 <- read_csv("~/Downloads/SarasReads2019.csv", col_names = TRUE)

## Parsed with column specification:
## cols(
##   Title = col_character(),
##   Pages = col_double(),
##   date_started = col_date(format = ""),
##   date_read = col_date(format = ""),
##   Book.ID = col_double(),
##   Author = col_character(),
##   AdditionalAuthors = col_character(),
##   AverageRating = col_double(),
##   OriginalPublicationYear = col_double(),
##   read_time = col_double(),
##   MyRating = col_double(),
##   Gender = col_double(),
##   Fiction = col_double(),
##   Childrens = col_double(),
##   Fantasy = col_double(),
##   SciFi = col_double(),
##   Mystery = col_double(),
##   SelfHelp = col_double()
## )

head(reads2019)

## # A tibble: 6 x 18
##   Title Page… date_started date_read  Book.ID Author
##   <chr> <dbl> <date>       <date>       <dbl> <chr>       
## 1 1Q84  925 2019-09-03   2019-09-10  1.04e7 Murakami, H…
## 2 A Diso… 256 2019-08-21   2019-08-22  5.46e4 Kalfus, Ken 
## 3 Alas, … 323 2019-12-21   2019-12-23  3.82e4 Frank, Pat  
## 4 Artemis 305 2019-04-08   2019-04-11  3.49e7 Weir, Andy  
## 5 Bird B… 262 2019-02-07   2019-02-13  1.85e7 Malerman, J…
## 6 Bounda… 314 2019-04-23   2019-04-26  9.44e5 Cloud, Henry
## # … with 12 more variables: AdditionalAuthors <chr>, AverageRating <dbl>,
## #   OriginalPublicationYear <dbl>, read_time <dbl>, MyRating <dbl>,
## #   Gender <dbl>, Fiction <dbl>, Childrens <dbl>, Fantasy <dbl>, SciFi <dbl>,
## #   Mystery <dbl>, SelfHelp <dbl>

Let's sort the data first by page length (longest to shortest), then by author (A to Z).

reads2019 <- reads2019 %>%
  arrange(desc(Pages), Author)

head(reads2019)

## # A tibble: 6 x 18
##   Title Page… date_started date_read  Book.ID Author
##   <chr> <dbl> <date>       <date>       <dbl> <chr>       
## 1 It    1156 2019-08-14   2019-08-21  2.79e7 King, Steph…
## 2 1Q84  925 2019-09-03   2019-09-10  1.04e7 Murakami, H…
## 3 Insomn… 890 2019-08-10   2019-08-13  1.06e4 King, Steph…
## 4 The In… 576 2019-12-06   2019-12-11  4.38e7 King, Steph…
## 5 The Ro… 528 2019-04-11   2019-04-20  1.76e4 Atwood, Mar…
## 6 Life o… 460 2019-07-13   2019-07-16  4.21e3 Martel, Yann
## # … with 12 more variables: AdditionalAuthors <chr>, AverageRating <dbl>,
## #   OriginalPublicationYear <dbl>, read_time <dbl>, MyRating <dbl>,
## #   Gender <dbl>, Fiction <dbl>, Childrens <dbl>, Fantasy <dbl>, SciFi <dbl>,
## #   Mystery <dbl>, SelfHelp <dbl>

The longest book I read was It by Stephen King, which was 1,156 pages, followed by 1Q84 by Haruki Murakami, 925 pages.

Arrange becomes very helpful if you're planning on displaying a small dataset, summarizing data, or for charting. We'll get to all that later.