tag:blogger.com,1999:blog-45948329393344102202024-03-17T22:03:32.760-05:00Deeply TrivialUnknownnoreply@blogger.comBlogger1018125tag:blogger.com,1999:blog-4594832939334410220.post-15756995332809919402021-04-01T20:35:00.004-05:002021-04-01T20:35:56.404-05:00Noise in the Middle: Movie Review<p>I've been on a horror movie kick for a while (as I've said before particularly <a href="https://www.deeplytrivial.com/2011/09/keep-telling-yourself-its-only-movie.html">here</a> and <a href="https://www.deeplytrivial.com/2011/09/keep-telling-yourself-its-only-movie_21.html">here</a>, I love a good horror movie, and I also think that after the last year+ of insanity, nothing really scares me anymore, or at least fiction doesn't scare me more than reality). I've been checking out every horror movie I can find on my various streaming services and, well, I've definitely watched some crappy ones. Maybe I'll blog about them sometime.</p><p>This evening, I watched <i>Noise in the Middle</i>, the story of a grieving widow and his daughter with severe autism, who seek out an experimental treatment (what appears to be transcranial magnetic stimulation therapy or something like it). What he doesn't realize is that the house he rents is haunted by an occult-loving sadist and the ghosts of the children from his poor house/orphanage that he bound to be trapped in the house after their death. Or something. It's not completely clear but it apparently involved branding the children with an infinity symbol and also the children killing him and themselves with a fire. Or something. </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhUBf4gCmd1IfBD-OtZadz-if13o2ooAbD1NYui8VhtsHkkYqsb8YsjMG7ErpLiSdWw4h4-ny9ixapKf0KKjByr7W9kGLQAY1A6sHwIeiM3cmdAbmC8Ms7JsoCXvGy5FaDW6zmkSk4ZT9h/s1024/noise_in_the_middle.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1024" data-original-width="768" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhUBf4gCmd1IfBD-OtZadz-if13o2ooAbD1NYui8VhtsHkkYqsb8YsjMG7ErpLiSdWw4h4-ny9ixapKf0KKjByr7W9kGLQAY1A6sHwIeiM3cmdAbmC8Ms7JsoCXvGy5FaDW6zmkSk4ZT9h/s320/noise_in_the_middle.jpg" /></a></div><p>The concept was promising - although I find the "kid with autism has special powers (in this case, is a conduit and can see spirits)" concept to be problematic, just like I found the "woman with dementia is actually possessed" concept to be problematic in <i><a href="https://www.deeplytrivial.com/2015/09/totally-superfluous-movie-review-taking.html">The Taking of Deborah Logan</a> </i>- and the movie started off great. We established the background, got some ominous shots and glowing eyes in dark rooms. We also saw some really interesting symbolic imagery after Emmy's (the daughter) treatments with TMS, very Ring-video type images, which could have been used more fully in connection with the haunted house and the concept but sadly was not. We even had the "person randomly finds occult shop/enthusiast who believes the main character and helps them" trope used for more humorous and uplifting effect. </p><p>In the middle, things began to drag and become more convoluted, which I thought might be used to tie in the symbolic imagery from Emmy's sessions, but sadly was not. The end was just a big old mess. It felt like the writer had a great idea, spent lots of time on the beginning, lost steam in the middle, and then had to just finish the damn thing by the end. The movie toyed with so many horror concepts (haunted house starts to bring out the darkness in/infect the father, like <i>The Shining</i>; seemingly random images have more concrete meaning for the mystery, like <i>Ringu/The Ring</i>; grief manifested as a spirit or entity, like <i>The Babadook)</i> but never really fully committed to any of it.</p><p>Overall, I'd say don't bother with this one. The beginning made me have high expectations that this movie would be good/meaningful/even a little scary, but I ended up with "WTF did I just watch and why?". </p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-54502002312900881672021-02-07T22:58:00.004-06:002021-02-07T23:02:42.761-06:00Super Bowl Reactions<p> I watched the Super Bowl tonight rooting hard for my Chiefs - I was even wearing a Mahomes jersey (full disclosure: it was supposed to be a Kelce jersey, since he's my favorite player, but due to a royal f-up by the post office, that jersey never arrived, so I was able to get a quick backup Mahomes, my second favorite player, jersey). I was disheartened that my Chiefs lost, but am happy for the Buccs to make an amazing comeback as a franchise (except you, Tom Brady, I still don't like you).</p><p>So my reactions:</p><p><i>America the Beautiful </i>by H.E.R., the National Anthem by Eric Church and Jazmine Sullivan, and the poem by Amanda Gorman were wonderful. Honoring our frontline workers and the message from our President and First Lady - beautiful. Plus the first female ref at the Super Bowl - all the feels.</p><p>The penalties were a bit ridiculous, and mostly being called on the Chiefs. It's ballsy to call pass interference on an uncatchable pass. I understand that many penalty calls in football are based on what we scientists call the counterfactual - what would have happened if a condition (such as, a defensive player pushing a receiver out of the way) was not present - but when a call equals free yards or another try at a down, they need to be used thoughtfully. The penalty calls felt very one-sided. Yes, the Buccs finally got their own "unsportsmanlike conduct" call as well as a much-needed "roughing the passer" (but it took 3 guys hitting Mahomes in much the same whiplash way that caused a concussion 2 games ago). They say homefield advantage doesn't exist in playoff games. I beg to differ.</p><p>I'm surprised at the hate I'm seeing about the Halftime Show by The Weeknd. I went into halftime knowing a couple songs by him, and finished it as a fan. We're used to these blow-out halftime shows with 3 big-name artists plus 10 high schools worth of marching bands and drill teams on the field, but in COVID-land, that's just not possible. Instead, we got an artist who was able to showcase choral and dancing talent while still respecting social distance and safety. The dancers wrapped in face bandages for "I Can't Feel My Face" was super clever - guys, those were face masks! (NOT JOCK STRAPS, as some have joked.) They were able to have dancers in close quarters wearing face masks in a way that made sense with the song. In fact, they looked so little like face masks that... see jock strap comment. I was super impressed!</p><p>In the second half, we saw a bit of the old Chiefs, but sadly not enough to score a single point. The Buccs' defensive line was just too good - I mean, they ran a blitz on every f***ing play, and our offensive line couldn't hold them back long enough to give Mahomes as much time in the pocket as he's used to. This is something to work on for next season. Mahomes is an amazing quarterback but he's used to hanging in the pocket long enough to survey the field, pick his receiver, and pass; let's work on decreasing the time he needs in the pocket. And let's work on an offensive line that can predict how the blitz is going to work and knock those guys down. Yeah, a team that blitzes on every defensive play is unusual, but as we saw tonight, IT HAPPENS! Practice defusing a blitz from every angle.</p><p>Also, WHY DIDN'T YOU SHOW US AN INSTANT REPLAY ON THE RANDOM FAN ON THE FIELD?! I wanted to see that again/closer up.</p><p>Overall, I'm sad the Chiefs lost and annoyed at the one-sided-ness/overeagerness of the penalties. I enjoyed the game, the commercials (ALL the celebs came out for those, including a Wayne's World call-back with Cardi B???!), and the performances. I'm happy for the Buccs and hopeful for the Chief's next season (I mean, winning Super Bowl last year plus being AFC Champs again this year is nothing to sneeze at). And okay, Tom Brady proved that a quarterback can still be good and (pretty much automatically, because we always honor quarterbacks and ignore the other positions - like how about TE Gronkowski?) be Super Bowl MVP at 43. You're on top, dude; how about you retire?</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8MsJiP-vXY5ePjQ4EaaxGwFtAhyfMWWEBUTq6E4ylpchrIMr49uVDTgLlNJBRIEybMc4qoel9vM3q0T7lL8-rq6ur2nogR4weIs0kawo3FFT7NMBy2Q2SHYVQocT5EzI3_ManEMAKg0m4/s2048/AP21038823597273.jpg.webp" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1152" data-original-width="2048" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8MsJiP-vXY5ePjQ4EaaxGwFtAhyfMWWEBUTq6E4ylpchrIMr49uVDTgLlNJBRIEybMc4qoel9vM3q0T7lL8-rq6ur2nogR4weIs0kawo3FFT7NMBy2Q2SHYVQocT5EzI3_ManEMAKg0m4/s320/AP21038823597273.jpg.webp" width="320" /></a></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-37741767396956376292021-01-04T15:10:00.000-06:002021-01-04T15:10:19.694-06:00My Dark Vanessa: Book Review<p>Content Warning: sexual abuse, child abuse, and rape</p><p>Earlier today, I finished <i>My Dark Vanessa</i>, a debut novel by Kate Elizabeth Russell. The book is told by Vanessa Wye, a young woman who was abused by her boarding school English teacher starting when she was 15. The book spans 17 years, jumping between Vanessa's youth and adulthood. Before I get into my (slightly spoiler-y) review, I want to say: I loved this book, and I also have no desire to ever read it again.</p><p>As you can imagine from the title and brief synopsis, this a difficult book, as we hear everything that happened in the mind of a young woman who was gaslighted into believing she had all the power in situations when she had almost none and at the same time, that she had no power in situations where she could do something to stop the abuse. It's a deep and disturbing dive into the way an older man selects and grooms his victim, changing her thinking and behavior for decades, and convincing her that she's a willing participant, even when she describes very clear dissociation (being outside of herself) during the episodes of abuse - a reaction often seen in victims of child abuse.</p><p>The book also digs into two really key issues, that I haven't often seen explored or explored this well: 1. The narrative that women have "feminine gifts" that allow them to have power over a man, and make these men do things they wouldn't otherwise do. And 2. That coming forward is the only responsible thing a victim of such abuse can do.</p><p>The first issue (the "power" of femininity to take away men's agency) is such a pervasive part of rape culture. But this book also explores how this narrative has been romanticized to even apply to situations of a very young girl and a much older man, in stories like <i>Lolita, American Beauty, </i>and <i>Pretty Baby</i><i>. </i>It is this romanticization and narrative that makes Vanessa continue her relationship with her abuser, Jacob Strane, even when it actively hurts her. He convinces her that he has so much more to lose than she does if their "love" becomes public, that he cannot help himself, that she has the power in the situation to consent or decline (even when he ignores her requests for him to stop and/or fails to ask for consent for very extreme sexual acts), and that, most of all, she is special because of this power she has. For a lonely girl, away from home for the first time, it's so easy to see how he selects and grooms her. But perhaps one of the most frustrating things is, even as I was reading and feeling what Vanessa feels, the descriptions and behaviors were so clear, I would shout at her as I read that there's some textbook-level gaslighting going on. It's why this is such a good book - that the author can give us those really clear cues while still telling the book in first-person, and avoid the "unreliable narrator" trope - and also one I hope to never read again.</p><p>This narrative of feminine wiles is perhaps ones of biggest issues we need to contend with if we want to do away with rape culture for good. It's a narrative that, on its surface, appears to assign all the power to the woman and none to her rapist or abuser, when at its core, it instead makes the woman powerless to stop (and deserving of) whatever harm is done to her. It's also a narrative that can be so easily spun as a positive thing when it is actually toxic and harmful. </p><p>The second issue is a bit more ambiguous, at least for me, because before I read this book, I would have agreed with this second statement, that victims <i>must </i>come forward so that the abuser can be brought to justice and that others can be protected. I believed this even as a person who did not bring my own rapist to justice, something I was very ashamed of about myself. But this book made me realize just how tricky this issue is. </p><p>At a surface level, it seems like a conflict between the needs of the individual and the needs of many, and from a philosophical standpoint, the needs of the many should outweigh the needs of the individual. But framing it in such a way takes away the individual's autonomy, a major issue considering that the abuse/rape was all about taking away one individual's autonomy. And victims already feel a great deal of guilt and self-blame for the event; they don't need the guilt of believing they failed others, or that they are in some way responsible for the reprehensible actions of another. </p><p>Framing it as needs of the individual vs. needs of the many oversimplifies exactly what the needs of the individuals are (privacy, self-care, fear of reprisals, and so on), while also making that individual an accomplice in how another person's actions affects the many. In the case of adults in positions of power abusing the people they should be protecting, no victim should ever be to blame; this is on the system that put (and often helps to keep) that person in power, and on all of us, for the ways (big and small) that we may contribute to these power dynamics and rape culture.</p><p>This book was very triggering for me (even though my personal experiences do not resemble Vanessa), and I'm still working through the emotions it's brought up. I was reminded of a book I read in college, <i>Bastard Out of Carolina</i>, which also details years of sexual abuse of a child. When I finished that book, I threw it against the wall. Fortunately this book didn't elicit that reaction, but I didn't have a super-positive reaction to the ending either. </p><p>I'm still glad I read it, though I probably wouldn't recommend it to anyone who might also be triggered, especially if they haven't been able to work through their own trauma through therapy or treatment. And I'll definitely keep an eye out for future books by Kate Elizabeth Russell.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHvUxhbYbVT_uxhCTRaQKNUwzo0bMQ3FLYdEAoWAQLL81IevtJbhMNBBocQq9juKl0MxZsVY3zZsZ3OzcdzdAQehfzSZRMCPp8H9D0dFnB4ihfIZ13CC2xd7u1mKP1kjN9wTqntBg91Ks6/s300/MyDarkVanessa_200x300.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="300" data-original-width="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHvUxhbYbVT_uxhCTRaQKNUwzo0bMQ3FLYdEAoWAQLL81IevtJbhMNBBocQq9juKl0MxZsVY3zZsZ3OzcdzdAQehfzSZRMCPp8H9D0dFnB4ihfIZ13CC2xd7u1mKP1kjN9wTqntBg91Ks6/s0/MyDarkVanessa_200x300.jpg" /></a></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-43921467922593004152020-12-19T11:11:00.003-06:002020-12-19T11:11:16.295-06:00Some Music for Your Holidays<div>Hey everyone,</div><div><br /></div><div>One thing I've been doing during the pandemic is making music on my own. For our holiday season, I dropped my very first album: Winter Delights. You can read about the album and download tracks <a href="https://www.deeplytrivial.com/p/winterdelights.html">here</a> or <a href="https://soundcloud.com/user-284955725/sets/winter-delights">stream me on Soundcloud</a>. I'm working on more arrangements (and upgraded my audio recording equipment) so I'm hoping to drop a full album early in the New Year!</div><div><br /></div><div>And to give you a little extra something, here's a selection of performances from my choir's annual cabaret benefit, Apollo After Hours:</div><div style="text-align: center;"><br /></div><div style="text-align: center;"><iframe frameborder="0" height="360" src="https://youtube.com/embed/lO_Nfcg3ZK8?list=PLJ2LMyCrmBAUD2aeYK-R45rgEV10Qx8DL" width="480"></iframe></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-82162146101520611022020-12-13T13:02:00.004-06:002020-12-13T13:05:15.051-06:00A Follow-Up on Yesterday's Sexist Nonsense<p> Unsurprisingly, I'm not the only one who found Joseph Epstein's op-ed enraging. I give you this <a href="https://whineandsparkle.com/2020/12/13/dear-joseph-epstein-ba-your-time-is-done/">delicious takedown from Amanda Kohlhofer</a>.</p>
<blockquote>A privileged white man with no post-grad education telling a woman with a doctorate not to use her credentials. How very original of you, kiddo.
<br /><br />
To that end, let’s list Dr. Biden’s accomplishments:
<ul>
<li>She earned a Bachelor of Arts in English from the University of Delaware in 1975.</li>
<li>She earned a Master of Education, with a specialty in reading, from West Chester State College in 1981.</li>
<li>She earned a Master of Arts in Education from Villanova University in 1987.</li>
<li>She earned a Doctor of Education (Ed.D) in educational leadership from the University of Delaware in 2007</li></ul>
She accomplished all of this over the span of 32 years, all while becoming a wife, raising children, teaching at many different levels, running a non-profit, and accompanying her husband through multiple political campaigns. (And, who wants to tell him that not only has she earned all of these degrees, but she has also, in fact, delivered a child?)</blockquote>
Just as I did, Kohlhofer suspects this piece would never have been written if Jill Biden were a man. And even though Epstein's blatant sexism is very obviously jealousy over a woman who is more educated, there are definitely people who casually drop the Dr. (or refuse to even recognize that the title <i>could</i> be Dr.) among women more than men.<div><br /></div><div>In 2011, I earned a PhD in Social Psychology. I worked for many years as a health services researcher in the Department of Veterans Affairs, where I regularly worked with PhDs, MDs, and some of those crazy smart people with both. We all called each other by first name. (Except for colleagues who had just earned their doctorate - we called them Dr. at every opportunity until they got sick of it and begged us to go back to first name. Why? Because earning a doctorate is a freaking amazing achievement!) In college and grad school, we all called each other by first name. Academia or medicine is not what Kiddo Joe envisions of a bunch of people calling each other Dr. It was all pretty casual.</div><div><br /></div><div>BUT there are times when that title should be used, such as when introducing a panel of presenters at a conference. And it was very telling how the moderators would often introduce the men as Dr. So-and-So and the women by their first name. It's telling the number of times people have asked me if my title is Mrs. or Ms. in some of these types of settings. It's telling that when I worked at a hospital, people would immediately say, "Oh, you must be a nurse." Why not a doctor? (And even more interesting is when I was married, people would ask my husband what he did for a living but would often ask me if I work.)</div><div><br /></div><div>Women, either with or without higher degrees, constantly have to work harder to prove themselves. Gatekeeping is alive and well, not just in gamers and sports fans communities, testing women to see if they're legit, but in pretty much any field. I've interacted with fellow psychometrician and data scientists who I'm sure would prefer to call me "Kiddo" instead of Dr., or who waste valuable meeting time explaining core concepts "for Sara's benefit." I once had a psychometrician describe a concept and then urge me to read the chapter on this topic in the recent edition of the Institute of Credentialing Excellence Handbook. I was second author of that chapter. </div><div><br /></div><div>And as Epstein demonstrates, gatekeeping doesn't even have to come from someone with the same background or credentials. It can be some dude with a BA writing in the WSJ.</div><div><br /></div><div>Guys, women are exhausted with this nonsense. When interacting with a woman in a professional or academic environment, be aware of those little microaggressions, or the things you may be doing that make her have to work that much harder to be believed or respected. Introduce people with their titles. Assume women know about something unless they say otherwise. Stop wasting everyone's time and energy. And stop telling us to hang up our titles. </div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-37573640421611017242020-12-12T14:42:00.005-06:002020-12-12T16:38:49.554-06:00Sexist Nonsense in the Wall Street Journal<p> I really wish this were satire, but <a href="https://www.wsj.com/articles/is-there-a-doctor-in-the-white-house-not-if-you-need-an-m-d-11607727380">Joseph Epstein's recent opinion piece in the Wall Street Journal</a> is, sadly, a completely earnest bit of mansplaining and suspicion of the intellectual elite:</p>
<blockquote>Madame First Lady—Mrs. Biden—Jill—kiddo: a bit of advice on what may seem like a small but I think is a not unimportant matter. Any chance you might drop the “Dr.” before your name? “Dr. Jill Biden ” sounds and feels fraudulent, not to say a touch comic. Your degree is, I believe, an Ed.D., a doctor of education, earned at the University of Delaware through a dissertation with the unpromising title “Student Retention at the Community College Level: Meeting Students’ Needs.” A wise man once said that no one should call himself “Dr.” unless he has delivered a child. Think about it, Dr. Jill, and forthwith drop the doc.</blockquote>
<p>Epstein goes on to explain that he holds no higher degrees, other than an honorary doctorate. He talks of the hilarity of people referring to him by the title Dr. Yes, it is hilarious, because honorary doctorates are merely a beefed up way of thanking someone for speaking at a university, not recognition following years of hard work to demonstrate that one has earned a title that allows that person to be considered an expert. You see, that's what doctor means - expert. An M.D. is an expert in medicine, a person with a PhD is an expert in the subject of that PhD, and so on. Epstein's honorary doctorate is really more like the prize in a box of cereal. Yeah, he had to do some work for it, but nowhere near on par with the work Dr. Jill Biden did for hers.</p>
<p>Epstein also laments that doctoral requirements have gotten lax in recent years, which is rich coming from someone who has never attempted to earn a doctorate.</p>
<blockquote>Getting a doctorate was then an arduous proceeding: One had to pass examinations in two foreign languages, one of them Greek or Latin, defend one’s thesis, and take an oral examination on general knowledge in one’s field. At Columbia University of an earlier day, a secretary sat outside the room where these examinations were administered, a pitcher of water and a glass on her desk. The water and glass were there for the candidates who fainted.</blockquote>
<p>Is he correct that the doctoral examination no longer looks like this? Yes. There is no exam in Greek or Latin, nor an oral exam of general knowledge. But that's because the structure of doctoral education has shifted. In the past, doctoral education was very self-directed, with candidates choosing a course of study and pursuing it mostly on his (or her - but let's be real, back in the day mostly his) own. Candidates might spend years lurking around dark, dusty libraries, looking for some groundbreaking thesis to pursue. At the end, it was necessary to show that time hadn't simply been spent trying to write the most off-the-wall contribution to general knowledge, but that the candidate had also learned enough about the field of study to recognize how their contribution fits.</p>
<p>Today? Anyone interested in pursuing a doctorate must complete a certain amount of coursework, some elective but much of it required to establish the requisite knowledge in the chosen field. After that, they must also complete candidacy exams, which may be oral, as Epstein describes above, or written, or some combination. The point is to ensure the candidate has the foundational knowledge necessary to become an expert in the field. Then - and only then - can the candidate propose a dissertation. Other than Greek and Latin, the requirements are much the same, and in some ways, more stringent.</p>
<p>Honestly, not only do I think Epstein's dismissal of Dr. Biden's doctorate is ridiculous coming from someone with a Cracker Jack Prize of a doctorate, but I also suspect that if Dr. Biden were a man, using the well-earned title of Dr. wouldn't be an issue.</p>
<p>Seriously, WSJ? It's these kinds of articles that make me question whether I should keep subscribing to you. It's 2020. Do better.</p>
<div class="separator" style="clear: both;"><a href="https://images.wsj.net/im-271564?width=1260&size=1.5" style="display: block; padding: 1em 0px; text-align: center;"><img alt="" border="0" data-original-height="533" data-original-width="800" src="https://images.wsj.net/im-271564?width=1260&size=1.5" width="320" /></a></div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-52893451287505886572020-12-09T12:55:00.002-06:002020-12-09T12:55:11.236-06:00COVID<p> Hey all,</p><p>It's been a long time since I've updated! Though I've commented a bit on the pandemic on this blog, I've mostly stayed pretty quiet. Unfortunately, the COVID pandemic has hit home quite literally.</p><p>I'm currently in Kansas City with my family. My parents are older and have a variety of risk factors, so they've been staying in all the time. My brother, who lives with them, works in an elementary school, and though he's always been safe and careful, it appears he caught COVID shortly before Thanksgiving. Other than a bad cough, he reported feeling fine. Late last week, my dad had a COVID test done in advance of a procedure, and though he also felt fine, his test came back positive. Shortly after, my mom got a test that also came back positive. They're both experiencing more symptoms now, like shortness of breath and fatigue. My test done that same day came back negative, but yesterday, I started to feel some COVID symptoms myself, mostly fatigue (which could be as much due to stress as COVID).</p><p>We're all very lucky that our cases appear to be mild, and my parents' providers are checking in with them regularly to make sure they're recovering well. After this week, I'll probably take advantage of my excess vacation time and take time off from work to rest and recover. I'm in Kansas City for the rest of the year, and thanks to my parents' huge backyard, don't even have to leave to give Zep his much-needed outdoor time.</p><p>Stay safe and healthy, everyone! </p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-1882661316437413222020-09-05T20:41:00.004-05:002020-09-05T20:41:50.748-05:00A Weekend of Writing<p>Just a quick update post. I'm spending my weekend doing something I've wanted to do for years - I decided to join the <a href="https://www.3daynovel.com">International 3-Day Novel Contest</a>. Every year, people around the world spend Labor Day weekend hunched over their computer or notebook, trying to write approximately 100 double-spaced pages (or more) of a complete novel. Writers submit their work, and in the Spring, the winner gets their book published by <a href="https://www.anvilpress.com/index.php">Anvil Press</a>.</p><p>I'm stocked up on groceries, my dog is staying with a friend (who has also agreed to sign my witness affidavit, that I followed the rules of writing, most importantly that writing only occurred between Saturday from 12:00 am until Monday at 11:59 pm), and I've got 27 pages written. Let's do this.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3E-3W0fQy4RnTdbrZMPrLgJIVfGOIicEM7xQXYudmKVQZTkt_hDlRQXsqaB0P0xMHPEUkV2ElyAmeoPJFnPJIOYqVB3_heFDbzhA6CtY_TTBvaItIqHr_umjjTBjx3WNgSw-HXRrX7-f0/s512/3daynovel.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="512" data-original-width="512" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3E-3W0fQy4RnTdbrZMPrLgJIVfGOIicEM7xQXYudmKVQZTkt_hDlRQXsqaB0P0xMHPEUkV2ElyAmeoPJFnPJIOYqVB3_heFDbzhA6CtY_TTBvaItIqHr_umjjTBjx3WNgSw-HXRrX7-f0/s320/3daynovel.jpg" /></a></div><br /><p><br /></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-14455033324621744282020-08-15T14:57:00.002-05:002020-08-15T14:57:32.552-05:00Pets and Quarantine<p>I'm so thankful to have my sweet boy, Zeppelin, in my life. And when quarantine/shelter-in-place began, I was especially thankful to have him, because otherwise, I would have been completely alone. Unsurprisingly, <a href="https://fullscreen.com/wp-content/uploads/2020/07/Fullscreen-Culture-Hot-Take-Creature-Comfort.pdf">a recent study</a> found I'm not the only one to feel this way:</p>
<blockquote>Animal shelters across the country are being completely cleared out as people seek out creature comfort. In fact, more than one in four 18-37-year-olds with pets got their new friend during quarantine.<sup>1</sup> Pets are bringing much-needed doses of positivity: two-thirds of Gen Z and Millennials living with pets agree their pet has helped them stay positive during this time.<sup>1</sup><br /><br />
Pets are not only showing up in homes—we are seeing them brighten up our feeds, too. Online conversation around pet adoption spiked in mid-March, up 50% from the weekly average.5 Whether they have a furry friend or not, 80% of Gen Z and Millennials say seeing animal content on social media makes them happy, and 74% agree that they find comfort in animal content on social media.<sup>1</sup> Additionally, pet-related hashtags such as #MeetMyPet, #PetRoutine, and #TreatYourPet have been trending on TikTok throughout the pandemic.</blockquote>
<p>In fact, 68% of respondents said their pet helped them feel less alone, 65% said their pet helped them to "stay sane" during the pandemic, 54% believe having a pet has made them be healthier, and 39% said they'd been talking to their pet more during quarantine (guilty).</p>
<p>If you wish you had a four-legged friend during this difficult time, there are tons in need of a good home! I'm so glad this sweet guy is part of mine:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgizEZK4geDaZ9_Kuix6FfA7c8VJP2hbTzBGYJMcYRDidwpasy-cBPG8fFYKj1zC2WJtXGX10dTB_9n53EObvY_TEo7wepgApZDObR8O_z-z8vcOBnFKs7dygMBtzkJHN6eBlVK_wo4aza_/s2048/84DCAF4C-EEB4-4A17-A77C-D2AEC8110CE1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="2048" data-original-width="2048" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgizEZK4geDaZ9_Kuix6FfA7c8VJP2hbTzBGYJMcYRDidwpasy-cBPG8fFYKj1zC2WJtXGX10dTB_9n53EObvY_TEo7wepgApZDObR8O_z-z8vcOBnFKs7dygMBtzkJHN6eBlVK_wo4aza_/s640/84DCAF4C-EEB4-4A17-A77C-D2AEC8110CE1.JPG" /></a></div><p><br /></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-4989239416361897092020-08-12T20:03:00.003-05:002020-08-12T20:03:21.527-05:00Creating Things<p> Normally, this time of year, we'd be getting excited for my choir's new season and rehearsals to begin in early September. Sadly, with the pandemic, it's unlikely we'll be getting together then, and I'm not sure how long it will take before it's safe and people begin feeling comfortable gathering in such a way. So I've been seeking out ways to keep some creativity in my life.</p><p>I've started drawing again, something I haven't done in years. I'm a bit rusty but hey - practice practice, right? I started with some pretty flowers from my parents' backyard, in a combination of soft chalk pastels (my favorite medium) and colored pencil:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVoS5aYDtoSKHfe-aAPGZKwA7HlkeUG47cIvx6An-avRRiGYM__tGX3XQbHoDewyreR_NKjukE1weZ2l_3NG7M1czCYPJSD8NBMIORtNBHlYdtebD5bVwSHbrcHxK8wdIKjMRrdrQEmpVb/s2048/IMG_9785.HEIC" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="2048" data-original-width="1536" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVoS5aYDtoSKHfe-aAPGZKwA7HlkeUG47cIvx6An-avRRiGYM__tGX3XQbHoDewyreR_NKjukE1weZ2l_3NG7M1czCYPJSD8NBMIORtNBHlYdtebD5bVwSHbrcHxK8wdIKjMRrdrQEmpVb/s640/IMG_9785.HEIC" /></a></div><p>And my next project is going to be a self-portrait, something I've never done before. Some early work with pencil that I'll fill in soon (thinking again a combo of colored pencil and chalk pastels):</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1uNATddRjIfONrR1fVjwOV8WOrjKpqOoa6FxxB0y6M_7JuQV98YCWDpkYVmaIuT7lJFqeo4GOrjEB-c-iqF6j2Hzc-wK1eODnc5lUj1gOcjWmu9C3PwlSfY5Q4Foe0tH9-YlyeLsK25f1/s2048/IMG_0013.HEIC" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="2048" data-original-width="1536" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1uNATddRjIfONrR1fVjwOV8WOrjKpqOoa6FxxB0y6M_7JuQV98YCWDpkYVmaIuT7lJFqeo4GOrjEB-c-iqF6j2Hzc-wK1eODnc5lUj1gOcjWmu9C3PwlSfY5Q4Foe0tH9-YlyeLsK25f1/s640/IMG_0013.HEIC" /></a></div><p>I also had some fun putting together a Lego Architecture set of Paris:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1KNgS3VtnD0FDfi1AVClQ3mn5sBGiecJPeYvYMTayQdN-uIsOrwjG_wHvxzVgw_Iwi_5fNVNBTgXggP7L1YyiGY6jnNvZcPHv1YlzffGNPq2u2YauRt9P_qKla8VyeWLTAOZqXjYbRweP/s2048/IMG_4684.HEIC" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1536" data-original-width="2048" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1KNgS3VtnD0FDfi1AVClQ3mn5sBGiecJPeYvYMTayQdN-uIsOrwjG_wHvxzVgw_Iwi_5fNVNBTgXggP7L1YyiGY6jnNvZcPHv1YlzffGNPq2u2YauRt9P_qKla8VyeWLTAOZqXjYbRweP/s640/IMG_4684.HEIC" width="640" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjx0FmJTJkoXI7KkDOH9F1EYyWMgVfj_z9avF6TzIZAG5qH-YGSn443PgxsvFnO5dOMdtFZdXM2O8FsEBxydCsGeNN9JYRQGtKTH3QNvyIJOeU2CUR5fvd5kQoMdzNjRhoJmoEQHRLwVVMt/s2048/IMG_1535.HEIC" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1536" data-original-width="2048" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjx0FmJTJkoXI7KkDOH9F1EYyWMgVfj_z9avF6TzIZAG5qH-YGSn443PgxsvFnO5dOMdtFZdXM2O8FsEBxydCsGeNN9JYRQGtKTH3QNvyIJOeU2CUR5fvd5kQoMdzNjRhoJmoEQHRLwVVMt/s640/IMG_1535.HEIC" width="640" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhChGlwb4suKlenyZ7Ptoe-nZ8SBUJw0hbG6fRPkoKtw7lq5JmHGnULb1cOj9xUg-W07OVNn198McN384RF6ewX0u5b4DNDn2w-_vatHs21SPD5UqktktFevR6kLim_AxLgDbdxVuAEYsZy/s2048/IMG_2898.HEIC" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1536" data-original-width="2048" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhChGlwb4suKlenyZ7Ptoe-nZ8SBUJw0hbG6fRPkoKtw7lq5JmHGnULb1cOj9xUg-W07OVNn198McN384RF6ewX0u5b4DNDn2w-_vatHs21SPD5UqktktFevR6kLim_AxLgDbdxVuAEYsZy/s640/IMG_2898.HEIC" width="640" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlvMWq7JYHVPXxwoddAFNPkmfhGVN4aGld3khB-EP-v68LbE1KHJ6pwD5GlfF5X-mx7EfSGTZlA78unK8k_truE-lxdgyuXaAnyR2p6CuSfPvFx1Q3PsTOicQYcnIR4qkKbtHTKThyphenhyphenk83c/s2048/IMG_3781.HEIC" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="2048" data-original-width="1536" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlvMWq7JYHVPXxwoddAFNPkmfhGVN4aGld3khB-EP-v68LbE1KHJ6pwD5GlfF5X-mx7EfSGTZlA78unK8k_truE-lxdgyuXaAnyR2p6CuSfPvFx1Q3PsTOicQYcnIR4qkKbtHTKThyphenhyphenk83c/s640/IMG_3781.HEIC" /></a></div><p>What mainly sparked this round of creativity was writing and recording an arrangement for my choir's virtual benefit. I had so much fun with that, I'm going to keep doing it! I'm planning to share that video soon, and have also started recording some other a cappella arrangements I plan on sharing. </p><p>And lastly, because I needed to bring Zep into the fun too, I've finally set up an Instagram for him. If you're on the 'gram, you can follow him here: <a href="https://www.instagram.com/zeppelinblackdog/">https://www.instagram.com/zeppelinblackdog/</a></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-89570865135749308712020-08-11T11:11:00.003-05:002020-08-11T11:11:44.392-05:00Coronavirus "Truthers" and Men Without Masks<p>Two articles related to coronavirus crossed my newsfeed this morning. First is an inside l<a href="https://melmagazine.com/en-us/story/covid-coronavirus-truth-meme-page-facebook">ook at the various Coronavirus "Truth" sites on Facebook</a>, which peddle a variety of misinformation - from the argument that mask-wearing is a prelude to the imposition of Sharia law to masks as a way to increase child sex trafficking:</p>
<blockquote>Just searching “coronavirus” will take you to a host of legitimate resources: pages for the CDC, the World Health Organization and the American Medical Association. But add a word like “truth” and suddenly you’re on a different planet: groups that exist as safe spaces for coronavirus skeptics to share theories of what’s really going on.<br /><br />
For every post or meme that bears a “False Information” label and links to fact-checking sites, there are dozens that elude this moderation, often as they do not present a debunkable statement. How exactly are you supposed to disprove the notion that face-mask enforcement is a prelude to some requirement that women wear the Muslim niqab?<br /><br />
The misinformation is so diversified (yet interconnected and overlapping) that you are bound to find your personal bogeyman at the bottom of the rabbit hole. These memes and talking points are made to frighten while appealing to your “common sense,” to flatter your intellect as it suckers you in with specious “logic” and emotional whataboutery.</blockquote>
<p>Sadly, I've seen a lot of these memes and specious arguments on the pages of friends and acquaintances.</p>
<p>The <a href="https://www.vox.com/the-goods/21356150/american-men-wont-wear-masks-covid-19">second article</a> discusses research that attempts to explain why men are being hit harder with Coronavirus: performative masculinity:</p>
<blockquote><a href="https://www.pewresearch.org/politics/2020/06/25/republicans-democrats-move-even-further-apart-in-coronavirus-concerns/">Poll after poll</a>, most recently a <a href="https://news.gallup.com/poll/315590/americans-face-mask-usage-varies-greatly-demographics.aspx">Gallup poll from July 13</a>, has found American men are more likely to not wear masks compared to women. Specifically, the survey found that 34 percent of men compared to 54 percent of women responded they “always” wore a mask when outside their home and that 20 percent of men said they “never” wore a mask outside their home (compared to just 8 percent of women).<br /><br />
Tyler Reny, a postdoctoral research fellow at Washington University in St. Louis, found [similar results] by combing through data from the Democracy Fund + UCLA Nationscape project, a public opinion survey that’s been interviewing more than 6,000 Americans about the virus per week since March 19.
<br /><br />
“Those who had more sexist attitudes were far less likely to report feeling concerned about the pandemic, less likely to support state and local coronavirus policies, less likely to take precautions like washing their hands or wearing masks, and more likely to get sick than those with less sexist attitudes,” Reny told me. “What I found is that sexist attitudes are very predictive of all four sets of [aforementioned] outcomes, even after accounting for differences in partisanship, ideology, age, education, and population density.”</blockquote>
<p>Stay healthy, stay informed, and please:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5I2Dlc3R9pSP6ZWsWFU0UfcBRFozuUBupB3Aw2hGWGP8eXG6zCG7JlimYFDnxGseGCbT3iOxuxmuKFSmgrKPQFk5-nVeMeYyNOIYEN5P_S8zfU7QKZ-LdhDscfIz43hxhyeozzRwHctqe/s360/Mask-1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="360" data-original-width="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5I2Dlc3R9pSP6ZWsWFU0UfcBRFozuUBupB3Aw2hGWGP8eXG6zCG7JlimYFDnxGseGCbT3iOxuxmuKFSmgrKPQFk5-nVeMeYyNOIYEN5P_S8zfU7QKZ-LdhDscfIz43hxhyeozzRwHctqe/s0/Mask-1.jpg" /></a></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-48911233895127888102020-08-10T18:44:00.011-05:002020-08-10T18:46:44.169-05:00TV Shows on the "Big 3" Streaming Services<p>2020 has been a tough year, and I've been doing my best to keep busy (and distracted from all the insanity - both at the personal and worldwide levels). Earlier this year, I took a course in machine learning techniques and have been working on applying those techniques to work datasets, as well as fun sets through Kaggle.com.</p>
<p>Today, I thought I'd share another dataset I discovered <a href="https://www.kaggle.com/ruchi798/tv-shows-on-netflix-prime-video-hulu-and-disney">through Kaggle</a>: TV shows available on one or more streaming service (Netflix, Hulu, Prime, and Disney+). There are lots of fun things we could do with this dataset. Let's start with some basic visualization and summarization.</p>
<div class="chunk" id="unnamed-chunk-1"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl kwd">setwd</span><span class="hl std">(</span><span class="hl str">"~/Dropbox"</span><span class="hl std">)</span>
<span class="hl kwd">library</span><span class="hl std">(tidyverse)</span>
</pre></div>
<div class="message"><pre class="knitr r">## ── Attaching packages ────────────────────────────────────────────────────────── tidyverse 1.3.0 ──
</pre></div>
<div class="message"><pre class="knitr r">## ✓ ggplot2 3.3.0 ✓ purrr 0.3.4
## ✓ tibble 3.0.0 ✓ dplyr 0.8.5
## ✓ tidyr 1.0.2 ✓ stringr 1.4.0
## ✓ readr 1.3.1 ✓ forcats 0.5.0
</pre></div>
<div class="message"><pre class="knitr r">## ── Conflicts ───────────────────────────────────────────────────────────── tidyverse_conflicts() ──
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
</pre></div>
<div class="source"><pre class="knitr r"><span class="hl std">Shows</span> <span class="hl kwb"><-</span> <span class="hl kwd">read_csv</span><span class="hl std">(</span><span class="hl str">"tv_shows.csv"</span><span class="hl std">)</span>
</pre></div>
<div class="warning"><pre class="knitr r">## Warning: Missing column names filled in: 'X1' [1]
</pre></div>
<div class="message"><pre class="knitr r">## Parsed with column specification:
## cols(
## X1 = col_double(),
## Title = col_character(),
## Year = col_double(),
## Age = col_character(),
## IMDb = col_double(),
## `Rotten Tomatoes` = col_character(),
## Netflix = col_double(),
## Hulu = col_double(),
## `Prime Video` = col_double(),
## `Disney+` = col_double(),
## type = col_double()
## )
</pre></div>
</div></div>
<p>First, we can do some basic summaries, such as how many shows in the dataset are on each of the streaming services.</p>
<div class="chunk" id="unnamed-chunk-2"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">Counts</span> <span class="hl kwb"><-</span> <span class="hl std">Shows</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">Netflix</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(Netflix),</span>
<span class="hl kwc">Hulu</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(Hulu),</span>
<span class="hl kwc">Prime</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(`Prime Video`),</span>
<span class="hl kwc">Disney</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(`Disney+`))</span> <span class="hl opt">%>%</span>
<span class="hl kwd">pivot_longer</span><span class="hl std">(</span><span class="hl kwc">cols</span> <span class="hl std">= Netflix</span><span class="hl opt">:</span><span class="hl std">Disney,</span>
<span class="hl kwc">names_to</span> <span class="hl std">=</span> <span class="hl str">"Service"</span><span class="hl std">,</span>
<span class="hl kwc">values_to</span> <span class="hl std">=</span> <span class="hl str">"Count"</span><span class="hl std">)</span>
<span class="hl std">Counts</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(Service,Count))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">()</span>
</pre></div>
</div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimUYlgKo0rc1l_-cDyGEUqz4sVwqOgXYN_S42Vt7kPfhVwhqTMtjdPkDYVvBot1ebmlsOcbfLEjVKajWdXl8SGs3h3wy4__wAC1RrLzZj34pk-wkx9tk1ls3auTSLfJlZINRxCn-kDEkTm/s504/unnamed-chunk-2-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="504" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimUYlgKo0rc1l_-cDyGEUqz4sVwqOgXYN_S42Vt7kPfhVwhqTMtjdPkDYVvBot1ebmlsOcbfLEjVKajWdXl8SGs3h3wy4__wAC1RrLzZj34pk-wkx9tk1ls3auTSLfJlZINRxCn-kDEkTm/s0/unnamed-chunk-2-1.png" /></a></div><div class="rimage default"><br /></div></div>
<p></p><p>The biggest selling point of Disney+ is to watch their movies, though the few TV shows they offer can't really be viewed elsewhere (e.g., The Mandalorian). For the sake of simplicity, we'll drop Disney+, and focus on the big 3 services for TV shows.</p>
<p>The dataset also contains an indicator of recommended age, which we can plot.</p>
<div class="chunk" id="unnamed-chunk-3"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">Shows</span> <span class="hl kwb"><-</span> <span class="hl std">Shows</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">Age</span> <span class="hl std">=</span> <span class="hl kwd">factor</span><span class="hl std">(Age,</span>
<span class="hl kwc">labels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"all"</span><span class="hl std">,</span>
<span class="hl str">"7+"</span><span class="hl std">,</span>
<span class="hl str">"13+"</span><span class="hl std">,</span>
<span class="hl str">"16+"</span><span class="hl std">,</span>
<span class="hl str">"18+"</span><span class="hl std">),</span>
<span class="hl kwc">ordered</span> <span class="hl std">=</span> <span class="hl num">TRUE</span><span class="hl std">))</span>
<span class="hl std">Shows</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(Age))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_bar</span><span class="hl std">()</span>
</pre></div>
</div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZjtcNNrwN92HZhSjOpfBfgnIKYt_aBsEl0cr4PoENreztwTSeSWOBChZL0JFFegBPCPWHH0c7sfUUC5XkalvXHN4BXr289yNyh68meo18PKB8Dqr6T1MX8ixcOXnwphNFSFH17-b9MPcq/s504/unnamed-chunk-3-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="504" data-original-width="504" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZjtcNNrwN92HZhSjOpfBfgnIKYt_aBsEl0cr4PoENreztwTSeSWOBChZL0JFFegBPCPWHH0c7sfUUC5XkalvXHN4BXr289yNyh68meo18PKB8Dqr6T1MX8ixcOXnwphNFSFH17-b9MPcq/s0/unnamed-chunk-3-1.png" /></a></div><div class="rimage default"><br /></div></div>
<p>Many are 'NA' for age, though it isn't clear why. Are these older shows, added before these streaming services were required to add guidance on these issues? Is this issue seen more for a particular streaming site? Let's find out</p>
<div class="chunk" id="unnamed-chunk-4"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">Shows</span> <span class="hl opt">%>%</span>
<span class="hl kwd">group_by</span><span class="hl std">(Age)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">Count</span> <span class="hl std">=</span> <span class="hl kwd">n</span><span class="hl std">(),</span>
<span class="hl kwc">Year_min</span> <span class="hl std">=</span> <span class="hl kwd">min</span><span class="hl std">(Year),</span>
<span class="hl kwc">Year_max</span> <span class="hl std">=</span> <span class="hl kwd">max</span><span class="hl std">(Year),</span>
<span class="hl kwc">Prime</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(`Prime Video`)</span><span class="hl opt">/</span><span class="hl num">2144</span><span class="hl std">,</span>
<span class="hl kwc">Netflix</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(Netflix)</span><span class="hl opt">/</span><span class="hl num">1931</span><span class="hl std">,</span>
<span class="hl kwc">Hulu</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(Hulu)</span><span class="hl opt">/</span><span class="hl num">1754</span><span class="hl std">)</span>
</pre></div>
<div class="warning"><pre class="knitr r">## Warning: Factor `Age` contains implicit NA, consider using
## `forcats::fct_explicit_na`
</pre></div>
<div class="output"><pre class="knitr r">## # A tibble: 6 x 7
## Age Count Year_min Year_max Prime Netflix Hulu
## <ord> <int> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 all 4 1995 2003 0.000466 0.00155 0
## 2 7+ 1018 1955 2020 0.0975 0.206 0.293
## 3 13+ 750 1980 2020 0.0849 0.186 0.136
## 4 16+ 848 1943 2020 0.104 0.155 0.208
## 5 18+ 545 1932 2020 0.0896 0.0886 0.0906
## 6 <NA> 2446 1901 2020 0.623 0.363 0.272
</pre></div>
</div></div>
<p>It seems the biggest "offender" for missing age information is Prime - about 62% of the shows don't have an age indicator. More surprising, though, is the minimum year for some of these categories. I'm no expert in the history of TV, but I don't think any shows were being broadcast in 1901. What are these outliers?</p>
<div class="chunk" id="unnamed-chunk-5"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">YearOutliers</span> <span class="hl kwb"><-</span> <span class="hl std">Shows</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(Year</span> <span class="hl opt"><</span> <span class="hl num">1940</span><span class="hl std">)</span>
<span class="hl kwd">list</span><span class="hl std">(YearOutliers</span><span class="hl opt">$</span><span class="hl std">Title)</span>
</pre></div>
<div class="output"><pre class="knitr r">## [[1]]
## [1] "Born To Explore" "The Three Stooges"
## [3] "The Little Rascals Classics" "Space: The New Frontier"
## [5] "Gods & Monsters with Tony Robinson" "History of Westinghouse"
## [7] "Betty Boop"
</pre></div>
</div></div>
<p>Four of these entries are clearly in error - these are newer shows. This isn't important at the moment, but it's interesting nonetheless.</p>
<p>In terms of getting the most "bang for your buck," Amazon Prime has the most shows to offer (though if you're looking for data on recommended age for the TV show, Prime has the most missingness). But Hulu and Netflix, in terms of volume, are pretty comparable to Prime. What can be said about the quality of content on each of the 3?</p>
<p>The dataset offers some indicators of quality: IMDb rating and Rotten Tomatoes score. How do the 3 services measure up on these indicators?</p>
<div class="chunk" id="unnamed-chunk-6"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">Netflix</span> <span class="hl kwb"><-</span> <span class="hl std">Shows</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(Netflix</span> <span class="hl opt">==</span> <span class="hl num">1</span><span class="hl std">)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">select</span><span class="hl std">(IMDb, `Rotten Tomatoes`)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">Service</span> <span class="hl std">=</span> <span class="hl str">"Netflix"</span><span class="hl std">)</span>
<span class="hl std">Hulu</span> <span class="hl kwb"><-</span> <span class="hl std">Shows</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(Hulu</span> <span class="hl opt">==</span> <span class="hl num">1</span><span class="hl std">)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">select</span><span class="hl std">(IMDb, `Rotten Tomatoes`)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">Service</span> <span class="hl std">=</span> <span class="hl str">"Hulu"</span><span class="hl std">)</span>
<span class="hl std">Prime</span> <span class="hl kwb"><-</span> <span class="hl std">Shows</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(`Prime Video`</span> <span class="hl opt">==</span> <span class="hl num">1</span><span class="hl std">)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">select</span><span class="hl std">(IMDb, `Rotten Tomatoes`)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">Service</span> <span class="hl std">=</span> <span class="hl str">"Prime"</span><span class="hl std">)</span>
<span class="hl std">BigThree</span> <span class="hl kwb"><-</span> <span class="hl kwd">rbind</span><span class="hl std">(Netflix, Hulu, Prime)</span>
<span class="hl std">BigThree</span> <span class="hl kwb"><-</span> <span class="hl std">BigThree</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">RotTom</span> <span class="hl std">=</span> <span class="hl kwd">as.numeric</span><span class="hl std">(</span><span class="hl kwd">sub</span><span class="hl std">(</span><span class="hl str">"%"</span><span class="hl std">,</span><span class="hl str">""</span><span class="hl std">,`Rotten Tomatoes`))</span><span class="hl opt">/</span><span class="hl num">100</span><span class="hl std">)</span>
<span class="hl std">BigThree</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(Service, IMDb))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_boxplot</span><span class="hl std">()</span>
</pre></div>
<div class="warning"><pre class="knitr r">## Warning: Removed 1194 rows containing non-finite values (stat_boxplot).
</pre></div>
</div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIgogKxhfyIJlKqSaKFxcEzJPu2GP_bvlrfXCV2ZB88-hFSIKT2SLWttYiIJhP3ChWPJrQ9_4-zD9qLSFvLwUfI5G3QjUylJhkexUdpskWn2syCZ_RKNjdLt1SVvSMRezjPBpmL_tCRYU6/s504/unnamed-chunk-6-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="504" data-original-width="504" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIgogKxhfyIJlKqSaKFxcEzJPu2GP_bvlrfXCV2ZB88-hFSIKT2SLWttYiIJhP3ChWPJrQ9_4-zD9qLSFvLwUfI5G3QjUylJhkexUdpskWn2syCZ_RKNjdLt1SVvSMRezjPBpmL_tCRYU6/s0/unnamed-chunk-6-1.png" /></a></div><div class="rimage default"><br /></div><div class="rcode">
<div class="source"><pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(scales)</span>
</pre></div>
<div class="message"><pre class="knitr r">##
## Attaching package: 'scales'
</pre></div>
<div class="message"><pre class="knitr r">## The following object is masked from 'package:purrr':
##
## discard
</pre></div>
<div class="message"><pre class="knitr r">## The following object is masked from 'package:readr':
##
## col_factor
</pre></div>
<div class="source"><pre class="knitr r"><span class="hl std">BigThree</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(Service, RotTom))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_boxplot</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_y_continuous</span><span class="hl std">(</span><span class="hl kwc">labels</span> <span class="hl std">= percent)</span>
</pre></div>
<div class="warning"><pre class="knitr r">## Warning: Removed 4772 rows containing non-finite values (stat_boxplot).
</pre></div>
</div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8-ec-wmMvBcI7udjI14ETH2qOwT8WtYIM3AuTjXLCsDuMzFYeuIvuHTAEA8K2COf9XjI_yyGWsKtBTbnGSmBv0vLLKvgWQXN_FzzMBGdrJAk0mlvw0bggeL76muGZLJkrxyV3uAixu6jt/s504/unnamed-chunk-6-2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="504" data-original-width="504" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8-ec-wmMvBcI7udjI14ETH2qOwT8WtYIM3AuTjXLCsDuMzFYeuIvuHTAEA8K2COf9XjI_yyGWsKtBTbnGSmBv0vLLKvgWQXN_FzzMBGdrJAk0mlvw0bggeL76muGZLJkrxyV3uAixu6jt/s0/unnamed-chunk-6-2.png" /></a></div><div class="rimage default"><br /></div></div>
<p>It doesn't appear the 3 streaming services differ too much in terms of quality. But there's more analysis we can do of this dataset. More later.</p>
<p></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-48237192438702761372020-07-07T15:01:00.001-05:002020-07-07T15:01:45.589-05:00Free Virtual Concert!One of my hobbies is singing, and for the last 15 years, I've been a member of the <a href="https://www.apollochorus.org/">Apollo Chorus of Chicago</a>. As with many musical arts organizations, we canceled our Spring concerts, including our annual Apollo After Hours benefit, due to COVID-19. It's unclear when in the future music organizations will be able to have in-person concerts again - possibly years.<div><br /></div><div>But that doesn't mean we can't make - and share - beautiful music with you. On Friday, July 17 at 7PM, we'll be broadcasting our annual benefit as a free, virtual performance. Lots of singers in my choir have created videos to be included in the broadcast, including me! Here's a photo preview:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKcWOOsYocYWW0raphyWSdSpe4pDK6LCbT05Y6OaNySd55KTIGC9VQ1UQC2QESOQ9jJ745n3qXr31ZoVUbM7iVtaIAbkzDN2OyjI0Ss4uM6UT0p-Kg2Oj9UuxrVY_x-ZMCNeXrKiuey8Sa/s848/AAHvideo.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="478" data-original-width="848" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKcWOOsYocYWW0raphyWSdSpe4pDK6LCbT05Y6OaNySd55KTIGC9VQ1UQC2QESOQ9jJ745n3qXr31ZoVUbM7iVtaIAbkzDN2OyjI0Ss4uM6UT0p-Kg2Oj9UuxrVY_x-ZMCNeXrKiuey8Sa/s320/AAHvideo.jpg" width="320" /></a></div><div><br /></div><div>I'll be performing an a cappella arrangement I wrote of a Sara Bareilles song, "Breathe Again." If you want to hear it, you'll have to tune in! Find out more and sign up to get the link once it goes live <a href="https://www.apollochorus.org/benefit/">here</a>.</div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-19237865069708983952020-06-25T12:06:00.007-05:002020-07-01T08:23:04.439-05:00Flying Saucers and Bright Lights: A Data Visualization<head>
<style type="text/css">
.inline {
background-color: #f7f7f7;
border:solid 1px #B0B0B0;
}
.error {
font-weight: bold;
color: #FF0000;
}
.warning {
font-weight: bold;
}
.message {
font-style: italic;
}
.source, .output, .warning, .error, .message {
padding: 0 1em;
border:solid 1px #F7F7F7;
}
.source {
background-color: #f5f5f5;
}
.left {
text-align: left;
}
.right {
text-align: right;
}
.center {
text-align: center;
}
.hl.num {
color: #AF0F91;
}
.hl.str {
color: #317ECC;
}
.hl.com {
color: #AD95AF;
font-style: italic;
}
.hl.opt {
color: #000000;
}
.hl.std {
color: #585858;
}
.hl.kwa {
color: #295F94;
font-weight: bold;
}
.hl.kwb {
color: #B05A65;
}
.hl.kwc {
color: #55aa55;
}
.hl.kwd {
color: #BC5A65;
font-weight: bold;
}
</style>
<title>UFO Sightings by Shape and Year</title>
</head>
<body>
<p>Earlier last week, I taught part 2 of a course on using R and tidyverse for my work colleagues. I wanted a fun dataset to use as an example for coding exercises throughout. There was really only one choice.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8x3jd2FtPwXZucRKpI8wy-ZbnlxN3qXoRx60HAGi7G15ztqt9gfFeXUvr3JkhaqeySjD_VNEPPNLpgrD8aKxZy1J4ciaNBupF6brq8l-mY94LkHrU1llRhuPnmCedh17NSyEMstgDGO3P/s389/alens.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="292" data-original-width="389" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8x3jd2FtPwXZucRKpI8wy-ZbnlxN3qXoRx60HAGi7G15ztqt9gfFeXUvr3JkhaqeySjD_VNEPPNLpgrD8aKxZy1J4ciaNBupF6brq8l-mY94LkHrU1llRhuPnmCedh17NSyEMstgDGO3P/s320/alens.jpg" width="320" /></a></div>
<p></p>
<p>I found this great dataset through kaggle.com - <a href="https://www.kaggle.com/NUFORC/ufo-sightings">UFO sightings reported to the National UFO Reporting Center (NUFORC) through 2014</a>. This dataset gave lots of variables we could play around with, and I'd like to use it in a future session with my colleagues to talk about the process of cleaning data.</p>
<p>If you're interested in learning more about R and tidyverse, you can access my slides from the sessions <a href="https://www.dropbox.com/s/k19denplcxq1mcz/Introduction%20to%20R.pptx?dl=1">here</a>. (We stopped at filtering and picked up there for part 2, so everything is in one Powerpoint file.)</p>
<p>While working with the dataset to plan my learning sessions, I started playing around and thought it would be fun to show the various shapes of UFOs reported over time, to see if there were any shifts. Spoiler: There were. But I needed to clean the data a bit first.</p>
<div class="chunk" id="unnamed-chunk-1"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl kwd">setwd</span><span class="hl std">(</span><span class="hl str">"~/Downloads/UFO Data"</span><span class="hl std">)</span>
<span class="hl kwd">library</span><span class="hl std">(tidyverse)</span>
</pre></div>
<div class="message"><pre class="knitr r">## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --
</pre></div>
<div class="message"><pre class="knitr r">## v ggplot2 3.3.1 v purrr 0.3.4
## v tibble 3.0.1 v dplyr 1.0.0
## v tidyr 1.1.0 v stringr 1.4.0
## v readr 1.3.1 v forcats 0.5.0
</pre></div>
<div class="message"><pre class="knitr r">## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
</pre></div>
<div class="source"><pre class="knitr r"><span class="hl kwd">options</span><span class="hl std">(</span><span class="hl kwc">scipen</span> <span class="hl std">=</span> <span class="hl num">999</span><span class="hl std">)</span>
<span class="hl std">UFOs</span> <span class="hl kwb"><-</span> <span class="hl kwd">read_csv</span><span class="hl std">(</span><span class="hl str">"UFOsightings.csv"</span><span class="hl std">,</span> <span class="hl kwc">col_names</span> <span class="hl std">=</span> <span class="hl num">TRUE</span><span class="hl std">)</span>
</pre></div>
<div class="message"><pre class="knitr r">## Parsed with column specification:
## cols(
## datetime = col_character(),
## city = col_character(),
## state = col_character(),
## country = col_character(),
## shape = col_character(),
## `duration (seconds)` = col_double(),
## `duration (hours/min)` = col_character(),
## comments = col_character(),
## `date posted` = col_character(),
## latitude = col_double(),
## longitude = col_double()
## )
</pre></div>
<div class="warning"><pre class="knitr r">## Warning: 4 parsing failures.
## row col expected actual file
## 27823 duration (seconds) no trailing characters ` 'UFOsightings.csv'
## 35693 duration (seconds) no trailing characters ` 'UFOsightings.csv'
## 43783 latitude no trailing characters q.200088 'UFOsightings.csv'
## 58592 duration (seconds) no trailing characters ` 'UFOsightings.csv'
</pre></div>
</div></div>
<p>There are 30 shapes represented in the data. That's a lot to show in a single figure.</p>
<div class="chunk" id="unnamed-chunk-2"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">UFOs</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">shapes</span> <span class="hl std">=</span> <span class="hl kwd">n_distinct</span><span class="hl std">(shape))</span>
</pre></div>
<div class="output"><pre class="knitr r">## # A tibble: 1 x 1
## shapes
## <int>
## 1 30
</pre></div>
</div></div>
<p>If we look at the different shapes in the data, we can see some overlap, as well as shapes with low counts.</p>
<div class="chunk" id="unnamed-chunk-3"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">UFOs</span> <span class="hl opt">%>%</span>
<span class="hl kwd">group_by</span><span class="hl std">(shape)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">count</span> <span class="hl std">=</span> <span class="hl kwd">n</span><span class="hl std">())</span>
</pre></div>
<div class="message"><pre class="knitr r">## `summarise()` ungrouping output (override with `.groups` argument)
</pre></div>
<div class="output"><pre class="knitr r">## # A tibble: 30 x 2
## shape count
## <chr> <int>
## 1 changed 1
## 2 changing 1962
## 3 chevron 952
## 4 cigar 2057
## 5 circle 7608
## 6 cone 316
## 7 crescent 2
## 8 cross 233
## 9 cylinder 1283
## 10 delta 7
## # ... with 20 more rows
</pre></div>
</div></div>
<p>For instance, "changed" only appears in one record. But "changing," which appears in 1,962 records should be grouped with "changed." After inspecting all the shapes, I identified the following categories that accounted for most of the different shapes:</p>
<ul><li>changing, which includes both changed and changing</li>
<li>circles, like disks, domes, and spheres</li>
<li>triangles, like deltas, pyramids, and triangles</li>
<li>four or more sided: rectangles, diamonds, and chevrons</li>
<li>light, which counts things like flares, fireballs, and lights</li>
</ul>
<p>I also made an "other" category for shapes with very low counts that didn't seem to fit in the categories above, like crescents, teardrops, and formations with no further specification of shape. Finally, shape was blank for some records, so I made an "unknown" category. Here's the code I used to recategorize shape.</p>
<div class="chunk" id="unnamed-chunk-4"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">changing</span> <span class="hl kwb"><-</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"changed"</span><span class="hl std">,</span> <span class="hl str">"changing"</span><span class="hl std">)</span>
<span class="hl std">circles</span> <span class="hl kwb"><-</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"circle"</span><span class="hl std">,</span> <span class="hl str">"disk"</span><span class="hl std">,</span> <span class="hl str">"dome"</span><span class="hl std">,</span> <span class="hl str">"egg"</span><span class="hl std">,</span> <span class="hl str">"oval"</span><span class="hl std">,</span><span class="hl str">"round"</span><span class="hl std">,</span> <span class="hl str">"sphere"</span><span class="hl std">)</span>
<span class="hl std">triangles</span> <span class="hl kwb"><-</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"cone"</span><span class="hl std">,</span><span class="hl str">"delta"</span><span class="hl std">,</span><span class="hl str">"pyramid"</span><span class="hl std">,</span><span class="hl str">"triangle"</span><span class="hl std">)</span>
<span class="hl std">fourormore</span> <span class="hl kwb"><-</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"chevron"</span><span class="hl std">,</span><span class="hl str">"cross"</span><span class="hl std">,</span><span class="hl str">"diamond"</span><span class="hl std">,</span><span class="hl str">"hexagon"</span><span class="hl std">,</span><span class="hl str">"rectangle"</span><span class="hl std">)</span>
<span class="hl std">light</span> <span class="hl kwb"><-</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"fireball"</span><span class="hl std">,</span><span class="hl str">"flare"</span><span class="hl std">,</span><span class="hl str">"flash"</span><span class="hl std">,</span><span class="hl str">"light"</span><span class="hl std">)</span>
<span class="hl std">other</span> <span class="hl kwb"><-</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"cigar"</span><span class="hl std">,</span><span class="hl str">"cylinder"</span><span class="hl std">,</span><span class="hl str">"crescent"</span><span class="hl std">,</span><span class="hl str">"formation"</span><span class="hl std">,</span><span class="hl str">"other"</span><span class="hl std">,</span><span class="hl str">"teardrop"</span><span class="hl std">)</span>
<span class="hl std">unknown</span> <span class="hl kwb"><-</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"unknown"</span><span class="hl std">,</span> <span class="hl str">'NA'</span><span class="hl std">)</span>
<span class="hl std">UFOs</span> <span class="hl kwb"><-</span> <span class="hl std">UFOs</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">shape2</span> <span class="hl std">=</span> <span class="hl kwd">ifelse</span><span class="hl std">(shape</span> <span class="hl opt">%in%</span> <span class="hl std">changing,</span>
<span class="hl str">"changing"</span><span class="hl std">,</span>
<span class="hl kwd">ifelse</span><span class="hl std">(shape</span> <span class="hl opt">%in%</span> <span class="hl std">circles,</span>
<span class="hl str">"circular"</span><span class="hl std">,</span>
<span class="hl kwd">ifelse</span><span class="hl std">(shape</span> <span class="hl opt">%in%</span> <span class="hl std">triangles,</span>
<span class="hl str">"triangular"</span><span class="hl std">,</span>
<span class="hl kwd">ifelse</span><span class="hl std">(shape</span> <span class="hl opt">%in%</span> <span class="hl std">fourormore,</span>
<span class="hl str">"four+-sided"</span><span class="hl std">,</span>
<span class="hl kwd">ifelse</span><span class="hl std">(shape</span> <span class="hl opt">%in%</span> <span class="hl std">light,</span>
<span class="hl str">"light"</span><span class="hl std">,</span>
<span class="hl kwd">ifelse</span><span class="hl std">(shape</span> <span class="hl opt">%in%</span> <span class="hl std">other,</span>
<span class="hl str">"other"</span><span class="hl std">,</span><span class="hl str">"unknown"</span><span class="hl std">)))))))</span>
</pre></div>
</div></div>
<p>My biggest question mark was cigar and cylinder. They're not really circles, nor do they fall in the four or more sided category. I could create another category called "tubes," but ultimately just put them in other. Using the code above as an example, you could see what happens to the chart if you put them in another category or create one of their own.</p>
<p>For the chart, I dropped the unknowns.</p>
<div class="chunk" id="unnamed-chunk-5"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">UFOs</span> <span class="hl kwb"><-</span> <span class="hl std">UFOs</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(shape2</span> <span class="hl opt">!=</span> <span class="hl str">"unknown"</span><span class="hl std">)</span>
</pre></div>
</div></div>
<p>Now, to plot shapes over time, I need to extract date information. The "datetime" variable is currently a character, so I have to convert that to a date. I then pulled out year, so that each point on my figure was the count of that shape observed during a given year.</p>
<div class="chunk" id="unnamed-chunk-6"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(lubridate)</span>
</pre></div>
<div class="message"><pre class="knitr r">##
## Attaching package: 'lubridate'
</pre></div>
<div class="message"><pre class="knitr r">## The following objects are masked from 'package:base':
##
## date, intersect, setdiff, union
</pre></div>
<div class="source"><pre class="knitr r"><span class="hl std">UFOs</span> <span class="hl kwb"><-</span> <span class="hl std">UFOs</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">Date2</span> <span class="hl std">=</span> <span class="hl kwd">as.Date</span><span class="hl std">(datetime,</span> <span class="hl kwc">format</span> <span class="hl std">=</span> <span class="hl str">"%m/%d/%Y"</span><span class="hl std">),</span>
<span class="hl kwc">Year</span> <span class="hl std">=</span> <span class="hl kwd">year</span><span class="hl std">(Date2))</span>
</pre></div>
</div></div>
<p>Now we have all the information we need to plot shapes over time, to see if there have been changes. We'll create a summary dataframe by Year and shape2, then create a line chart with that information.</p>
<div class="chunk" id="unnamed-chunk-7"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">Years</span> <span class="hl kwb"><-</span> <span class="hl std">UFOs</span> <span class="hl opt">%>%</span>
<span class="hl kwd">group_by</span><span class="hl std">(Year, shape2)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">count</span> <span class="hl std">=</span> <span class="hl kwd">n</span><span class="hl std">())</span>
</pre></div>
<div class="message"><pre class="knitr r">## `summarise()` regrouping output by 'Year' (override with `.groups` argument)
</pre></div>
<div class="source"><pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(scales)</span>
</pre></div>
<div class="message"><pre class="knitr r">##
## Attaching package: 'scales'
</pre></div>
<div class="message"><pre class="knitr r">## The following object is masked from 'package:purrr':
##
## discard
</pre></div>
<div class="message"><pre class="knitr r">## The following object is masked from 'package:readr':
##
## col_factor
</pre></div>
<div class="source"><pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(ggthemes)</span>
<span class="hl std">Years</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(Year, count,</span> <span class="hl kwc">color</span> <span class="hl std">= shape2))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_point</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_line</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_x_continuous</span><span class="hl std">(</span><span class="hl kwc">breaks</span> <span class="hl std">=</span> <span class="hl kwd">seq</span><span class="hl std">(</span><span class="hl num">1910</span><span class="hl std">,</span><span class="hl num">2020</span><span class="hl std">,</span><span class="hl num">10</span><span class="hl std">))</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_y_continuous</span><span class="hl std">(</span><span class="hl kwc">breaks</span> <span class="hl std">=</span> <span class="hl kwd">seq</span><span class="hl std">(</span><span class="hl num">0</span><span class="hl std">,</span><span class="hl num">3000</span><span class="hl std">,</span><span class="hl num">500</span><span class="hl std">),</span> <span class="hl kwc">labels</span> <span class="hl std">= comma)</span> <span class="hl opt">+</span>
<span class="hl kwd">labs</span><span class="hl std">(</span><span class="hl kwc">color</span> <span class="hl std">=</span> <span class="hl str">"Object Shape"</span><span class="hl std">,</span> <span class="hl kwc">title</span> <span class="hl std">=</span> <span class="hl str">"From Flying Saucers to Bright Lights:\nSightings of UFO Shapes Over Time"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">ylab</span><span class="hl std">(</span><span class="hl str">"Number of Sightings"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">theme_economist_white</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_color_tableau</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">theme</span><span class="hl std">(</span><span class="hl kwc">plot.title</span> <span class="hl std">=</span> <span class="hl kwd">element_text</span><span class="hl std">(</span><span class="hl kwc">hjust</span> <span class="hl std">=</span> <span class="hl num">0.5</span><span class="hl std">))</span>
</pre></div><br />
</div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYjRMAAPYXzCEYUEO2sBb34OKBXsS2I_-C7sTQzwekwt_RBN9sUuIWcGKguvma1Gq8QKMXxwJ9nekn0BTA9jvzvOFXHilsdygXN3bJtESzdjzpu_fjylJHrXHhUEPYulV6ZzpiOQVmtQWx/s648/unnamed-chunk-7-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="648" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYjRMAAPYXzCEYUEO2sBb34OKBXsS2I_-C7sTQzwekwt_RBN9sUuIWcGKguvma1Gq8QKMXxwJ9nekn0BTA9jvzvOFXHilsdygXN3bJtESzdjzpu_fjylJHrXHhUEPYulV6ZzpiOQVmtQWx/s320/unnamed-chunk-7-1.png" /></a></div></div>
<p>Until the mid-90s, the most commonly seen UFO was circular. After that, light shapes became much more common. I'm wondering if this could be explained in part by UFOs in pop culture, moving from the flying saucers of earlier sci-fi to the bright lights without discernible shape in the more recent sci-fi. The third most common shape is our "other" category, which suggests we might want to rethink that one. It could be that some of the shapes within that category are common enough to warrant their own category, while receiving other for those that don't have a good category of their own. Cigar and cylinder, for instance, have high counts and could be put in their own category. Feel free to play around with the data and see what you come up with!</p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-29346817828642417252020-06-10T08:00:00.000-05:002020-06-10T08:00:03.814-05:00Space Force: A ReviewI've continued to work from home during our shelter-in-place (something my boss recently told me we'll be doing for a while). During my copious downtime, I've gotten to watch a lot of things I've had on my watch-list, including the Netflix original series, <i>Space Force</i>.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBIcm_f7SiLO7RRgO7WfQSkw666eLZL7BNGLGrnnq9sTEh4nsBJb9gifhLTUGkWkp_7gu6gO_ttXboUDag30R5nTOzh5ZBgcOZJQGEPJMYE7sTsz18wdG4YRXCoWBHmi2X8whJmYti9kyX/s1600/space_force.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="926" data-original-width="1390" height="213" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBIcm_f7SiLO7RRgO7WfQSkw666eLZL7BNGLGrnnq9sTEh4nsBJb9gifhLTUGkWkp_7gu6gO_ttXboUDag30R5nTOzh5ZBgcOZJQGEPJMYE7sTsz18wdG4YRXCoWBHmi2X8whJmYti9kyX/s320/space_force.jpg" width="320" /></a></div>
<br />
I've made my way through season 1 of the series, and thoroughly enjoyed it. I was surprised to learn - partway through watching - that critics did not enjoy the series nearly as much as I did. I'll get to that shortly.<br />
<br />
I loved the political satire element of the show, that it was inspired by statements by our buffoon of a president. And I loved the periodic texts and tweets they referenced from a character they only referred to as "POTUS" (although, we all know who they mean). But really, I felt the critics were expecting something very different from what the show gave us, and that is the reason for their negative review.<br />
<br />
While the concept is hilarious, and Mark Naird (Steve Carrell's character) is often a buffoon, the show is really a family drama framed by absurdist comedy. General Naird is a single father of a teenage daughter (Diana Silver, from <i>Booksmart</i>, which I also thoroughly enjoyed), after his wife (played brilliantly by Lisa Kudrow) is imprisoned for an unmentioned crime (which earned her 40-60 years, so clearly really bad). The show deals with a variety of family issues, not just the aforementioned single parenthood, but also teenage rebellion and substance abuse, fear of abandonment, and a parent who often feels married to their job. It dealt with the concept of an open marriage in a way that was authentic, while also being heartbreaking and funny at the same time. The show made me cry just as often as it made me laugh, and I could often relate to Mark's character - his heartbreak when his wife suggested an open marriage was so real, I bawled. It poked fun at the full political spectrum, as well as at Boomers, X-ers, and Millennials alike.<br />
<br />
I think a lot of people were expecting Michael Scott as a general, but Mark Naird - though often a goof who really didn't understand science, which was an important part of his job, personified by his chief scientist (played so wonderfully by John Malkovich: better casting does not exist) - showed a surprising depth and understanding of people, in ways that both surprised and confirmed the conclusions of his scientists. Michael Scott seemed oblivious to the people who worked from him and showed zero understanding of people skills, while Mark Naird thought first and foremost about the people, and spoke eloquently on the topic.<br />
<br />
I especially loved the character of Captain Ali (played by Tawny Newsome) and look forward (hopefully) to learning more about her character. Of all the characters on the show, she's my favorite.<br />
<br />
It was also a joy to see Fred Willard as Mark's elderly father, who since filming his role has passed away. He will very much be missed and I'll be interested in seeing how they deal with the actor's death (since season 2 has not even been greenlit, let alone filmed). My only complaint was with the cheap jokes at his elderly mother's expense, including at one point showing the caretaker giving her CPR while Mark's father obliviously (and jovially) spoke on the phone. Mark's mother obviously has both lung (due to her being on oxygen) and heart (due to the CPR) issues, and as the daughter of a man with similar issues, I would have wished a show with so much heart had been more delicate with these conditions, rather than using them for cheap laughs.<br />
<br />
My only disappointment with <i>Space Force </i>(other than my complaint above) is with the critics' reaction to it. I sincerely hope there is a season 2.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-32470144948921285762020-05-12T08:58:00.001-05:002020-05-12T08:58:29.949-05:00ZoomiesCheck out this adorable Zoom meeting:<br />
<div style="text-align: center;">
<br /></div>
<blockquote class="twitter-tweet">
<div dir="ltr" lang="en">
<div style="text-align: center;">
Still having the company meetings online. <a href="https://t.co/aR3LfuSdKl">pic.twitter.com/aR3LfuSdKl</a></div>
</div>
<div style="text-align: center;">
— Andrew Cotter (@MrAndrewCotter) <a href="https://twitter.com/MrAndrewCotter/status/1259931151403290624?ref_src=twsrc%5Etfw">May 11, 2020</a></div>
</blockquote>
<script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-77013369096918660572020-05-03T09:00:00.000-05:002020-05-03T09:00:03.453-05:00Statistics Sunday: My 2019 ReadingI've spent the month of April blogging my way through the tidyverse, while using my reading dataset from 2019 as the example. Today, I thought I'd bring many of those analyses and data manipulation techniques together to do a post about my reading habits for the year.<br />
<div class="chunk" id="unnamed-chunk-1">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(tidyverse)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --
</pre>
</div>
<div class="message">
<pre class="knitr r">## <U+2713> ggplot2 3.2.1 <U+2713> purrr 0.3.3
## <U+2713> tibble 2.1.3 <U+2713> dplyr 0.8.3
## <U+2713> tidyr 1.0.0 <U+2713> stringr 1.4.0
## <U+2713> readr 1.3.1 <U+2713> forcats 0.4.0
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl kwb"><-</span> <span class="hl kwd">read_csv</span><span class="hl std">(</span><span class="hl str">"~/Downloads/Blogging A to Z/SaraReads2019_allchanges.csv"</span><span class="hl std">,</span>
<span class="hl kwc">col_names</span> <span class="hl std">=</span> <span class="hl num">TRUE</span><span class="hl std">)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## Parsed with column specification:
## cols(
## Title = col_character(),
## Pages = col_double(),
## date_started = col_character(),
## date_read = col_character(),
## Book.ID = col_double(),
## Author = col_character(),
## AdditionalAuthors = col_character(),
## AverageRating = col_double(),
## OriginalPublicationYear = col_double(),
## read_time = col_double(),
## MyRating = col_double(),
## Gender = col_double(),
## Fiction = col_double(),
## Childrens = col_double(),
## Fantasy = col_double(),
## SciFi = col_double(),
## Mystery = col_double(),
## SelfHelp = col_double()
## )
</pre>
</div>
</div>
</div>
As you recall, I read 87 books last year, by 42 different authors.<br />
<div class="chunk" id="unnamed-chunk-2">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">Books</span> <span class="hl std">=</span> <span class="hl kwd">n</span><span class="hl std">(),</span>
<span class="hl kwc">Authors</span> <span class="hl std">=</span> <span class="hl kwd">n_distinct</span><span class="hl std">(Author))</span>
</pre>
</div>
<div class="output">
<pre class="knitr r">## # A tibble: 1 x 2
## Books Authors
## <int> <int>
## 1 87 42
</pre>
</div>
</div>
</div>
Using summarise, we can get some basic information about each author.<br />
<div class="chunk" id="unnamed-chunk-3">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">authors</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">group_by</span><span class="hl std">(Author)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">Books</span> <span class="hl std">=</span> <span class="hl kwd">n</span><span class="hl std">(),</span>
<span class="hl kwc">Pages</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(Pages),</span>
<span class="hl kwc">AvgRating</span> <span class="hl std">=</span> <span class="hl kwd">mean</span><span class="hl std">(MyRating),</span>
<span class="hl kwc">Oldest</span> <span class="hl std">=</span> <span class="hl kwd">min</span><span class="hl std">(OriginalPublicationYear),</span>
<span class="hl kwc">Newest</span> <span class="hl std">=</span> <span class="hl kwd">max</span><span class="hl std">(OriginalPublicationYear),</span>
<span class="hl kwc">AvgRT</span> <span class="hl std">=</span> <span class="hl kwd">mean</span><span class="hl std">(read_time),</span>
<span class="hl kwc">Gender</span> <span class="hl std">=</span> <span class="hl kwd">first</span><span class="hl std">(Gender),</span>
<span class="hl kwc">Fiction</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(Fiction),</span>
<span class="hl kwc">Childrens</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(Childrens),</span>
<span class="hl kwc">Fantasy</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(Fantasy),</span>
<span class="hl kwc">Sci</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(SciFi),</span>
<span class="hl kwc">Mystery</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(Mystery))</span>
</pre>
</div>
</div>
</div>
Let's plot number of books by each author, with the bars arranged by number of books.<br />
<div class="chunk" id="unnamed-chunk-4">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">authors</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(</span><span class="hl kwd">reorder</span><span class="hl std">(Author,</span> <span class="hl kwd">desc</span><span class="hl std">(Books)), Books))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">theme</span><span class="hl std">(</span><span class="hl kwc">axis.text.x</span> <span class="hl std">=</span> <span class="hl kwd">element_text</span><span class="hl std">(</span><span class="hl kwc">angle</span> <span class="hl std">=</span> <span class="hl num">90</span><span class="hl std">))</span> <span class="hl opt">+</span>
<span class="hl kwd">xlab</span><span class="hl std">(</span><span class="hl str">"Author"</span><span class="hl std">)</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjUXpyiHyKeONzirWxvEjaD_Y6eNpTTlCwBT7HVWzHdUnHCEYy8Eak9icrQemfLhZot7XLqLl9K3NHziWYqq8ByW2BG_lCvIby9HcL1bjQPIRvbUgJvqa2GpyQ3Jr_n5k216-25GZOm4zp/s1600/unnamed-chunk-4-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="504" data-original-width="504" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjUXpyiHyKeONzirWxvEjaD_Y6eNpTTlCwBT7HVWzHdUnHCEYy8Eak9icrQemfLhZot7XLqLl9K3NHziWYqq8ByW2BG_lCvIby9HcL1bjQPIRvbUgJvqa2GpyQ3Jr_n5k216-25GZOm4zp/s320/unnamed-chunk-4-1.png" width="320" /></a></div>
<br /></div>
</div>
I could simplify this chart quite a bit by only showing authors with 2 or more books in the set, and also by flipping the axes so author can be read along the side.<br />
<div class="chunk" id="unnamed-chunk-5">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">authors</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">Author</span> <span class="hl std">=</span> <span class="hl kwd">fct_reorder</span><span class="hl std">(Author,</span> <span class="hl kwd">desc</span><span class="hl std">(Author)))</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(Books</span> <span class="hl opt">></span> <span class="hl num">1</span><span class="hl std">)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(</span><span class="hl kwd">reorder</span><span class="hl std">(Author, Books), Books))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">coord_flip</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">xlab</span><span class="hl std">(</span><span class="hl str">"Author"</span><span class="hl std">)</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjEbMg_uzcyMRzXf-bFDhBUyReNLZGoZhUr1ktxFw7MzfpwTdyFnYNhnGy3ggMheSWMOWfBgAZSOsae1wj_eESrthVga7tCpHAKQ0FfKLjk_T4yuYNFzq4gl-4QojzcrtEQ5YB4cxL9sbe/s1600/unnamed-chunk-5-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="504" data-original-width="504" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjEbMg_uzcyMRzXf-bFDhBUyReNLZGoZhUr1ktxFw7MzfpwTdyFnYNhnGy3ggMheSWMOWfBgAZSOsae1wj_eESrthVga7tCpHAKQ0FfKLjk_T4yuYNFzq4gl-4QojzcrtEQ5YB4cxL9sbe/s320/unnamed-chunk-5-1.png" width="320" /></a></div>
<br /></div>
</div>
Based on this data, I read the most books by L. Frank Baum (which makes sense, because I made a goal to reread all 14 Oz series books), followed by Terry Pratchett (which makes sense, because I love him). The code above is slightly more complex, because when I use coord_flip(), the author names were displayed in reverse alphabetical order. Using the factor reorder code plus the reorder in ggplot allowed me to display the chart in order by number of books then by author alphabetical order.<br />
<br />
We can also plot average rating by author, which can tell me a little more about how much I like particular authors. Let's plot those for any author who contributed at least 2 books to my dataset.<br />
<div class="chunk" id="unnamed-chunk-6">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">authors</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(Books</span> <span class="hl opt">></span> <span class="hl num">1</span><span class="hl std">)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(Author, AvgRating))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_x_discrete</span><span class="hl std">(</span><span class="hl kwc">labels</span><span class="hl std">=</span><span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">){</span><span class="hl kwd">sub</span><span class="hl std">(</span><span class="hl str">"\\s"</span><span class="hl std">,</span> <span class="hl str">"\n"</span><span class="hl std">, x)})</span> <span class="hl opt">+</span>
<span class="hl kwd">ylab</span><span class="hl std">(</span><span class="hl str">"Average Rating"</span><span class="hl std">)</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhL92Lzez2BjFLrdgR8tqggz1ahOJSCrtUF6Oh11hmKNtdwB8SiDYF5mi_ohOpyCeestYMZGhtACykc1rY_7jwkAvZZTFn4qDtWlc5ptgHFBR_H5H9pczxp6bYDPGr3J2F9RMSyY68eyrU/s1600/unnamed-chunk-6-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="504" data-original-width="504" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhL92Lzez2BjFLrdgR8tqggz1ahOJSCrtUF6Oh11hmKNtdwB8SiDYF5mi_ohOpyCeestYMZGhtACykc1rY_7jwkAvZZTFn4qDtWlc5ptgHFBR_H5H9pczxp6bYDPGr3J2F9RMSyY68eyrU/s320/unnamed-chunk-6-1.png" width="320" /></a></div>
<br /></div>
</div>
I only read 2 books by Ann Patchett, but I rated both of her books as 5, giving her the highest average rating. If I look at one of the authors who contributed more than 2 books, John Scalzi (tied for 3rd most read in 2019) has the highest rating, followed by Terry Pratchett (2nd most read). Obviously, though, I really like any of the authors I read at least 2 books from, because they all have fairly high average ratings. Stephen King is the only one with an average below 4, and that's only because I read <i>Cujo</i>, which I hated (more on that later on in this post).<br />
<br />
We can also look at how genre affected ratings. Using the genre labels I generated before, let's plot average rating.<br />
<div class="chunk" id="unnamed-chunk-7">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">genre</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">group_by</span><span class="hl std">(Fiction, Childrens, Fantasy, SciFi, Mystery)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">Books</span> <span class="hl std">=</span> <span class="hl kwd">n</span><span class="hl std">(),</span>
<span class="hl kwc">AvgRating</span> <span class="hl std">=</span> <span class="hl kwd">mean</span><span class="hl std">(MyRating))</span> <span class="hl opt">%>%</span>
<span class="hl kwd">bind_cols</span><span class="hl std">(</span><span class="hl kwc">Genre</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"Non-Fiction"</span><span class="hl std">,</span>
<span class="hl str">"General Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Mystery"</span><span class="hl std">,</span>
<span class="hl str">"Science Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Fantasy"</span><span class="hl std">,</span>
<span class="hl str">"Fantasy Sci-Fi"</span><span class="hl std">,</span>
<span class="hl str">"Children's Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Children's Fantasy"</span><span class="hl std">))</span>
<span class="hl std">genre</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(</span><span class="hl kwd">reorder</span><span class="hl std">(Genre,</span> <span class="hl kwd">desc</span><span class="hl std">(AvgRating)), AvgRating))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_x_discrete</span><span class="hl std">(</span><span class="hl kwc">labels</span><span class="hl std">=</span><span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">){</span><span class="hl kwd">sub</span><span class="hl std">(</span><span class="hl str">"\\s"</span><span class="hl std">,</span> <span class="hl str">"\n"</span><span class="hl std">, x)})</span> <span class="hl opt">+</span>
<span class="hl kwd">xlab</span><span class="hl std">(</span><span class="hl str">"Genre"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">ylab</span><span class="hl std">(</span><span class="hl str">"Average Rating"</span><span class="hl std">)</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizyK3fP8Y0bR5XyX0z5lpd6rgTXHZgVkgDJEC_l1QMa1yeCnDnPVLLUaFQpYRvnIrG3siXTq4kWGTjyAvrifIdkBa-FeDILa0gF6xzrglFwvetmuDPysw4ZYMGNsVlz9t-BkYaCIY0rWNs/s1600/unnamed-chunk-7-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="504" data-original-width="504" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizyK3fP8Y0bR5XyX0z5lpd6rgTXHZgVkgDJEC_l1QMa1yeCnDnPVLLUaFQpYRvnIrG3siXTq4kWGTjyAvrifIdkBa-FeDILa0gF6xzrglFwvetmuDPysw4ZYMGNsVlz9t-BkYaCIY0rWNs/s320/unnamed-chunk-7-1.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
Based on this plot, my favorite genres appear to be fantasy, sci-fi, and especially books with elements of both. No surprises here.<br />
<br /></div>
</div>
Let's dig into ratings on individual books. In my filter post, I identified the 25 books I liked the most (i.e., gave them a 5-star rating). What about the books I disliked? The lowest rating I gave was a 2, but it's safe to say I hated those books. And I also probably didn't like the books I rated as 3.<br />
<div class="chunk" id="unnamed-chunk-8">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">lowratings</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(MyRating</span> <span class="hl opt"><=</span> <span class="hl num">3</span><span class="hl std">)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">Rating</span> <span class="hl std">=</span> <span class="hl kwd">case_when</span><span class="hl std">(MyRating</span> <span class="hl opt">==</span> <span class="hl num">2</span> <span class="hl opt">~</span> <span class="hl str">"Hated"</span><span class="hl std">,</span>
<span class="hl std">MyRating</span> <span class="hl opt">==</span> <span class="hl num">3</span> <span class="hl opt">~</span> <span class="hl str">"Disliked"</span><span class="hl std">))</span> <span class="hl opt">%>%</span>
<span class="hl kwd">arrange</span><span class="hl std">(</span><span class="hl kwd">desc</span><span class="hl std">(MyRating), Author)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">select</span><span class="hl std">(Title, Author, Rating)</span>
<span class="hl kwd">library</span><span class="hl std">(expss)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">##
## Attaching package: 'expss'
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following objects are masked from 'package:stringr':
##
## fixed, regex
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following objects are masked from 'package:dplyr':
##
## between, compute, contains, first, last, na_if, recode, vars
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following objects are masked from 'package:purrr':
##
## keep, modify, modify_if, transpose
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following objects are masked from 'package:tidyr':
##
## contains, nest
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following object is masked from 'package:ggplot2':
##
## vars
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl kwd">as.etable</span><span class="hl std">(lowratings,</span> <span class="hl kwc">rownames_as_row_labels</span> <span class="hl std">=</span> <span class="hl num">FALSE</span><span class="hl std">)</span>
</pre>
</div>
<table class="gmisc_table" style="border-collapse: collapse; margin-bottom: 1em; margin-top: 1em;">
<thead>
<tr>
<th style="border-bottom: 1px solid grey; border-top: 2px solid grey; font-weight: 900; text-align: center;">Title</th>
<th style="border-bottom: 1px solid grey; border-top: 2px solid grey; text-align: center;"> Author </th>
<th style="border-bottom: 1px solid grey; border-top: 2px solid grey; text-align: center;"> Rating </th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align: left;"> The Scarecrow of Oz (Oz, #9) </td>
<td style="text-align: right;">Baum, L. Frank</td>
<td style="text-align: right;">Disliked</td>
</tr>
<tr>
<td style="text-align: left;"> The Tin Woodman of Oz (Oz, #12) </td>
<td style="text-align: right;">Baum, L. Frank</td>
<td style="text-align: right;">Disliked</td>
</tr>
<tr>
<td style="text-align: left;"> Herself Surprised </td>
<td style="text-align: right;">Cary, Joyce</td>
<td style="text-align: right;">Disliked</td>
</tr>
<tr>
<td style="text-align: left;"> The 5 Love Languages: The Secret to Love That Lasts </td>
<td style="text-align: right;">Chapman, Gary</td>
<td style="text-align: right;">Disliked</td>
</tr>
<tr>
<td style="text-align: left;"> Boundaries: When to Say Yes, How to Say No to Take Control of Your Life </td>
<td style="text-align: right;">Cloud, Henry</td>
<td style="text-align: right;">Disliked</td>
</tr>
<tr>
<td style="text-align: left;"> Summerdale </td>
<td style="text-align: right;">Collins, David Jay</td>
<td style="text-align: right;">Disliked</td>
</tr>
<tr>
<td style="text-align: left;"> When We Were Orphans </td>
<td style="text-align: right;">Ishiguro, Kazuo</td>
<td style="text-align: right;">Disliked</td>
</tr>
<tr>
<td style="text-align: left;"> Bird Box (Bird Box, #1) </td>
<td style="text-align: right;">Malerman, Josh</td>
<td style="text-align: right;">Disliked</td>
</tr>
<tr>
<td style="text-align: left;"> Oz in Perspective: Magic and Myth in the L. Frank Baum Books </td>
<td style="text-align: right;">Tuerk, Richard</td>
<td style="text-align: right;">Disliked</td>
</tr>
<tr>
<td style="text-align: left;"> Cujo </td>
<td style="text-align: right;">King, Stephen</td>
<td style="text-align: right;">Hated</td>
</tr>
<tr>
<td style="border-bottom: 2px solid grey; text-align: left;"> Just Evil (Evil Secrets Trilogy, #1) </td>
<td style="border-bottom: 2px solid grey; text-align: right;">McKeehan, Vickie</td>
<td style="border-bottom: 2px solid grey; text-align: right;">Hated</td>
</tr>
</tbody>
</table>
</div>
</div>
I'm a little surprised at some of this, because several books I rated as 3 I liked and only a few I legitimately didn't like. The 2 books I rated as 2 I really did hate, and probably should have rated as 1 instead. So based on my new understanding of how I've been using (misusing) those ratings, I'd probably update 3 ratings.<br />
<div class="chunk" id="unnamed-chunk-9">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">MyRating</span> <span class="hl std">=</span> <span class="hl kwd">replace</span><span class="hl std">(MyRating,</span>
<span class="hl std">MyRating</span> <span class="hl opt">==</span> <span class="hl num">2</span><span class="hl std">,</span> <span class="hl num">1</span><span class="hl std">),</span>
<span class="hl kwc">MyRating</span> <span class="hl std">=</span> <span class="hl kwd">replace</span><span class="hl std">(MyRating,</span>
<span class="hl std">Title</span> <span class="hl opt">==</span> <span class="hl str">"Herself Surprised"</span><span class="hl std">,</span> <span class="hl num">2</span><span class="hl std">))</span>
<span class="hl std">lowratings</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(MyRating</span> <span class="hl opt"><=</span> <span class="hl num">2</span><span class="hl std">)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">Rating</span> <span class="hl std">=</span> <span class="hl kwd">case_when</span><span class="hl std">(MyRating</span> <span class="hl opt">==</span> <span class="hl num">1</span> <span class="hl opt">~</span> <span class="hl str">"Hated"</span><span class="hl std">,</span>
<span class="hl std">MyRating</span> <span class="hl opt">==</span> <span class="hl num">2</span> <span class="hl opt">~</span> <span class="hl str">"Disliked"</span><span class="hl std">))</span> <span class="hl opt">%>%</span>
<span class="hl kwd">arrange</span><span class="hl std">(</span><span class="hl kwd">desc</span><span class="hl std">(MyRating), Author)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">select</span><span class="hl std">(Title, Author, Rating)</span>
<span class="hl kwd">library</span><span class="hl std">(expss)</span>
<span class="hl kwd">as.etable</span><span class="hl std">(lowratings,</span> <span class="hl kwc">rownames_as_row_labels</span> <span class="hl std">=</span> <span class="hl num">FALSE</span><span class="hl std">)</span>
</pre>
</div>
<table class="gmisc_table" style="border-collapse: collapse; margin-bottom: 1em; margin-top: 1em;">
<thead>
<tr>
<th style="border-bottom: 1px solid grey; border-top: 2px solid grey; font-weight: 900; text-align: center;">Title</th>
<th style="border-bottom: 1px solid grey; border-top: 2px solid grey; text-align: center;"> Author </th>
<th style="border-bottom: 1px solid grey; border-top: 2px solid grey; text-align: center;"> Rating </th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align: left;"> Herself Surprised </td>
<td style="text-align: right;">Cary, Joyce</td>
<td style="text-align: right;">Disliked</td>
</tr>
<tr>
<td style="text-align: left;"> Cujo </td>
<td style="text-align: right;">King, Stephen</td>
<td style="text-align: right;">Hated</td>
</tr>
<tr>
<td style="border-bottom: 2px solid grey; text-align: left;"> Just Evil (Evil Secrets Trilogy, #1) </td>
<td style="border-bottom: 2px solid grey; text-align: right;">McKeehan, Vickie</td>
<td style="border-bottom: 2px solid grey; text-align: right;">Hated</td>
</tr>
</tbody>
</table>
</div>
</div>
There! Now I have a much more accurate representation of the books I actually disliked/hated, and know how I should be rating books going forward to better reflect how I think of the categories. Of the two I hated, <i>Just Evil...</i> was an e-book I won in a Goodreads giveaway that I read on my phone when I didn't have a physical book with me: convoluted storyline, problematic romantic relationships, and a main character who talked about how much her dog was her baby, and yet the dog was forgotten half the time (even left alone for long periods of time while she was off having her problematic relationship) except when the dog's reaction or protection became important to the storyline. The other, <i>Cujo</i>, I reviewed <a href="http://www.deeplytrivial.com/2019/09/totally-superfluous-book-review-stephen.html">here</a>; while I'm glad I read it, I have no desire to ever read it again.<br />
<br />
Let's look again at my top books, but this time, classify them by long genre descriptions from above. I can get that information into my full reading dataset with a join, using the genre flags. Then I can plot the results from that dataset without having to summarize first.<br />
<div class="chunk" id="unnamed-chunk-10">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">topbygenre</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">left_join</span><span class="hl std">(genre,</span> <span class="hl kwc">by</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"Fiction"</span><span class="hl std">,</span><span class="hl str">"Childrens"</span><span class="hl std">,</span><span class="hl str">"Fantasy"</span><span class="hl std">,</span><span class="hl str">"SciFi"</span><span class="hl std">,</span><span class="hl str">"Mystery"</span><span class="hl std">))</span> <span class="hl opt">%>%</span>
<span class="hl kwd">select</span><span class="hl std">(</span><span class="hl opt">-</span><span class="hl std">Books,</span> <span class="hl opt">-</span><span class="hl std">AvgRating)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(MyRating</span> <span class="hl opt">==</span> <span class="hl num">5</span><span class="hl std">)</span>
<span class="hl std">topbygenre</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(</span><span class="hl kwd">fct_infreq</span><span class="hl std">(Genre)))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_bar</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_x_discrete</span><span class="hl std">(</span><span class="hl kwc">labels</span><span class="hl std">=</span><span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">){</span><span class="hl kwd">sub</span><span class="hl std">(</span><span class="hl str">"\\s"</span><span class="hl std">,</span> <span class="hl str">"\n"</span><span class="hl std">, x)})</span> <span class="hl opt">+</span>
<span class="hl kwd">xlab</span><span class="hl std">(</span><span class="hl str">"Genre"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">ylab</span><span class="hl std">(</span><span class="hl str">"Books"</span><span class="hl std">)</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdHUfo7JieZwTna7POfMRw2yjGRqqmik4qToFFcmyoASdyx8mWsLdpNpX8C_uAozLw3qP9tvytgNVHmv2Q_VzAHnC-D2LwIkk3ZNbeWaaPmSu3sqeocASAmlMbgJqllarfsD9fNX5Hmcbt/s1600/unnamed-chunk-10-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="504" data-original-width="504" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdHUfo7JieZwTna7POfMRw2yjGRqqmik4qToFFcmyoASdyx8mWsLdpNpX8C_uAozLw3qP9tvytgNVHmv2Q_VzAHnC-D2LwIkk3ZNbeWaaPmSu3sqeocASAmlMbgJqllarfsD9fNX5Hmcbt/s320/unnamed-chunk-10-1.png" width="320" /></a></div>
<br /></div>
</div>
This chart helps me to better understand my average rating by genre chart above. Only 1 book with elements of both fantasy and sci-fi was rated as a 5, and the average rating is 4.5, meaning there's only 1 other book in that category that had to be rated as a 4. It might be a good idea to either filter my genre rating table to categories with more than 1 book, <i>or</i> add the counts as labels to that plot. Let's try the latter.<br />
<div class="chunk" id="unnamed-chunk-11">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">genre</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(</span><span class="hl kwd">reorder</span><span class="hl std">(Genre,</span> <span class="hl kwd">desc</span><span class="hl std">(AvgRating)), AvgRating,</span> <span class="hl kwc">label</span> <span class="hl std">= Books))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_x_discrete</span><span class="hl std">(</span><span class="hl kwc">labels</span><span class="hl std">=</span><span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">){</span><span class="hl kwd">sub</span><span class="hl std">(</span><span class="hl str">"\\s"</span><span class="hl std">,</span> <span class="hl str">"\n"</span><span class="hl std">, x)})</span> <span class="hl opt">+</span>
<span class="hl kwd">xlab</span><span class="hl std">(</span><span class="hl str">"Genre"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">ylab</span><span class="hl std">(</span><span class="hl str">"Average Rating"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_text</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(</span><span class="hl kwc">x</span> <span class="hl std">= Genre,</span> <span class="hl kwc">y</span> <span class="hl std">= AvgRating</span><span class="hl opt">-</span><span class="hl num">0.25</span><span class="hl std">),</span> <span class="hl kwc">size</span> <span class="hl std">=</span> <span class="hl num">5</span><span class="hl std">,</span>
<span class="hl kwc">color</span> <span class="hl std">=</span> <span class="hl str">"white"</span><span class="hl std">)</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgI8hn-0KeHzAUy-xOMCwZBOf2HZmvO-x8vtK7SHpYkvEKLlMuX0RUvhM0Vd2yiJBATdMtI7M-v0bOrLFYTcehz4_imk-Y2_-lbDZpMWJU1LeOmeOGJ2gAPrXaBWLLVPxCdVpDfpXX5KJMu/s1600/unnamed-chunk-11-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="504" data-original-width="504" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgI8hn-0KeHzAUy-xOMCwZBOf2HZmvO-x8vtK7SHpYkvEKLlMuX0RUvhM0Vd2yiJBATdMtI7M-v0bOrLFYTcehz4_imk-Y2_-lbDZpMWJU1LeOmeOGJ2gAPrXaBWLLVPxCdVpDfpXX5KJMu/s320/unnamed-chunk-11-1.png" width="320" /></a></div>
<br /></div>
</div>
Let's redo this chart, excluding those genres with only 1 or 2 books represented.<br />
<div class="chunk" id="unnamed-chunk-12">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">genre</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(Books</span> <span class="hl opt">></span> <span class="hl num">2</span><span class="hl std">)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(</span><span class="hl kwd">reorder</span><span class="hl std">(Genre,</span> <span class="hl kwd">desc</span><span class="hl std">(AvgRating)), AvgRating,</span> <span class="hl kwc">label</span> <span class="hl std">= Books))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_x_discrete</span><span class="hl std">(</span><span class="hl kwc">labels</span><span class="hl std">=</span><span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">){</span><span class="hl kwd">sub</span><span class="hl std">(</span><span class="hl str">"\\s"</span><span class="hl std">,</span> <span class="hl str">"\n"</span><span class="hl std">, x)})</span> <span class="hl opt">+</span>
<span class="hl kwd">xlab</span><span class="hl std">(</span><span class="hl str">"Genre"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">ylab</span><span class="hl std">(</span><span class="hl str">"Average Rating"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_text</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(</span><span class="hl kwc">x</span> <span class="hl std">= Genre,</span> <span class="hl kwc">y</span> <span class="hl std">= AvgRating</span><span class="hl opt">-</span><span class="hl num">0.25</span><span class="hl std">),</span> <span class="hl kwc">size</span> <span class="hl std">=</span> <span class="hl num">5</span><span class="hl std">,</span>
<span class="hl kwc">color</span> <span class="hl std">=</span> <span class="hl str">"white"</span><span class="hl std">)</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhmAS5lH4XBU57LAdwEbHeX9EvHo7eyzbrNfKM1xbYWg9JzNsPn87x0hAOYeNhwP_vhbNZEh2fM3MTyCwGBxjGZWsOBcyuW3K7a8zoL2XZfXuKXEZvdLBXcsqitSD7OFqp5HQfv2V2RoRGc/s1600/unnamed-chunk-12-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="504" data-original-width="504" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhmAS5lH4XBU57LAdwEbHeX9EvHo7eyzbrNfKM1xbYWg9JzNsPn87x0hAOYeNhwP_vhbNZEh2fM3MTyCwGBxjGZWsOBcyuW3K7a8zoL2XZfXuKXEZvdLBXcsqitSD7OFqp5HQfv2V2RoRGc/s320/unnamed-chunk-12-1.png" width="320" /></a></div>
<br /></div>
</div>
While I love both science fiction and fantasy - reading equal numbers of books in those genres - I seem to like science fiction a bit more, based on the slightly higher average rating.Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-4594832939334410220.post-72404333876218717862020-04-30T09:00:00.000-05:002020-04-30T09:00:01.283-05:00Z is for Additional AxesHere we are at the last post in Blogging A to Z! Today, I want to talk about adding additional axes to your ggplot, using the options for fill or color. While these aren't true z-axes in the geometric sense, I think of them as a third, z, axis.<br />
<br />
Some of you may be surprised to learn that fill and color are different, and that you could use one or both in a given plot.<br />
<br />
Color refers to the outline of the object (bar, piechart wedge, etc.), while fill refers to the inside of the object. For scatterplots, the default shape doesn't have a fill, so you'd just use color to change the appearance of those points.<br />
<br />
Let's recreate the pages read over 2019 chart, but this time, I'll just use fiction books and separate them as either fantasy or other fiction; this divides that dataset pretty evenly in half. Here's how I'd generate the pages read over time separately by those two genre categories.<br />
<div class="chunk" id="unnamed-chunk-1">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(tidyverse)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --
</pre>
</div>
<div class="message">
<pre class="knitr r">## <U+2713> ggplot2 3.2.1 <U+2713> purrr 0.3.3
## <U+2713> tibble 2.1.3 <U+2713> dplyr 0.8.3
## <U+2713> tidyr 1.0.0 <U+2713> stringr 1.4.0
## <U+2713> readr 1.3.1 <U+2713> forcats 0.4.0
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl kwb"><-</span> <span class="hl kwd">read_csv</span><span class="hl std">(</span><span class="hl str">"~/Downloads/Blogging A to Z/SaraReads2019_allchanges.csv"</span><span class="hl std">,</span>
<span class="hl kwc">col_names</span> <span class="hl std">=</span> <span class="hl num">TRUE</span><span class="hl std">)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## Parsed with column specification:
## cols(
## Title = col_character(),
## Pages = col_double(),
## date_started = col_character(),
## date_read = col_character(),
## Book.ID = col_double(),
## Author = col_character(),
## AdditionalAuthors = col_character(),
## AverageRating = col_double(),
## OriginalPublicationYear = col_double(),
## read_time = col_double(),
## MyRating = col_double(),
## Gender = col_double(),
## Fiction = col_double(),
## Childrens = col_double(),
## Fantasy = col_double(),
## SciFi = col_double(),
## Mystery = col_double(),
## SelfHelp = col_double()
## )
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">fantasy</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(Fiction</span> <span class="hl opt">==</span> <span class="hl num">1</span><span class="hl std">)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">date_read</span> <span class="hl std">=</span> <span class="hl kwd">as.Date</span><span class="hl std">(date_read,</span> <span class="hl kwc">format</span> <span class="hl std">=</span> <span class="hl str">'%m/%d/%Y'</span><span class="hl std">),</span>
<span class="hl kwc">Fantasy</span> <span class="hl std">=</span> <span class="hl kwd">factor</span><span class="hl std">(Fantasy,</span> <span class="hl kwc">levels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl num">0</span><span class="hl std">,</span><span class="hl num">1</span><span class="hl std">),</span>
<span class="hl kwc">labels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"Other Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Fantasy"</span><span class="hl std">)))</span> <span class="hl opt">%>%</span>
<span class="hl kwd">group_by</span><span class="hl std">(Fantasy)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">GenreRead</span> <span class="hl std">=</span> <span class="hl kwd">order_by</span><span class="hl std">(date_read,</span> <span class="hl kwd">cumsum</span><span class="hl std">(Pages)))</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ungroup</span><span class="hl std">()</span>
</pre>
</div>
</div>
</div>
Now I'd just plug that information into my ggplot code, but add a third variable in the aesthetics (aes) for ggplot - color = Fantasy.<br />
<div class="chunk" id="unnamed-chunk-2">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(scales)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">##
## Attaching package: 'scales'
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following object is masked from 'package:purrr':
##
## discard
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following object is masked from 'package:readr':
##
## col_factor
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">myplot</span> <span class="hl kwb"><-</span> <span class="hl std">fantasy</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(date_read, GenreRead,</span> <span class="hl kwc">color</span> <span class="hl std">= Fantasy))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_point</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">xlab</span><span class="hl std">(</span><span class="hl str">"Date"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">ylab</span><span class="hl std">(</span><span class="hl str">"Pages"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_x_date</span><span class="hl std">(</span><span class="hl kwc">date_labels</span> <span class="hl std">=</span> <span class="hl str">"%b"</span><span class="hl std">,</span>
<span class="hl kwc">date_breaks</span> <span class="hl std">=</span> <span class="hl str">"1 month"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_y_continuous</span><span class="hl std">(</span><span class="hl kwc">labels</span> <span class="hl std">= comma,</span> <span class="hl kwc">breaks</span> <span class="hl std">=</span> <span class="hl kwd">seq</span><span class="hl std">(</span><span class="hl num">0</span><span class="hl std">,</span><span class="hl num">30000</span><span class="hl std">,</span><span class="hl num">5000</span><span class="hl std">))</span> <span class="hl opt">+</span>
<span class="hl kwd">labs</span><span class="hl std">(</span><span class="hl kwc">color</span> <span class="hl std">=</span> <span class="hl str">"Genre of Fiction"</span><span class="hl std">)</span>
</pre>
</div>
</div>
</div>
This plot uses the default R colorscheme. I could change those colors, using an existing colorscheme, or define my own. Let's make a fivethirtyeight style figure, using their theme for the overall plot, and their color scheme for the genre variable.<br />
<div class="chunk" id="unnamed-chunk-3">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(ggthemes)</span>
</pre>
</div>
<div class="warning">
<pre class="knitr r">## Warning: package 'ggthemes' was built under R version 3.6.3
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">myplot</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_color_fivethirtyeight</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">theme_fivethirtyeight</span><span class="hl std">()</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSeSO3gm5Fk7WKJDApRkUjBV2AVUx8wZz2oIWdUyV1KxrV-ZDiJ7vNGmZe6qJpOHFrI-7QabJCcMwbk_tUVbgLFw-BLgqc8PF7tCw0j-XaXIQXn_CSFJz56K0EfW2yFcSgnBO8ktow2KfP/s1600/unnamed-chunk-3-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="504" height="274" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSeSO3gm5Fk7WKJDApRkUjBV2AVUx8wZz2oIWdUyV1KxrV-ZDiJ7vNGmZe6qJpOHFrI-7QabJCcMwbk_tUVbgLFw-BLgqc8PF7tCw0j-XaXIQXn_CSFJz56K0EfW2yFcSgnBO8ktow2KfP/s320/unnamed-chunk-3-1.png" width="320" /></a></div>
<br /></div>
</div>
I can also specify my own colors.<br />
<div class="chunk" id="unnamed-chunk-4">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">myplot</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_color_manual</span><span class="hl std">(</span><span class="hl kwc">values</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"#4b0082"</span><span class="hl std">,</span><span class="hl str">"#ffd700"</span><span class="hl std">))</span> <span class="hl opt">+</span>
<span class="hl kwd">theme_minimal</span><span class="hl std">()</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNnfaPFNDcbuCXnpXbR8g_AC44lqQ5_qFWV585UP6qFCmHb2GX1YUnoTpfbF5Gmk7eTYPs0e1zrYN_emkOh7ZGp7rK3sZeNKRo_dVpr3aDbDE85WdP7FL995_nZgr7FI-UHGtVoYBfj0xy/s1600/unnamed-chunk-4-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="504" height="274" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNnfaPFNDcbuCXnpXbR8g_AC44lqQ5_qFWV585UP6qFCmHb2GX1YUnoTpfbF5Gmk7eTYPs0e1zrYN_emkOh7ZGp7rK3sZeNKRo_dVpr3aDbDE85WdP7FL995_nZgr7FI-UHGtVoYBfj0xy/s320/unnamed-chunk-4-1.png" width="320" /></a></div>
<br /></div>
</div>
The geom_point offers many point shapes; 21-25 allow you to specify both color and fill. But for the rest, only use color.<br />
<div class="chunk" id="unnamed-chunk-5">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(ggpubr)</span>
</pre>
</div>
<div class="warning">
<pre class="knitr r">## Warning: package 'ggpubr' was built under R version 3.6.3
</pre>
</div>
<div class="message">
<pre class="knitr r">## Loading required package: magrittr
</pre>
</div>
<div class="message">
<pre class="knitr r">##
## Attaching package: 'magrittr'
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following object is masked from 'package:purrr':
##
## set_names
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following object is masked from 'package:tidyr':
##
## extract
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">ggpubr</span><span class="hl opt">::</span><span class="hl kwd">show_point_shapes</span><span class="hl std">()</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## Scale for 'y' is already present. Adding another scale for 'y', which will
## replace the existing scale.
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMofal4Sd-IokKx1heheF0GVu6_PPQRl9rCsmeRRxc9tBdW-eF9UDYruN0Gx5ZRY8E8QVdU3scayTb6Gq3EuA5fn8UYRpLWqt3epWJZ_08Qc63G2nBRrflNiSgzolY3kxX9deUhpxcEsW6/s1600/unnamed-chunk-5-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="504" data-original-width="504" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMofal4Sd-IokKx1heheF0GVu6_PPQRl9rCsmeRRxc9tBdW-eF9UDYruN0Gx5ZRY8E8QVdU3scayTb6Gq3EuA5fn8UYRpLWqt3epWJZ_08Qc63G2nBRrflNiSgzolY3kxX9deUhpxcEsW6/s320/unnamed-chunk-5-1.png" width="320" /></a></div>
<br /></div>
</div>
Of course, you may have plots where changing fill is best, such as on a bar plot. In my summarize example, I created a stacked bar chart of fiction versus non-fiction with author gender as the fill.<br />
<div class="chunk" id="unnamed-chunk-6">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">Gender</span> <span class="hl std">=</span> <span class="hl kwd">factor</span><span class="hl std">(Gender,</span> <span class="hl kwc">levels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl num">0</span><span class="hl std">,</span><span class="hl num">1</span><span class="hl std">),</span>
<span class="hl kwc">labels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"Male"</span><span class="hl std">,</span>
<span class="hl str">"Female"</span><span class="hl std">)),</span>
<span class="hl kwc">Fiction</span> <span class="hl std">=</span> <span class="hl kwd">factor</span><span class="hl std">(Fiction,</span> <span class="hl kwc">levels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl num">0</span><span class="hl std">,</span><span class="hl num">1</span><span class="hl std">),</span>
<span class="hl kwc">labels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"Non-Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Fiction"</span><span class="hl std">),</span>
<span class="hl kwc">ordered</span> <span class="hl std">=</span> <span class="hl num">TRUE</span><span class="hl std">))</span> <span class="hl opt">%>%</span>
<span class="hl kwd">group_by</span><span class="hl std">(Gender, Fiction)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">Books</span> <span class="hl std">=</span> <span class="hl kwd">n</span><span class="hl std">())</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(Fiction, Books,</span> <span class="hl kwc">fill</span> <span class="hl std">=</span> <span class="hl kwd">reorder</span><span class="hl std">(Gender,</span> <span class="hl kwd">desc</span><span class="hl std">(Gender))))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_fill_economist</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">xlab</span><span class="hl std">(</span><span class="hl str">"Genre"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">labs</span><span class="hl std">(</span><span class="hl kwc">fill</span> <span class="hl std">=</span> <span class="hl str">"Author Gender"</span><span class="hl std">)</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwlEDigDFHNCSXlvssnCVWhJRQ98pyBGe6di2hb0MkPyjtSs2xheGY2qJkZHggDjEkBzj8N2o85qVLQTzQ-RGYJeytQPjmntckABTKskUEjFxD-cJ4WlLQjvMAmB-qepxBkp2nqOVKZo3V/s1600/unnamed-chunk-6-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="504" height="274" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwlEDigDFHNCSXlvssnCVWhJRQ98pyBGe6di2hb0MkPyjtSs2xheGY2qJkZHggDjEkBzj8N2o85qVLQTzQ-RGYJeytQPjmntckABTKskUEjFxD-cJ4WlLQjvMAmB-qepxBkp2nqOVKZo3V/s320/unnamed-chunk-6-1.png" width="320" /></a></div>
<br /></div>
</div>
Stacking is the default, but I could also have the bars next to each other.<br />
<div class="chunk" id="unnamed-chunk-7">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">Gender</span> <span class="hl std">=</span> <span class="hl kwd">factor</span><span class="hl std">(Gender,</span> <span class="hl kwc">levels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl num">0</span><span class="hl std">,</span><span class="hl num">1</span><span class="hl std">),</span>
<span class="hl kwc">labels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"Male"</span><span class="hl std">,</span>
<span class="hl str">"Female"</span><span class="hl std">)),</span>
<span class="hl kwc">Fiction</span> <span class="hl std">=</span> <span class="hl kwd">factor</span><span class="hl std">(Fiction,</span> <span class="hl kwc">levels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl num">0</span><span class="hl std">,</span><span class="hl num">1</span><span class="hl std">),</span>
<span class="hl kwc">labels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"Non-Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Fiction"</span><span class="hl std">),</span>
<span class="hl kwc">ordered</span> <span class="hl std">=</span> <span class="hl num">TRUE</span><span class="hl std">))</span> <span class="hl opt">%>%</span>
<span class="hl kwd">group_by</span><span class="hl std">(Gender, Fiction)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">Books</span> <span class="hl std">=</span> <span class="hl kwd">n</span><span class="hl std">())</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(Fiction, Books,</span> <span class="hl kwc">fill</span> <span class="hl std">=</span> <span class="hl kwd">reorder</span><span class="hl std">(Gender,</span> <span class="hl kwd">desc</span><span class="hl std">(Gender))))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">(</span><span class="hl kwc">position</span> <span class="hl std">=</span> <span class="hl str">"dodge"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_fill_economist</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">xlab</span><span class="hl std">(</span><span class="hl str">"Genre"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">labs</span><span class="hl std">(</span><span class="hl kwc">fill</span> <span class="hl std">=</span> <span class="hl str">"Author Gender"</span><span class="hl std">)</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7IbBQrqcjxyqgwL7watKIHkxH_xiFRsAY-djI8heZujSxtPxzNBP4p4dCJooVnKoG3C_Tao4pKMdVlPxcam-D9_pzHrkoHXSVmMwDrVRgGUlOG8aQkXlyfiNnSm2oFnwGTH9bbF5WNqqU/s1600/unnamed-chunk-7-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="504" height="274" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7IbBQrqcjxyqgwL7watKIHkxH_xiFRsAY-djI8heZujSxtPxzNBP4p4dCJooVnKoG3C_Tao4pKMdVlPxcam-D9_pzHrkoHXSVmMwDrVRgGUlOG8aQkXlyfiNnSm2oFnwGTH9bbF5WNqqU/s320/unnamed-chunk-7-1.png" width="320" /></a></div>
<br /></div>
</div>
You can also use fill (or color) with the same variable you used for x or y; that is, instead of having it be a third scale, it could add some color and separation to distinguish categories from the x or y variable. This is especially helpful if you have multiple categories being plotted, because it helps break up the wall of bars. If you do this, I'd recommend choosing a color palette with highly complementary colors, rather than highly contrasting ones; you probably also want to drop the legend, though, since the axis will also be labeled.<br />
<div class="chunk" id="unnamed-chunk-8">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">genres</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">group_by</span><span class="hl std">(Fiction, Childrens, Fantasy, SciFi, Mystery)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">Books</span> <span class="hl std">=</span> <span class="hl kwd">n</span><span class="hl std">())</span>
<span class="hl std">genres</span> <span class="hl kwb"><-</span> <span class="hl std">genres</span> <span class="hl opt">%>%</span>
<span class="hl kwd">bind_cols</span><span class="hl std">(</span><span class="hl kwc">Genre</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"Non-Fiction"</span><span class="hl std">,</span>
<span class="hl str">"General Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Mystery"</span><span class="hl std">,</span>
<span class="hl str">"Science Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Fantasy"</span><span class="hl std">,</span>
<span class="hl str">"Fantasy Sci-Fi"</span><span class="hl std">,</span>
<span class="hl str">"Children's Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Children's Fantasy"</span><span class="hl std">))</span>
<span class="hl std">genres</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(Genre</span> <span class="hl opt">!=</span> <span class="hl str">"Non-Fiction"</span><span class="hl std">)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(</span><span class="hl kwd">reorder</span><span class="hl std">(Genre,</span> <span class="hl opt">-</span><span class="hl std">Books), Books,</span> <span class="hl kwc">fill</span> <span class="hl std">= Genre))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">xlab</span><span class="hl std">(</span><span class="hl str">"Genre"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_x_discrete</span><span class="hl std">(</span><span class="hl kwc">labels</span><span class="hl std">=</span><span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">){</span><span class="hl kwd">sub</span><span class="hl std">(</span><span class="hl str">"\\s"</span><span class="hl std">,</span> <span class="hl str">"\n"</span><span class="hl std">, x)})</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_fill_economist</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">theme</span><span class="hl std">(</span><span class="hl kwc">legend.position</span> <span class="hl std">=</span> <span class="hl str">"none"</span><span class="hl std">)</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOKoQ_7-YC_YdN1Fa5FuhU4DJdpBYjgqxYOAI2hJgt96n0Bct4R-7vEXnBgmKqHCWodoLVq28ms0bjAwqzlVRToAgx4g54CXMyZ5XiNpmPppEDzyTZGbAa3ZBXXFwobNkP6ZNVkZm-IIpv/s1600/unnamed-chunk-8-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="504" height="274" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOKoQ_7-YC_YdN1Fa5FuhU4DJdpBYjgqxYOAI2hJgt96n0Bct4R-7vEXnBgmKqHCWodoLVq28ms0bjAwqzlVRToAgx4g54CXMyZ5XiNpmPppEDzyTZGbAa3ZBXXFwobNkP6ZNVkZm-IIpv/s320/unnamed-chunk-8-1.png" width="320" /></a></div>
<br /></div>
</div>
If you only have a couple categories and want to draw a contrast, that's when you can use contrasting shades: for instance, at work, when I plot performance on an item, I use red for incorrect and blue for correct, to maximize the contrast between the two performance levels for whatever data I'm presenting.<br />
<br />
I hope you enjoyed this series! There's so much more you can do with tidyverse than what I covered this month. Hopefully this has given you enough to get started and sparked your interest to learn more. Once again, I highly recommend checking out <i><a href="https://r4ds.had.co.nz/">R for Data Science</a></i>.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-46610299033776312012020-04-29T09:00:00.000-05:002020-04-29T09:00:00.887-05:00Y is for scale_yYesterday, I talked about scale_x. Today, I'll continue on that topic, focusing on the y-axis.<br />
<br />
The key to using any of the scale_ functions is to know what sort of data you're working with (e.g., date, continuous, discrete). Yesterday, I talked about scale_x_date and scale_x_discrete. We often put these types of data on the x-axis, while the y-axis is frequently used for counts. When displaying counts, we want to think about the major breaks that make sense, as well as any additional formatting to make them easier to read.<br />
<br />
<a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a>If I go back to my pages over time plot, you'll notice the major breaks are in the tens of thousands. We're generally used to seeing those values with a comma separating the thousands from the hundreds. I could add those to my plot like this (with a little help from the scales package).<br />
<div class="chunk" id="unnamed-chunk-1">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(tidyverse)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --
</pre>
</div>
<div class="message">
<pre class="knitr r">## <U+2713> ggplot2 3.2.1 <U+2713> purrr 0.3.3
## <U+2713> tibble 2.1.3 <U+2713> dplyr 0.8.3
## <U+2713> tidyr 1.0.0 <U+2713> stringr 1.4.0
## <U+2713> readr 1.3.1 <U+2713> forcats 0.4.0
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl kwb"><-</span> <span class="hl kwd">read_csv</span><span class="hl std">(</span><span class="hl str">"~/Downloads/Blogging A to Z/SaraReads2019_allchanges.csv"</span><span class="hl std">,</span>
<span class="hl kwc">col_names</span> <span class="hl std">=</span> <span class="hl num">TRUE</span><span class="hl std">)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## Parsed with column specification:
## cols(
## Title = col_character(),
## Pages = col_double(),
## date_started = col_character(),
## date_read = col_character(),
## Book.ID = col_double(),
## Author = col_character(),
## AdditionalAuthors = col_character(),
## AverageRating = col_double(),
## OriginalPublicationYear = col_double(),
## read_time = col_double(),
## MyRating = col_double(),
## Gender = col_double(),
## Fiction = col_double(),
## Childrens = col_double(),
## Fantasy = col_double(),
## SciFi = col_double(),
## Mystery = col_double(),
## SelfHelp = col_double()
## )
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">date_started</span> <span class="hl std">=</span> <span class="hl kwd">as.Date</span><span class="hl std">(reads2019</span><span class="hl opt">$</span><span class="hl std">date_started,</span> <span class="hl kwc">format</span> <span class="hl std">=</span> <span class="hl str">'%m/%d/%Y'</span><span class="hl std">),</span>
<span class="hl kwc">date_read</span> <span class="hl std">=</span> <span class="hl kwd">as.Date</span><span class="hl std">(date_read,</span> <span class="hl kwc">format</span> <span class="hl std">=</span> <span class="hl str">'%m/%d/%Y'</span><span class="hl std">),</span>
<span class="hl kwc">PagesRead</span> <span class="hl std">=</span> <span class="hl kwd">order_by</span><span class="hl std">(date_read,</span> <span class="hl kwd">cumsum</span><span class="hl std">(Pages)))</span>
<span class="hl kwd">library</span><span class="hl std">(scales)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">##
## Attaching package: 'scales'
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following object is masked from 'package:purrr':
##
## discard
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following object is masked from 'package:readr':
##
## col_factor
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(date_read, PagesRead))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_point</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_x_date</span><span class="hl std">(</span><span class="hl kwc">date_labels</span> <span class="hl std">=</span> <span class="hl str">"%B"</span><span class="hl std">,</span>
<span class="hl kwc">date_breaks</span> <span class="hl std">=</span> <span class="hl str">"1 month"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_y_continuous</span><span class="hl std">(</span><span class="hl kwc">labels</span> <span class="hl std">= comma)</span> <span class="hl opt">+</span>
<span class="hl kwd">labs</span><span class="hl std">(</span><span class="hl kwc">title</span> <span class="hl std">=</span> <span class="hl str">"Cumulative Pages Read Over 2019"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">theme</span><span class="hl std">(</span><span class="hl kwc">plot.title</span> <span class="hl std">=</span> <span class="hl kwd">element_text</span><span class="hl std">(</span><span class="hl kwc">hjust</span> <span class="hl std">=</span> <span class="hl num">0.5</span><span class="hl std">))</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhu3HD4M3A4UBDHrjshuDEXs7TN_1qPVIjf5gS30E2-wZmf0NWMgU-xKbPE6bu0BM7IggaS6iW_pCZCvn_hC8QrBJu7ugAXtB-VMpzodODUXEhMCAXDTK6XRWjDjrVO7KdR3EC1HoKOEuQh/s1600/unnamed-chunk-1-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="684" height="202" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhu3HD4M3A4UBDHrjshuDEXs7TN_1qPVIjf5gS30E2-wZmf0NWMgU-xKbPE6bu0BM7IggaS6iW_pCZCvn_hC8QrBJu7ugAXtB-VMpzodODUXEhMCAXDTK6XRWjDjrVO7KdR3EC1HoKOEuQh/s320/unnamed-chunk-1-1.png" width="320" /></a></div>
</div>
</div>
I could also add more major breaks.<br />
<div class="chunk" id="unnamed-chunk-2">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(date_read, PagesRead))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_point</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_x_date</span><span class="hl std">(</span><span class="hl kwc">date_labels</span> <span class="hl std">=</span> <span class="hl str">"%B"</span><span class="hl std">,</span>
<span class="hl kwc">date_breaks</span> <span class="hl std">=</span> <span class="hl str">"1 month"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_y_continuous</span><span class="hl std">(</span><span class="hl kwc">labels</span> <span class="hl std">= comma,</span>
<span class="hl kwc">breaks</span> <span class="hl std">=</span> <span class="hl kwd">seq</span><span class="hl std">(</span><span class="hl num">0</span><span class="hl std">,</span> <span class="hl num">30000</span><span class="hl std">,</span> <span class="hl num">5000</span><span class="hl std">))</span> <span class="hl opt">+</span>
<span class="hl kwd">labs</span><span class="hl std">(</span><span class="hl kwc">title</span> <span class="hl std">=</span> <span class="hl str">"Cumulative Pages Read Over 2019"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">theme</span><span class="hl std">(</span><span class="hl kwc">plot.title</span> <span class="hl std">=</span> <span class="hl kwd">element_text</span><span class="hl std">(</span><span class="hl kwc">hjust</span> <span class="hl std">=</span> <span class="hl num">0.5</span><span class="hl std">))</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHcSK4s4eAygSAk5Hh3ZBq-F58e_UXsJ3S2Tp9Kmz_lhHbDMqXuRtJpuNjdhucWLgswrSs94ti8TZYR_RHYuxf5xbRH4O1lkWE_U7KB85MncG0FYRtnAl_uFa5EWsE77vwe8Y1AJb4ts5L/s1600/unnamed-chunk-2-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="684" height="202" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHcSK4s4eAygSAk5Hh3ZBq-F58e_UXsJ3S2Tp9Kmz_lhHbDMqXuRtJpuNjdhucWLgswrSs94ti8TZYR_RHYuxf5xbRH4O1lkWE_U7KB85MncG0FYRtnAl_uFa5EWsE77vwe8Y1AJb4ts5L/s320/unnamed-chunk-2-1.png" width="320" /></a></div>
</div>
</div>
<a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a>The scales package offers other ways to format data besides the 3 I've shown in this series (log transformation, percent, and now continuous with comma). It also lets you format data with currency, bytes, ranks, and scientific notation.<br />
<br />
Last post tomorrow!Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-71524818801359110392020-04-28T09:00:00.000-05:002020-04-28T09:00:00.744-05:00X is for scale_xThese next two posts will deal with formatting scales in ggplot2 - x-axis, y-axis - so I'll try to limit the amount of overlap and repetition.<br />
<br />
Let's say I wanted to plot my reading over time, specifically as a cumulative sum of pages across the year. My x-axis will be a date. Since my reads2019 file initially formats my dates as character, I'll need to use my mutate code to turn them into dates, plus compute my cumulative sum of pages read.<br />
<div class="chunk" id="unnamed-chunk-1">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(tidyverse)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --
</pre>
</div>
<div class="message">
<pre class="knitr r">## <U+2713> ggplot2 3.2.1 <U+2713> purrr 0.3.3
## <U+2713> tibble 2.1.3 <U+2713> dplyr 0.8.3
## <U+2713> tidyr 1.0.0 <U+2713> stringr 1.4.0
## <U+2713> readr 1.3.1 <U+2713> forcats 0.4.0
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl kwb"><-</span> <span class="hl kwd">read_csv</span><span class="hl std">(</span><span class="hl str">"~/Downloads/Blogging A to Z/SaraReads2019_allchanges.csv"</span><span class="hl std">,</span>
<span class="hl kwc">col_names</span> <span class="hl std">=</span> <span class="hl num">TRUE</span><span class="hl std">)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## Parsed with column specification:
## cols(
## Title = col_character(),
## Pages = col_double(),
## date_started = col_character(),
## date_read = col_character(),
## Book.ID = col_double(),
## Author = col_character(),
## AdditionalAuthors = col_character(),
## AverageRating = col_double(),
## OriginalPublicationYear = col_double(),
## read_time = col_double(),
## MyRating = col_double(),
## Gender = col_double(),
## Fiction = col_double(),
## Childrens = col_double(),
## Fantasy = col_double(),
## SciFi = col_double(),
## Mystery = col_double(),
## SelfHelp = col_double()
## )
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">date_started</span> <span class="hl std">=</span> <span class="hl kwd">as.Date</span><span class="hl std">(reads2019</span><span class="hl opt">$</span><span class="hl std">date_started,</span> <span class="hl kwc">format</span> <span class="hl std">=</span> <span class="hl str">'%m/%d/%Y'</span><span class="hl std">),</span>
<span class="hl kwc">date_read</span> <span class="hl std">=</span> <span class="hl kwd">as.Date</span><span class="hl std">(date_read,</span> <span class="hl kwc">format</span> <span class="hl std">=</span> <span class="hl str">'%m/%d/%Y'</span><span class="hl std">),</span>
<span class="hl kwc">PagesRead</span> <span class="hl std">=</span> <span class="hl kwd">order_by</span><span class="hl std">(date_read,</span> <span class="hl kwd">cumsum</span><span class="hl std">(Pages)))</span>
</pre>
</div>
</div>
</div>
This gives me the variables I need to plot my pages read over time.<br />
<div class="chunk" id="unnamed-chunk-2">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(date_read, PagesRead))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_point</span><span class="hl std">()</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMjl78uMYGt8gB3TMvtUvDSziVUyitUFbwg0axfHljZrm6L1gZcbqCYAkmdcxbORwPDjopABbICWsybq75u-aZ-9Il7pBfH-DT2EePU4D274imVGpVe_hXc0XeC2hQ2n4f-Vu2mYyf5FZu/s1600/unnamed-chunk-2-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="504" height="274" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMjl78uMYGt8gB3TMvtUvDSziVUyitUFbwg0axfHljZrm6L1gZcbqCYAkmdcxbORwPDjopABbICWsybq75u-aZ-9Il7pBfH-DT2EePU4D274imVGpVe_hXc0XeC2hQ2n4f-Vu2mYyf5FZu/s320/unnamed-chunk-2-1.png" width="320" /></a></div>
<br /></div>
</div>
ggplot2 did a fine job of creating this plot using default settings. Since my date_read variable is a date, the plot automatically ordered date_read, formatted as "Month Year", and used quarters as breaks. But we can still use the scale_x functions to make this plot look even better.<br />
<br />
One way could be to format years as 2-digit instead of 4. We could also have month breaks instead of quarters.<br />
<div class="chunk" id="unnamed-chunk-3">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(date_read, PagesRead))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_point</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_x_date</span><span class="hl std">(</span><span class="hl kwc">date_labels</span> <span class="hl std">=</span> <span class="hl str">"%b %y"</span><span class="hl std">,</span>
<span class="hl kwc">date_breaks</span> <span class="hl std">=</span> <span class="hl str">"1 month"</span><span class="hl std">)</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfVhDgiJMkImRj4igQwONcte8QDDK5uuMOZT21F8LFhrrL50hBMFLif1FBOfxoxQkji55TiGzkt8sEh5F6P0sQlbkHUMCQM-zs11Rf81-ExXWnZju4aksDiPi0mKp943mfw4Dk-n3ZGWpm/s1600/unnamed-chunk-3-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="504" height="274" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfVhDgiJMkImRj4igQwONcte8QDDK5uuMOZT21F8LFhrrL50hBMFLif1FBOfxoxQkji55TiGzkt8sEh5F6P0sQlbkHUMCQM-zs11Rf81-ExXWnZju4aksDiPi0mKp943mfw4Dk-n3ZGWpm/s320/unnamed-chunk-3-1.png" width="320" /></a></div>
<br /></div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"></a></div>
Of course, we could drop year completely and just show month, since all of this data is for 2019. We could then note that in the title instead.<br />
<div class="chunk" id="unnamed-chunk-4">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(date_read, PagesRead))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_point</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_x_date</span><span class="hl std">(</span><span class="hl kwc">date_labels</span> <span class="hl std">=</span> <span class="hl str">"%B"</span><span class="hl std">,</span>
<span class="hl kwc">date_breaks</span> <span class="hl std">=</span> <span class="hl str">"1 month"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">labs</span><span class="hl std">(</span><span class="hl kwc">title</span> <span class="hl std">=</span> <span class="hl str">"Cumulative Pages Read Over 2019"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">theme</span><span class="hl std">(</span><span class="hl kwc">plot.title</span> <span class="hl std">=</span> <span class="hl kwd">element_text</span><span class="hl std">(</span><span class="hl kwc">hjust</span> <span class="hl std">=</span> <span class="hl num">0.5</span><span class="hl std">))</span>
</pre>
<pre class="knitr r"><span class="hl std">
</span></pre>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj29_Y_ElUwiwHyrlKZ0ojYsEN_mlrxGgCQcPoFSTnGDj59aGo9J9GUkmo0KROGU9O_3m7-dS0MvLIXrF-oT1CJgmaSCS6wGH_ziTppB_Cgdw-xLVFO1kZsStST8BS0LzSPWkh_PFWM8Eji/s1600/unnamed-chunk-4-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="684" height="202" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj29_Y_ElUwiwHyrlKZ0ojYsEN_mlrxGgCQcPoFSTnGDj59aGo9J9GUkmo0KROGU9O_3m7-dS0MvLIXrF-oT1CJgmaSCS6wGH_ziTppB_Cgdw-xLVFO1kZsStST8BS0LzSPWkh_PFWM8Eji/s320/unnamed-chunk-4-1.png" width="320" /></a></div>
<pre class="knitr r"><span class="hl std">
</span></pre>
</div>
</div>
<div class="rimage default">
<div class="" style="clear: both; text-align: left;">
<a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a><a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a><a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a><a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="" style="clear: both; text-align: left;">
Tomorrow, I'll show some tricks for how we can format the y-axis of this plot. But let's see what else we can do to the x-axis. Let's create a bar graph with my genre data. I'll use the genre names I created for my summarized data last week.</div>
</div>
</div>
<div class="chunk" id="unnamed-chunk-5">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">genres</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">group_by</span><span class="hl std">(Fiction, Childrens, Fantasy, SciFi, Mystery)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">Books</span> <span class="hl std">=</span> <span class="hl kwd">n</span><span class="hl std">())</span>
<span class="hl std">genres</span> <span class="hl kwb"><-</span> <span class="hl std">genres</span> <span class="hl opt">%>%</span>
<span class="hl kwd">bind_cols</span><span class="hl std">(</span><span class="hl kwc">Genre</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"Non-Fiction"</span><span class="hl std">,</span>
<span class="hl str">"General Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Mystery"</span><span class="hl std">,</span>
<span class="hl str">"Science Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Fantasy"</span><span class="hl std">,</span>
<span class="hl str">"Fantasy Sci-Fi"</span><span class="hl std">,</span>
<span class="hl str">"Children's Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Children's Fantasy"</span><span class="hl std">))</span>
<span class="hl std">genres</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(Genre, Books))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">()</span>
</pre>
</div>
</div>
<div class="rimage default">
</div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjw7qlOwQH31XJ7BQAUJwN5eNW6QBTVnfzF_hWMZUkgG51EWfQu33Hb_sLlrZvQeSQVhS-BV0u_Q3DW6R5Op_lFEJw0dSXLUshlkR7jx2BOqPJbgYXedflail379j7_bUyGfVptXQYO_8Md/s1600/unnamed-chunk-5-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="504" height="274" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjw7qlOwQH31XJ7BQAUJwN5eNW6QBTVnfzF_hWMZUkgG51EWfQu33Hb_sLlrZvQeSQVhS-BV0u_Q3DW6R5Op_lFEJw0dSXLUshlkR7jx2BOqPJbgYXedflail379j7_bUyGfVptXQYO_8Md/s320/unnamed-chunk-5-1.png" width="320" /></a></div>
<br />
Unfortunately, my new genre names are a bit long, and overlap each other unless I make my plot really wide. There are a few ways I can deal with that. First, I could ask ggplot2 to abbreviate the names.<br />
<div class="chunk" id="unnamed-chunk-6">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">genres</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(Genre, Books))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_x_discrete</span><span class="hl std">(</span><span class="hl kwc">labels</span> <span class="hl std">= abbreviate)</span>
</pre>
</div>
</div>
<div class="rimage default">
</div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFfXGxn6kOywRQ-AalqdWdLc0RVotsbtUJOrZx3iZmbS6Ba5yzyy3ASekty77KFGrJH3UfD49cNlvz4MPuSIoAiGE0opCMgukp_f8tRXn0UmoxR2vX1vnwTo4UWEwxVRyrdXxxbTgasJUL/s1600/unnamed-chunk-6-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="504" height="274" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFfXGxn6kOywRQ-AalqdWdLc0RVotsbtUJOrZx3iZmbS6Ba5yzyy3ASekty77KFGrJH3UfD49cNlvz4MPuSIoAiGE0opCMgukp_f8tRXn0UmoxR2vX1vnwTo4UWEwxVRyrdXxxbTgasJUL/s320/unnamed-chunk-6-1.png" width="320" /></a></div>
<br />
These abbreviations were generated automatically by R, and I'm not a huge fan. A better way might be to add line breaks to any two-word genres. This <a href="https://stackoverflow.com/questions/20123147/add-line-break-to-axis-labels-and-ticks-in-ggplot">Stack Overflow post</a> gave me a function I can add to my scale_x_discrete to do just that.<br />
<div class="chunk" id="unnamed-chunk-7">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">genres</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(Genre, Books))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_x_discrete</span><span class="hl std">(</span><span class="hl kwc">labels</span><span class="hl std">=</span><span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">){</span><span class="hl kwd">sub</span><span class="hl std">(</span><span class="hl str">"\\s"</span><span class="hl std">,</span> <span class="hl str">"\n"</span><span class="hl std">, x)})</span>
</pre>
</div>
</div>
<div class="rimage default">
</div>
</div>
<a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><br /></a>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj68t3WAhUpaIalHKwPkRZwbHFMOFmtaiPlrkdXZ_SoBaVW42AZf6Sk8riz_h0fW3BkP7RjVKyCqZWC3p9LAB970axhoTXdOlYe6bunXIc2AdTVsK-0MmiJIPHFHiT0cN27o5HKDjrSv2gC/s1600/unnamed-chunk-7-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="432" data-original-width="504" height="274" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj68t3WAhUpaIalHKwPkRZwbHFMOFmtaiPlrkdXZ_SoBaVW42AZf6Sk8riz_h0fW3BkP7RjVKyCqZWC3p9LAB970axhoTXdOlYe6bunXIc2AdTVsK-0MmiJIPHFHiT0cN27o5HKDjrSv2gC/s320/unnamed-chunk-7-1.png" width="320" /></a></div>
<br />
MUCH better!<br />
<a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a><a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a><a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a><a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a><br />
As you can see, the scale_x function you use depends on the type of data you're working with. For dates, scale_x_date; for categories, scale_x_discrete. Tomorrow, we'll show some ways to format continuous data, since that's often what you see on the y-axis. See you then!<br />
<br />
By the way, this is my 1000th post on my blog!Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-74965545084402694622020-04-27T09:00:00.000-05:002020-04-27T09:00:08.570-05:00W is for Write and Read Data - FastOnce again, I'm dipping outside of the tidyverse, but this package and its functions have been really useful in getting data quickly in (and out) of R.<br />
<br />
For work, I have to pull in data from a few different sources, and manipulate and work with them to give me the final dataset that I use for much of my analysis. So that I don't have to go through all of that joining, recoding, and calculating each time, I created a final merged dataset as a CSV file that I can load when I need to continue my analysis. The problem is that the most recent version of that file, which contains 13 million+ records, was so large, writing it (and subsequently reading it in later) took forever and sometimes timed out.<br />
<br />
That's when I discovered the data.table library, and its fread and fwrite functions. Tidyverse is great for working with CSV files, but a lot of the memory and loading time is used for formatting. fread and fwrite are leaner and get the job done a bit faster. For regular-sized CSV files (like my reads2019 set), the time difference is pretty minimal. But for a 5GB datafile, it makes a huge difference.<br />
<div class="chunk" id="unnamed-chunk-1">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(tidyverse)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --
</pre>
</div>
<div class="message">
<pre class="knitr r">## <U+2713> ggplot2 3.2.1 <U+2713> purrr 0.3.3
## <U+2713> tibble 2.1.3 <U+2713> dplyr 0.8.3
## <U+2713> tidyr 1.0.0 <U+2713> stringr 1.4.0
## <U+2713> readr 1.3.1 <U+2713> forcats 0.4.0
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl kwd">system.time</span><span class="hl std">(reads2019</span> <span class="hl kwb"><-</span> <span class="hl kwd">read_csv</span><span class="hl std">(</span><span class="hl str">"~/Downloads/Blogging A to Z/SaraReads2019_allchanges.csv"</span><span class="hl std">,</span>
<span class="hl kwc">col_names</span> <span class="hl std">=</span> <span class="hl num">TRUE</span><span class="hl std">))</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## Parsed with column specification:
## cols(
## Title = col_character(),
## Pages = col_double(),
## date_started = col_character(),
## date_read = col_character(),
## Book.ID = col_double(),
## Author = col_character(),
## AdditionalAuthors = col_character(),
## AverageRating = col_double(),
## OriginalPublicationYear = col_double(),
## read_time = col_double(),
## MyRating = col_double(),
## Gender = col_double(),
## Fiction = col_double(),
## Childrens = col_double(),
## Fantasy = col_double(),
## SciFi = col_double(),
## Mystery = col_double(),
## SelfHelp = col_double()
## )
</pre>
</div>
<div class="output">
<pre class="knitr r">## user system elapsed
## 0.00 0.10 0.14
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl kwd">rm</span><span class="hl std">(reads2019)</span>
<span class="hl kwd">library</span><span class="hl std">(data.table)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">##
## Attaching package: 'data.table'
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following objects are masked from 'package:dplyr':
##
## between, first, last
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following object is masked from 'package:purrr':
##
## transpose
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl kwd">system.time</span><span class="hl std">(reads2019</span> <span class="hl kwb"><-</span> <span class="hl kwd">fread</span><span class="hl std">(</span><span class="hl str">"~/Downloads/Blogging A to Z/SaraReads2019_allchanges.csv"</span><span class="hl std">))</span>
</pre>
</div>
<div class="output">
<pre class="knitr r">## user system elapsed
## 0 0 0
</pre>
</div>
</div>
</div>
But let's show how long it took to read my work datafile. Here's the elapsed time from the system.time output.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">read_csv:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">user system elapsed</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">61.14 11.72 90.56</span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">fread:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">user system elapsed</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">57.97 16.40 57.19</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
But the real win is in how quickly this package writes CSV data. Using a package called wakefield, I'll randomly generate 10,000,000 records of survey data, then see how it takes to write the data to file using both write_csv and fwrite.
<br />
<div class="chunk" id="unnamed-chunk-2">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(wakefield)</span>
</pre>
</div>
<div class="warning">
<pre class="knitr r">## Warning: package 'wakefield' was built under R version 3.6.3
</pre>
</div>
<div class="message">
<pre class="knitr r">##
## Attaching package: 'wakefield'
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following objects are masked from 'package:data.table':
##
## hour, minute, month, second, year
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following object is masked from 'package:dplyr':
##
## id
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl kwd">set.seed</span><span class="hl std">(</span><span class="hl num">42</span><span class="hl std">)</span>
<span class="hl std">reallybigshew</span> <span class="hl kwb"><-</span> <span class="hl kwd">r_data_frame</span><span class="hl std">(</span><span class="hl kwc">n</span> <span class="hl std">=</span> <span class="hl num">10000000</span><span class="hl std">,</span>
<span class="hl std">id,</span>
<span class="hl std">race,</span>
<span class="hl std">age,</span>
<span class="hl std">smokes,</span>
<span class="hl std">marital,</span>
<span class="hl kwc">Start</span> <span class="hl std">= hour,</span>
<span class="hl kwc">End</span> <span class="hl std">= hour,</span>
<span class="hl std">iq,</span>
<span class="hl std">height,</span>
<span class="hl std">died)</span>
<span class="hl kwd">system.time</span><span class="hl std">(</span><span class="hl kwd">write_csv</span><span class="hl std">(reallybigshew,</span> <span class="hl str">"~/Downloads/Blogging A to Z/bigdata1.csv"</span><span class="hl std">))</span>
</pre>
</div>
<div class="output">
<pre class="knitr r">## user system elapsed
## 134.22 2.52 137.80
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl kwd">system.time</span><span class="hl std">(</span><span class="hl kwd">fwrite</span><span class="hl std">(reallybigshew,</span> <span class="hl str">"~/Downloads/Blogging A to Z/bigdata2.csv"</span><span class="hl std">))</span>
</pre>
</div>
<div class="output">
<pre class="knitr r">## user system elapsed
## 8.65 0.32 2.77
</pre>
</div>
</div>
</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-49862295444440501312020-04-25T09:00:00.000-05:002020-04-27T08:18:41.732-05:00V is for VerbsIn this series, I've covered five terms for data manipulation:<br />
<ul>
<li>arrange</li>
<li>filter</li>
<li>mutate</li>
<li>select</li>
<li>summarise</li>
</ul>
These are the verbs that make up the grammar of data manipulation. They all work with group_by to perform these functions groupwise.<br />
<br />
There are scoped versions of these verbs, which add _all, _if, or _at, that allow you to perform these verbs on multiple variables simultaneously. For instance, I could get means for all of my numeric variables like this. (Quick note: I created an updated reading dataset that has all publication years filled in. You can download it <a href="https://www.dropbox.com/s/w9fusv2i47j170c/SaraReads2019_allchanges.csv?dl=1">here</a>.)<br />
<div class="chunk" id="unnamed-chunk-1">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(tidyverse)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --
</pre>
</div>
<div class="message">
<pre class="knitr r">## <U+2713> ggplot2 3.2.1 <U+2713> purrr 0.3.3
## <U+2713> tibble 2.1.3 <U+2713> dplyr 0.8.3
## <U+2713> tidyr 1.0.0 <U+2713> stringr 1.4.0
## <U+2713> readr 1.3.1 <U+2713> forcats 0.4.0
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl kwb"><-</span> <span class="hl kwd">read_csv</span><span class="hl std">(</span><span class="hl str">"~/Downloads/Blogging A to Z/SaraReads2019_allchanges.csv"</span><span class="hl std">,</span>
<span class="hl kwc">col_names</span> <span class="hl std">=</span> <span class="hl num">TRUE</span><span class="hl std">)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r"><a href="https://www.blogger.com/blogger.g?blogID=4594832939334410220" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a>## Parsed with column specification:
## cols(
## Title = col_character(),
## Pages = col_double(),
## date_started = col_character(),
## date_read = col_character(),
## Book.ID = col_double(),
## Author = col_character(),
## AdditionalAuthors = col_character(),
## AverageRating = col_double(),
## OriginalPublicationYear = col_double(),
## read_time = col_double(),
## MyRating = col_double(),
## Gender = col_double(),
## Fiction = col_double(),
## Childrens = col_double(),
## Fantasy = col_double(),
## SciFi = col_double(),
## Mystery = col_double(),
## SelfHelp = col_double()
## )
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise_if</span><span class="hl std">(is.numeric,</span> <span class="hl kwd">list</span><span class="hl std">(mean))</span>
</pre>
</div>
<div class="output">
<pre class="knitr r">## # A tibble: 1 x 13
## Pages Book.ID AverageRating OriginalPublica… read_time MyRating Gender Fiction
## <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 341. 1.36e7 3.94 1989. 3.92 4.14 0.310 0.931
## # … with 5 more variables: Childrens <dbl>, Fantasy <dbl>, SciFi <dbl>,
## # Mystery <dbl>, SelfHelp <dbl>
</pre>
</div>
</div>
</div>
This function generated the mean for every numeric variable in my dataset. But even though they're all numeric, the mean isn't the best statistic for many of them, for instance book ID or publication year. We could just generate means for specific variables with summarise_at.<br />
<div class="chunk" id="unnamed-chunk-2">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise_at</span><span class="hl std">(</span><span class="hl kwd">vars</span><span class="hl std">(Pages, AverageRating, read_time, MyRating),</span> <span class="hl kwd">list</span><span class="hl std">(mean))</span>
</pre>
</div>
<div class="output">
<pre class="knitr r">## # A tibble: 1 x 4
## Pages AverageRating read_time MyRating
## <dbl> <dbl> <dbl> <dbl>
## 1 341. 3.94 3.92 4.14
</pre>
</div>
</div>
</div>
You can also request more than one piece of information in your list, and request that R create a new label for each variable.<br />
<div class="chunk" id="unnamed-chunk-3">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">numeric_summary</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise_at</span><span class="hl std">(</span><span class="hl kwd">vars</span><span class="hl std">(Pages, AverageRating, read_time, MyRating),</span> <span class="hl kwd">list</span><span class="hl std">(</span><span class="hl str">"mean"</span> <span class="hl std">= mean,</span> <span class="hl str">"median"</span> <span class="hl std">= median))</span>
</pre>
</div>
</div>
</div>
I use the basic verbs anytime I use R. I only learned about scoped verbs recently, and I'm sure I'll add them to my toolkit over time.<br />
<br />
Next week is the last week of Blogging A to Z! See you then!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjztrjkq4_M8UKvmtWlRf7Exc4myBC2mRdAMgGoDYRYy3TRE2JMmxRnIfjVg1v7u36PLkZHcLxaOfq8_08xiC7TKdmr3aNBaFpOZ9ur1ETeksc970f-dw-LIbmmNzUCbCOFuSXsuwb53RwZ/s1600/A+to+Z+of+tidyverse.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1067" data-original-width="1600" height="213" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjztrjkq4_M8UKvmtWlRf7Exc4myBC2mRdAMgGoDYRYy3TRE2JMmxRnIfjVg1v7u36PLkZHcLxaOfq8_08xiC7TKdmr3aNBaFpOZ9ur1ETeksc970f-dw-LIbmmNzUCbCOFuSXsuwb53RwZ/s320/A+to+Z+of+tidyverse.jpg" width="320" /></a></div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-119622429273531142020-04-24T09:00:00.000-05:002020-04-24T09:00:01.795-05:00U is for Useful TrickThis will be a very short post for a line of code I've found unbelievably useful as I analyze data for work. I'm working with datasets containing millions of rows of data. (The most recent one I worked with had about 13 million records.) Because R loads datasets into memory, you can run out of RAM pretty quickly when working with data that large. As I start getting access to more services for databasing and cloud computing, I'm hoping to move some of that data out of my own memory, and onto something with more memory. But for now, I found this quick fix.<br />
<br />
I increased my paging file (virtual memory) on my computer as high as it will let me, but R doesn't automatically increase its memory limits. But a single line of code will do that for you.<br />
<div class="chunk" id="unnamed-chunk-1">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">invisible</span><span class="hl std">(utils</span><span class="hl opt">::</span><span class="hl kwd">memory.limit</span><span class="hl std">(</span><span class="hl num">64000</span><span class="hl std">))</span>
</pre>
</div>
</div>
</div>
Set that value to whatever your virtual memory is set for. (Note that this value is in MB.) Huge thanks for this <a href="https://stackoverflow.com/questions/39876328/forcing-r-and-rstudio-to-use-the-virtual-memory-on-windows/39913887">Stack Overflow post</a> that taught me how to do this.<br />
<br />
Monday, I'll talk about some functions that allow you more quickly read (and write) large files.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjztrjkq4_M8UKvmtWlRf7Exc4myBC2mRdAMgGoDYRYy3TRE2JMmxRnIfjVg1v7u36PLkZHcLxaOfq8_08xiC7TKdmr3aNBaFpOZ9ur1ETeksc970f-dw-LIbmmNzUCbCOFuSXsuwb53RwZ/s1600/A+to+Z+of+tidyverse.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1067" data-original-width="1600" height="213" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjztrjkq4_M8UKvmtWlRf7Exc4myBC2mRdAMgGoDYRYy3TRE2JMmxRnIfjVg1v7u36PLkZHcLxaOfq8_08xiC7TKdmr3aNBaFpOZ9ur1ETeksc970f-dw-LIbmmNzUCbCOFuSXsuwb53RwZ/s320/A+to+Z+of+tidyverse.jpg" width="320" /></a></div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-32694391195671075902020-04-23T09:00:00.000-05:002020-04-23T09:00:07.596-05:00T is for ThemesOne of the easiest ways to make a beautiful ggplot is by using a theme. ggplot2 comes with a variety of pre-existing themes. I'll use the genre statistics summary table I created in yesterday's post, and create the same chart with different themes.<br />
<div class="chunk" id="unnamed-chunk-1">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(tidyverse)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --
</pre>
</div>
<div class="message">
<pre class="knitr r">## <U+2713> ggplot2 3.2.1 <U+2713> purrr 0.3.3
## <U+2713> tibble 2.1.3 <U+2713> dplyr 0.8.3
## <U+2713> tidyr 1.0.0 <U+2713> stringr 1.4.0
## <U+2713> readr 1.3.1 <U+2713> forcats 0.4.0
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl kwb"><-</span> <span class="hl kwd">read_csv</span><span class="hl std">(</span><span class="hl str">"~/Downloads/Blogging A to Z/SaraReads2019_allrated.csv"</span><span class="hl std">,</span>
<span class="hl kwc">col_names</span> <span class="hl std">=</span> <span class="hl num">TRUE</span><span class="hl std">)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## Parsed with column specification:
## cols(
## Title = col_character(),
## Pages = col_double(),
## date_started = col_character(),
## date_read = col_character(),
## Book.ID = col_double(),
## Author = col_character(),
## AdditionalAuthors = col_character(),
## AverageRating = col_double(),
## OriginalPublicationYear = col_double(),
## read_time = col_double(),
## MyRating = col_double(),
## Gender = col_double(),
## Fiction = col_double(),
## Childrens = col_double(),
## Fantasy = col_double(),
## SciFi = col_double(),
## Mystery = col_double(),
## SelfHelp = col_double()
## )
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">genrestats</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(Fiction</span> <span class="hl opt">==</span> <span class="hl num">1</span><span class="hl std">)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">arrange</span><span class="hl std">(OriginalPublicationYear)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">group_by</span><span class="hl std">(Childrens, Fantasy, SciFi, Mystery)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">Books</span> <span class="hl std">=</span> <span class="hl kwd">n</span><span class="hl std">(),</span>
<span class="hl kwc">WomenAuthors</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(Gender),</span>
<span class="hl kwc">AvgLength</span> <span class="hl std">=</span> <span class="hl kwd">mean</span><span class="hl std">(Pages),</span>
<span class="hl kwc">AvgRating</span> <span class="hl std">=</span> <span class="hl kwd">mean</span><span class="hl std">(MyRating))</span>
<span class="hl std">genrestats</span> <span class="hl kwb"><-</span> <span class="hl std">genrestats</span> <span class="hl opt">%>%</span>
<span class="hl kwd">bind_cols</span><span class="hl std">(</span><span class="hl kwc">Genre</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"General Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Mystery"</span><span class="hl std">,</span>
<span class="hl str">"Science Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Fantasy"</span><span class="hl std">,</span>
<span class="hl str">"Fantasy SciFi"</span><span class="hl std">,</span>
<span class="hl str">"Children's Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Children's Fantasy"</span><span class="hl std">))</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ungroup</span><span class="hl std">()</span> <span class="hl opt">%>%</span>
<span class="hl kwd">select</span><span class="hl std">(Genre,</span> <span class="hl kwd">everything</span><span class="hl std">(),</span> <span class="hl opt">-</span><span class="hl std">Childrens,</span> <span class="hl opt">-</span><span class="hl std">Fantasy,</span> <span class="hl opt">-</span><span class="hl std">SciFi,</span> <span class="hl opt">-</span><span class="hl std">Mystery)</span>
<span class="hl std">genre</span> <span class="hl kwb"><-</span> <span class="hl std">genrestats</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(Genre, Books))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_y_continuous</span><span class="hl std">(</span><span class="hl kwc">breaks</span> <span class="hl std">=</span> <span class="hl kwd">seq</span><span class="hl std">(</span><span class="hl num">0</span><span class="hl std">,</span><span class="hl num">20</span><span class="hl std">,</span><span class="hl num">1</span><span class="hl std">))</span>
</pre>
</div>
</div>
</div>
Since I've created a new object for my figure, I can add a theme by typing genre + [theme]. Here's a handful of the ggplot2 themes.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBN_qSLmRy2Q5rhb1tZrOOEjjSruPqTij6B9SbnEC8uhY5V7cHeejEZKVoDXRSxaYSv-ZGV0ZUK_9MQG8l2E5k_CZhjNQStGboaPBiPLk8rOFMZ_PRdpF_CYWIhHmQLUtLLqIju-FcXEpH/s1600/ggplot2+themes.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1095" data-original-width="1600" height="219" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBN_qSLmRy2Q5rhb1tZrOOEjjSruPqTij6B9SbnEC8uhY5V7cHeejEZKVoDXRSxaYSv-ZGV0ZUK_9MQG8l2E5k_CZhjNQStGboaPBiPLk8rOFMZ_PRdpF_CYWIhHmQLUtLLqIju-FcXEpH/s320/ggplot2+themes.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
You can also get more themes with additional packages. My new favorite is ggthemes. I've been loving their Economist themes (particularly economist_white), which I've been using for most of the plots I create at work. Here are some of my favorites.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNwTQ91G4lHbA-uk8VUh_coPO_67wALDzmzaki1koKwbDcR1KQ7X26qbSo4zl0YkZWvQhpxR6YK2p8gIlB1zDzKfEIc-x3uWFcz713YUZ3pqfVWPtvpPHCbKrHOh4EC-6C6bmGPyCKbrCq/s1600/ggthemes+themes.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1437" data-original-width="1600" height="287" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNwTQ91G4lHbA-uk8VUh_coPO_67wALDzmzaki1koKwbDcR1KQ7X26qbSo4zl0YkZWvQhpxR6YK2p8gIlB1zDzKfEIc-x3uWFcz713YUZ3pqfVWPtvpPHCbKrHOh4EC-6C6bmGPyCKbrCq/s320/ggthemes+themes.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
You can also customize different elements of the plot with theme(). For instance, theme(plot.title = element_text(hjust = 0.5)) centers your plot title. theme(legend.position = "none") removes the legend. You could do both of these at once within the same theme() by separating them with commas. This is a great way to tweak tiny elements of your plot, or if you want to create your own custom theme.<br />
<div class="chunk" id="unnamed-chunk-2">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(ggthemes)</span>
</pre>
</div>
<div class="warning">
<pre class="knitr r">## Warning: package 'ggthemes' was built under R version 3.6.3
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">genre</span> <span class="hl opt">+</span>
<span class="hl kwd">theme_economist_white</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">theme</span><span class="hl std">(</span><span class="hl kwc">plot.background</span> <span class="hl std">=</span> <span class="hl kwd">element_rect</span><span class="hl std">(</span><span class="hl kwc">fill</span> <span class="hl std">=</span> <span class="hl str">"lightblue"</span><span class="hl std">))</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhjAzwYe0qSJ0xYv9i0mUPYdsnQazRuWS-35l-c2r7Z-rwJmYcurNZZFHTbrhzVUBr6wj2HE0fI1Rir16zVQKQ9UeqMHd-G90qnxiavYc-kYJgPk2f1sb4pUi4PXyDbGkhfz1iZOkSR6N8/s1600/unnamed-chunk-2-1.tiff" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="360" data-original-width="760" height="151" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhjAzwYe0qSJ0xYv9i0mUPYdsnQazRuWS-35l-c2r7Z-rwJmYcurNZZFHTbrhzVUBr6wj2HE0fI1Rir16zVQKQ9UeqMHd-G90qnxiavYc-kYJgPk2f1sb4pUi4PXyDbGkhfz1iZOkSR6N8/s320/unnamed-chunk-2-1.tiff" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
</div>
</div>
These themes also have color schemes you can add to your plot. We'll talk about that soon!Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4594832939334410220.post-22416678644166832302020-04-22T09:00:00.000-05:002020-04-22T09:00:03.590-05:00S is for summariseToday, we'll finally talk about summarise! It's very similar to mutate, but instead of adding or altering a variable in a dataset, it aggregates your data, creating a new tibble with the columns containing your requested summary data. The number of rows will be equal to the number of groups from group_by (if you don't specify any groups, your tibble will have one row that summarizes your entire dataset).<br />
<br />
These days, when I want descriptive statistics from a dataset, I generally use summarise, because I can specify the exact statistics I want in the exact order I want (for easy pasting of tables into a report or presentation).<br />
<br />
Also, if you're not a fan of the UK spelling, summarize works exactly the same. The same is true of other R/tidyverse functions, like color versus colour.<br />
<br />
Let's load the reads2019 dataset and start summarizing!<br />
<div class="chunk" id="unnamed-chunk-1">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(tidyverse)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Attaching packages ------------------------------------------- tidyverse 1.3.0 --
</pre>
</div>
<div class="message">
<pre class="knitr r">## <U+2713> ggplot2 3.2.1 <U+2713> purrr 0.3.3
## <U+2713> tibble 2.1.3 <U+2713> dplyr 0.8.3
## <U+2713> tidyr 1.0.0 <U+2713> stringr 1.4.0
## <U+2713> readr 1.3.1 <U+2713> forcats 0.4.0
</pre>
</div>
<div class="message">
<pre class="knitr r">## -- Conflicts ---------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl kwb"><-</span> <span class="hl kwd">read_csv</span><span class="hl std">(</span><span class="hl str">"~/Downloads/Blogging A to Z/SaraReads2019_allrated.csv"</span><span class="hl std">,</span>
<span class="hl kwc">col_names</span> <span class="hl std">=</span> <span class="hl num">TRUE</span><span class="hl std">)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">## Parsed with column specification:
## cols(
## Title = col_character(),
## Pages = col_double(),
## date_started = col_character(),
## date_read = col_character(),
## Book.ID = col_double(),
## Author = col_character(),
## AdditionalAuthors = col_character(),
## AverageRating = col_double(),
## OriginalPublicationYear = col_double(),
## read_time = col_double(),
## MyRating = col_double(),
## Gender = col_double(),
## Fiction = col_double(),
## Childrens = col_double(),
## Fantasy = col_double(),
## SciFi = col_double(),
## Mystery = col_double(),
## SelfHelp = col_double()
## )
</pre>
</div>
</div>
</div>
First, we could use summarise to give us some basic descriptives of the whole dataset. If we want to save the results to a tibble, we would give it a new name, or we could just have it display those results and not save them. Here's what happens when I request a summary without saving a new tibble.<br />
<div class="chunk" id="unnamed-chunk-2">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">AllPages</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(Pages),</span>
<span class="hl kwc">AvgLength</span> <span class="hl std">=</span> <span class="hl kwd">mean</span><span class="hl std">(Pages),</span>
<span class="hl kwc">AvgRating</span> <span class="hl std">=</span> <span class="hl kwd">mean</span><span class="hl std">(MyRating),</span>
<span class="hl kwc">AvgReadTime</span> <span class="hl std">=</span> <span class="hl kwd">mean</span><span class="hl std">(read_time),</span>
<span class="hl kwc">ShortRT</span> <span class="hl std">=</span> <span class="hl kwd">min</span><span class="hl std">(read_time),</span>
<span class="hl kwc">LongRT</span> <span class="hl std">=</span> <span class="hl kwd">max</span><span class="hl std">(read_time),</span>
<span class="hl kwc">TotalAuthors</span> <span class="hl std">=</span> <span class="hl kwd">n_distinct</span><span class="hl std">(Author))</span>
</pre>
</div>
<div class="output">
<pre class="knitr r">## # A tibble: 1 x 7
## AllPages AvgLength AvgRating AvgReadTime ShortRT LongRT TotalAuthors
## <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <int>
## 1 29696 341. 4.14 3.92 0 25 42
</pre>
</div>
</div>
</div>
Now, let's create a summary where we do save it as a tibble. And let's have it create some groups for us. In the dataset, I coded author gender, with female authors coded as 1, so I can find out how many women writers are represented in a group by summing that variable. I also want to fill in a few missing publication dates, which seems to happen for Kindle version of books or books by small publishers. This will let me find out my newest and oldest books in each group; I just arrange by publication year, then request last and first, respectively. Two books were published in 2019, so I'll replace the others based on title, then have R give the remaining NAs a year of 2019.<br />
<div class="chunk" id="unnamed-chunk-3">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(</span><span class="hl kwd">is.na</span><span class="hl std">(OriginalPublicationYear))</span> <span class="hl opt">%>%</span>
<span class="hl kwd">select</span><span class="hl std">(Title)</span>
</pre>
</div>
<div class="output">
<pre class="knitr r">## # A tibble: 5 x 1
## Title
## <chr>
## 1 Empath: A Complete Guide for Developing Your Gift and Finding Your Sense of S…
## 2 Perilous Pottery (Cozy Corgi Mysteries, #11)
## 3 Precarious Pasta (Cozy Corgi Mysteries, #14)
## 4 Summerdale
## 5 Swarm Theory
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">OriginalPublicationYear</span> <span class="hl std">=</span> <span class="hl kwd">replace</span><span class="hl std">(OriginalPublicationYear,</span>
<span class="hl std">Title</span> <span class="hl opt">==</span> <span class="hl str">"Empath: A Complete Guide for Developing Your Gift and Finding Your Sense of Self"</span><span class="hl std">,</span> <span class="hl num">2017</span><span class="hl std">),</span>
<span class="hl kwc">OriginalPublicationYear</span> <span class="hl std">=</span> <span class="hl kwd">replace</span><span class="hl std">(OriginalPublicationYear,</span>
<span class="hl std">Title</span> <span class="hl opt">==</span> <span class="hl str">"Summerdale"</span><span class="hl std">,</span> <span class="hl num">2018</span><span class="hl std">),</span>
<span class="hl kwc">OriginalPublicationYear</span> <span class="hl std">=</span> <span class="hl kwd">replace</span><span class="hl std">(OriginalPublicationYear,</span>
<span class="hl std">Title</span> <span class="hl opt">==</span> <span class="hl str">"Swarm Theory"</span><span class="hl std">,</span> <span class="hl num">2016</span><span class="hl std">),</span>
<span class="hl kwc">OriginalPublicationYear</span> <span class="hl std">=</span> <span class="hl kwd">replace_na</span><span class="hl std">(OriginalPublicationYear,</span> <span class="hl num">2019</span><span class="hl std">))</span>
<span class="hl std">genrestats</span> <span class="hl kwb"><-</span> <span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">filter</span><span class="hl std">(Fiction</span> <span class="hl opt">==</span> <span class="hl num">1</span><span class="hl std">)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">arrange</span><span class="hl std">(OriginalPublicationYear)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">group_by</span><span class="hl std">(Childrens, Fantasy, SciFi, Mystery)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">Books</span> <span class="hl std">=</span> <span class="hl kwd">n</span><span class="hl std">(),</span>
<span class="hl kwc">WomenAuthors</span> <span class="hl std">=</span> <span class="hl kwd">sum</span><span class="hl std">(Gender),</span>
<span class="hl kwc">AvgLength</span> <span class="hl std">=</span> <span class="hl kwd">mean</span><span class="hl std">(Pages),</span>
<span class="hl kwc">AvgRating</span> <span class="hl std">=</span> <span class="hl kwd">mean</span><span class="hl std">(MyRating),</span>
<span class="hl kwc">NewestBook</span> <span class="hl std">=</span> <span class="hl kwd">last</span><span class="hl std">(OriginalPublicationYear),</span>
<span class="hl kwc">OldestBook</span> <span class="hl std">=</span> <span class="hl kwd">first</span><span class="hl std">(OriginalPublicationYear))</span>
</pre>
</div>
</div>
</div>
Now let's turn this summary into a nicer, labeled table.<br />
<div class="chunk" id="unnamed-chunk-4">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl std">genrestats</span> <span class="hl kwb"><-</span> <span class="hl std">genrestats</span> <span class="hl opt">%>%</span>
<span class="hl kwd">bind_cols</span><span class="hl std">(</span><span class="hl kwc">Genre</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"General Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Mystery"</span><span class="hl std">,</span>
<span class="hl str">"Science Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Fantasy"</span><span class="hl std">,</span>
<span class="hl str">"Fantasy SciFi"</span><span class="hl std">,</span>
<span class="hl str">"Children's Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Children's Fantasy"</span><span class="hl std">))</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ungroup</span><span class="hl std">()</span> <span class="hl opt">%>%</span>
<span class="hl kwd">select</span><span class="hl std">(Genre,</span> <span class="hl kwd">everything</span><span class="hl std">(),</span> <span class="hl opt">-</span><span class="hl std">Childrens,</span> <span class="hl opt">-</span><span class="hl std">Fantasy,</span> <span class="hl opt">-</span><span class="hl std">SciFi,</span> <span class="hl opt">-</span><span class="hl std">Mystery)</span>
<span class="hl kwd">library</span><span class="hl std">(expss)</span>
</pre>
</div>
<div class="message">
<pre class="knitr r">##
## Attaching package: 'expss'
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following objects are masked from 'package:stringr':
##
## fixed, regex
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following objects are masked from 'package:dplyr':
##
## between, compute, contains, first, last, na_if, recode, vars
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following objects are masked from 'package:purrr':
##
## keep, modify, modify_if, transpose
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following objects are masked from 'package:tidyr':
##
## contains, nest
</pre>
</div>
<div class="message">
<pre class="knitr r">## The following object is masked from 'package:ggplot2':
##
## vars
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl kwd">as.etable</span><span class="hl std">(genrestats,</span> <span class="hl kwc">rownames_as_row_labels</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">)</span>
</pre>
</div>
<table class="gmisc_table" style="border-collapse: collapse; margin-bottom: 1em; margin-top: 1em;">
<thead>
<tr>
<th style="border-bottom: 1px solid grey; border-top: 2px solid grey; font-weight: 900; text-align: center;">Genre</th>
<th style="border-bottom: 1px solid grey; border-top: 2px solid grey; text-align: center;"> Books </th>
<th style="border-bottom: 1px solid grey; border-top: 2px solid grey; text-align: center;"> WomenAuthors </th>
<th style="border-bottom: 1px solid grey; border-top: 2px solid grey; text-align: center;"> AvgLength </th>
<th style="border-bottom: 1px solid grey; border-top: 2px solid grey; text-align: center;"> AvgRating </th>
<th style="border-bottom: 1px solid grey; border-top: 2px solid grey; text-align: center;"> NewestBook </th>
<th style="border-bottom: 1px solid grey; border-top: 2px solid grey; text-align: center;"> OldestBook </th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align: left;"> General Fiction </td>
<td style="text-align: right;">15</td>
<td style="text-align: right;">10</td>
<td style="text-align: right;">320.1</td>
<td style="text-align: right;">4.1</td>
<td style="text-align: right;">2019</td>
<td style="text-align: right;">1941</td>
</tr>
<tr>
<td style="text-align: left;"> Mystery </td>
<td style="text-align: right;">9</td>
<td style="text-align: right;">8</td>
<td style="text-align: right;">316.3</td>
<td style="text-align: right;">3.8</td>
<td style="text-align: right;">2019</td>
<td style="text-align: right;">1950</td>
</tr>
<tr>
<td style="text-align: left;"> Science Fiction </td>
<td style="text-align: right;">19</td>
<td style="text-align: right;">4</td>
<td style="text-align: right;">361.4</td>
<td style="text-align: right;">4.4</td>
<td style="text-align: right;">2019</td>
<td style="text-align: right;">1959</td>
</tr>
<tr>
<td style="text-align: left;"> Fantasy </td>
<td style="text-align: right;">19</td>
<td style="text-align: right;">3</td>
<td style="text-align: right;">426.3</td>
<td style="text-align: right;">4.2</td>
<td style="text-align: right;">2019</td>
<td style="text-align: right;">1981</td>
</tr>
<tr>
<td style="text-align: left;"> Fantasy SciFi </td>
<td style="text-align: right;">2</td>
<td style="text-align: right;">0</td>
<td style="text-align: right;">687.0</td>
<td style="text-align: right;">4.5</td>
<td style="text-align: right;">2009</td>
<td style="text-align: right;">2006</td>
</tr>
<tr>
<td style="text-align: left;"> Children's Fiction </td>
<td style="text-align: right;">1</td>
<td style="text-align: right;">0</td>
<td style="text-align: right;">181.0</td>
<td style="text-align: right;">4.0</td>
<td style="text-align: right;">2016</td>
<td style="text-align: right;">2016</td>
</tr>
<tr>
<td style="border-bottom: 2px solid grey; text-align: left;"> Children's Fantasy </td>
<td style="border-bottom: 2px solid grey; text-align: right;">16</td>
<td style="border-bottom: 2px solid grey; text-align: right;">1</td>
<td style="border-bottom: 2px solid grey; text-align: right;">250.6</td>
<td style="border-bottom: 2px solid grey; text-align: right;">4.2</td>
<td style="border-bottom: 2px solid grey; text-align: right;">2008</td>
<td style="border-bottom: 2px solid grey; text-align: right;">1900</td>
</tr>
</tbody>
</table>
</div>
</div>
I could have used other base R functions in my summary as well - such as sd, median, min, max, and so on. You can also summarize a dataset and create a plot of that summary in the same code.<br />
<div class="chunk" id="unnamed-chunk-5">
<div class="rcode">
<div class="source">
<pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(ggthemes)</span>
</pre>
</div>
<div class="warning">
<pre class="knitr r">## Warning: package 'ggthemes' was built under R version 3.6.3
</pre>
</div>
<div class="source">
<pre class="knitr r"><span class="hl std">reads2019</span> <span class="hl opt">%>%</span>
<span class="hl kwd">mutate</span><span class="hl std">(</span><span class="hl kwc">Gender</span> <span class="hl std">=</span> <span class="hl kwd">factor</span><span class="hl std">(Gender,</span> <span class="hl kwc">levels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl num">0</span><span class="hl std">,</span><span class="hl num">1</span><span class="hl std">),</span>
<span class="hl kwc">labels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"Male"</span><span class="hl std">,</span>
<span class="hl str">"Female"</span><span class="hl std">)),</span>
<span class="hl kwc">Fiction</span> <span class="hl std">=</span> <span class="hl kwd">factor</span><span class="hl std">(Fiction,</span> <span class="hl kwc">levels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl num">0</span><span class="hl std">,</span><span class="hl num">1</span><span class="hl std">),</span>
<span class="hl kwc">labels</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">"Non-Fiction"</span><span class="hl std">,</span>
<span class="hl str">"Fiction"</span><span class="hl std">),</span>
<span class="hl kwc">ordered</span> <span class="hl std">=</span> <span class="hl num">TRUE</span><span class="hl std">))</span> <span class="hl opt">%>%</span>
<span class="hl kwd">group_by</span><span class="hl std">(Gender, Fiction)</span> <span class="hl opt">%>%</span>
<span class="hl kwd">summarise</span><span class="hl std">(</span><span class="hl kwc">Books</span> <span class="hl std">=</span> <span class="hl kwd">n</span><span class="hl std">())</span> <span class="hl opt">%>%</span>
<span class="hl kwd">ggplot</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(Fiction, Books))</span> <span class="hl opt">+</span>
<span class="hl kwd">geom_col</span><span class="hl std">(</span><span class="hl kwd">aes</span><span class="hl std">(</span><span class="hl kwc">fill</span> <span class="hl std">=</span> <span class="hl kwd">reorder</span><span class="hl std">(Gender,</span> <span class="hl kwd">desc</span><span class="hl std">(Gender))))</span> <span class="hl opt">+</span>
<span class="hl kwd">scale_fill_economist</span><span class="hl std">()</span> <span class="hl opt">+</span>
<span class="hl kwd">xlab</span><span class="hl std">(</span><span class="hl str">"Genre"</span><span class="hl std">)</span> <span class="hl opt">+</span>
<span class="hl kwd">labs</span><span class="hl std">(</span><span class="hl kwc">fill</span> <span class="hl std">=</span> <span class="hl str">"Author Gender"</span><span class="hl std">)</span>
</pre>
</div>
</div>
<div class="rimage default">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBJ_l14trXbz_MIa7fAZG18C5_4OfcN-XjdDoOlQvhvQdkteq9TWNpWQpXJQhoDrURGYQTkMdi-DZ4F4maqr4dEKJ2WGqtR-n80tISQDAq11XZ5hwOubKGCfzy1aIXH4yHYYEbfk81oF-f/s1600/unnamed-chunk-5-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="504" data-original-width="504" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBJ_l14trXbz_MIa7fAZG18C5_4OfcN-XjdDoOlQvhvQdkteq9TWNpWQpXJQhoDrURGYQTkMdi-DZ4F4maqr4dEKJ2WGqtR-n80tISQDAq11XZ5hwOubKGCfzy1aIXH4yHYYEbfk81oF-f/s320/unnamed-chunk-5-1.png" width="320" /></a></div>
</div>
</div>
Unknownnoreply@blogger.com1