Statistical Sins: Unclear Terms

Though I'm a psychometrician/statistician in my job, I've been dipping my toe into the pool of data science, so I've blogged about data science and the related topics. In fact, one such post resulted in a lively discussion on Facebook about what data science is exactly.

I mostly remained a voyeur to this discussion. I don't see anything wrong with stepping back and listening to what others think, rather than always jumping in with my opinion.

But, I've still committed a sin in that I've used terms like data science, machine learning, and artificial intelligence in rather unclear, even sloppy ways. Fortunately, David Robinson of Variance Explained is here to save the day! So in lieu of my own statistical sins posts for the day, I'm calling myself out and recommending you read David's awesome post on the difference among these three terms.

As for me, I'm working on pulling together a dataset on reading habits of my Goodreads friends from 2017 - a dataset I started on a sleepless night earlier this week. I'm just about ready to start analyzing it. Stay tuned for some fun analyses! (But one thing I've already learned is that the most read book among my friends in 2017 was The Handmaid's Tale.)

This Response, Dough

I'm sure Mario Batali had no idea that his hastily thrown-together pizza dough cinnamon roll recipe included at the tail-end of his sexual misconduct apology would be such an apt metaphor for double standards and harassment. Check out this post on The Everywhereist:

Last night, I made cinnamon rolls. I’m not a huge fan of cinnamon rolls, per se, but this recipe was included in Mario Batali’s sexual misconduct apology letter., and so I feel compelled to make them. Batali is not the first powerful man to request forgiveness for “inappropriate actions” towards his coworkers and employees. He is not the most high profile, and he is ostensibly not even the worst offender. But he is the only one who included a recipe.

And of course, the glaring question is why? Was his PR team drunk? Is life suddenly a really long, depressing SNL sketch? Do these cinnamon rolls somehow destroy the patriarchy? Does the icing advocate for equal pay?

I figure the only way to answer these questions is to make the damn rolls.

It's one of the most brilliant posts I've read in a long time:

I find myself fluctuating between apathy and anger as I try to follow Batali’s recipe, which is sparse on details. The base of the rolls is pizza dough – Batali notes that you can either buy it, or use his recipe to make your own.

I make my own, because I’m a woman, and for us there are no fucking shortcuts. We spend 25 years working our asses off to be the most qualified Presidential candidate in U.S. history and we get beaten out by a sexual deviant who likely needs to call the front desk for help when he’s trying to order pornos in his hotel room.

Donald Trump is President, so I’m making the goddamn dough by scratch.

I roll out the dough – Batali specifies a thickness, but no dimensions, which is strange if you’re making a rolled dessert. There are pieces missing here, and I’m trying to fill in the gaps. The result will be sub-par because he hasn’t provided all the information, and I will blame myself.

I think about how the last conversation about compensation I had resulted in someone who made more yearly than I ever will telling me I was holding them “emotionally hostage” and then demanding to know, over and over again why I needed the money.

“Just tell me,” they demanded. “Tell me why you need it.” Over and over until it broke me.

If they are edible, I will eat every single one of these fucking rolls myself.

The pizza dough does not mix well with the sweetness. The icing is sickly sweet, the rolls themselves oddly savory. I was right about the texture – the dough is too tough. I hate them, but I keep eating them. Like I’m somehow destroying Batali’s shitty sexist horcrux in every bite.

Reading and Viewing List

Two meetings this morning, so I haven't had a chance to read or watch some open tabs. Today's reading:

• Fallows writes about how actual smart people talk about themselves (and in my favorite response thus far to "stable genius," a friend shared a picture of a stable genius on Facebook - her daughter's horse)
• The Tribune gives us Impossible Tacos - I'll bite
• Also from the Tribune, Medieval Times finally has a queen

And on my watch list:

What the Wrath of Khan's trailer would look like today

Also on the subject of trailers, a recut that makes The Room look like a good movie:

Cracked discusses how 9/11 changed 90s sitcoms:

And Lessons from the Screenplay examines The Fault in our Star Wars:

Statistics Sunday: On Birthdays, and One-Tailed and Two-Tailed Tests

It's my birthday today! I spent last night celebrating with friends. Today I'll be relaxing, reading, and writing, plus dinner at one of my favorite restaurants. Obviously, birthdays have some important implications:

For today's Statistics Sunday post, I wanted to revisit a topic I touched on briefly in my post on null and alternative hypotheses, using our silly hypothesis above as an example. We could update this hypothesis to be something about having more birthday parties, but obviously, there would still be a third variable issue here.

Whenever you set out to conduct statistical analyses, you have hypotheses you want to test. While I would argue that you should focus entirely on your research hypotheses, rather than actively writing out null and alternative hypotheses, it is a good thing to remember what it is you're testing and how you will know you have the expected effect. And that's one way your statistical hypotheses (null and alternative) can help guide you.

Those statistical hypotheses, among other things, allow you to derive your critical value - the cut-off between significant and non-significant. This is based on your selected alpha. When you're testing for a difference, one that could go in either direction, you divide your alpha into 2. In this case, your cut-off in your normal distribution would be in both tails.

But if I wanted to test the hypothesis above, that people who have had more birthdays live longer, it doesn't make sense to test for a difference. It makes sense to test in a certain direction: people who have had more birthday parties live longer. In this case, you can maximize the chance that you'll find a significant effect by putting all of your alpha in one tail.

If you did a really simple study, where you recruited people who had never had a birthday party and people who celebrated their birthdays, then tracked how long they lived, you would set your cut-off value so that all of your alpha is in the upper tail of your distribution - and you would only reject if your birthday partiers were significantly older at their time of death than your birthday non-partiers. Your cut-off value will be lower as a result, making it easier to reject the null hypothesis, but you also only get to reject the null hypothesis if the difference is in the expected direction (that is, the partiers were significantly older rather than significantly younger).

This concept is taught in statistics courses, and students are often tested on whether they understand one-tailed versus two-tailed tests. The thing is, I rarely see this done in journal articles. Not every statistical analysis allows for one- versus two-tailed tests, and it wouldn't make sense to do this for more complex analyses. But it's surprising that when people are doing simple t-tests, which I do see in journal articles, there doesn't seem to be any consideration of directionality of hypotheses.

What do you think? Have you encountered one- and two-tailed tests outside of statistics courses?

In Its Prime

What do you get when you take 2 to the power of 77,232,917 and subtract one?

The largest prime number ever discovered.

The number belongs to a rare group of so-called Mersenne prime numbers, named after the 17th century French monk Marin Mersenne. Like any prime number, a Mersenne prime is divisible only by itself and one, but is derived by multiplying twos together over and over before taking away one. The previous record-holding number was the 49th Mersenne prime ever found, making the new one the 50th.

And just like the Psychological Science Accelerator I blogged about earlier today, it's the product of teamwork:

The new prime number was originally found on Boxing Day by the Great Internet Mersenne Prime Search (Gimps) collaboration which harnesses the number-crunching power of volunteers’ computers all over the world. In the days after, four more computers sporting different hardware and software were set the task of verifying the discovery. Those computers confirmed the result, taking between 34 and 82 hours each.

To find M77232917 in the first place took six full days of nonstop computing on a PC owned by Jonathan Pace, a 51-year old electrical engineer from Germantown, Tennessee. It is the first prime that Pace’s computer has churned out in 14 years on the Gimps project. He is now eligible for a $3,000 award. 

Teamwork and the Reproducibility Problem

It has been known for some time that psychology has a reproducibility problem, though we may not always agree on how to handle or discuss these issues. I remember chatting with another researcher at a conference shortly after I finished my masters thesis on stereotype threat and its impact on math performance in women. I had failed to replicate stereotype threat effects in my study. She, on the other hand, said her effects were incredibly strong; she described a participant experiencing a panic attack when she was told she had to do math problems, and had even noticed her female participants' math performance was negatively affected when her research assistant had been knitting during the session. (I also remember a reviewer telling me I must have performed the study poorly, not because the reviewer found any flaws in my methods, but because I had failed to reproduce the stereotype threat effects in my research.)

Efforts to handle this crisis thus far have included making psychological research more transparent and large-scale meta-analyses. And a new effort is already underway to harness the power of multiple research labs across the world: the Psychological Science Accelerator. Christie Aschwanden of FiveThirtyEight has more:

[Psychologist Christopher] Chartier, a researcher at Ashland University, doesn’t think massively scaled group projects should only be the domain of physicists. So he’s starting the “Psychological Science Accelerator,” which has a simple idea behind it: Psychological studies will take place simultaneously at multiple labs around the globe. Through these collaborations, the research will produce much bigger data sets with a far more diverse pool of study subjects than if it were done in just one place.

The accelerator approach eliminates two problems that can contribute to psychology’s much-discussed reproducibility problem, the finding that some studies aren’t replicated in subsequent studies. It removes both small sample sizes and the so-called weird samples problem, which is what happens when studies rely on a very particular population — like relatively wealthy college students from Western countries — that may not represent the world at large.

So far, the project has enlisted 183 labs on six continents. The idea is to create a standing network of researchers who are available to consider and potentially take part in study proposals, Chartier said. Not every lab has to participate in any given study, but having so many teams in the network ensures that approved studies will have multiple labs conducting their research.

According to the blog, the Psychological Science Accelerator is taking on its second study, this one on gendered social category representation. And if you're attending the Association for Psychological Science meeting in May, you can check out a symposium on "Large Scale Research Collaborations: Applications in Crowd-Sourcing and Undergraduate Research Experience, Replications, and Cross-Cultural Research." (Day and time TBD - APS is still finalizing the program, and is still accepting poster submissions through the end of this month.)

David Bowie's Top 100 Books

I just learned today that David Bowie's son, Duncan Jones is starting a book club so people can read through David Bowie's Top 100 books. Nerdist has more:

Now we’ve learned, via Consequence of Sound, that in admiration for the rock icon’s favorite books, his son, director Duncan Jones, has launched the “David Bowie Book Club.”

My dad was a beast of a reader. One of his true loves was Peter Ackroyd’s sojourns into the history of Britain & its cities. I’ve been feeling a building sense of duty to go on the same literary marathon in tribute to dad. Time allowing...#Read-ItBig’nsTheBrain

— Duncan Jones (@ManMadeMoon) December 27, 2017

The book club will stick to Bowie’s 100 favorite books from the previously mentioned list, and the first book to be discussed will be the 1985 novel Hawksmoor by Peter Aykroyd, an author who Jones says was one of his father’s “true loves.” If you want to follow along with the book club, the deadline to finish the 288 page novel is February 1, so you better get to your local library or hit up your local bookseller or Amazon soon.

In the meantime, I took a quick inventory of all the books I own that have languished on my to-read pile(s) and also just received a literal box o' books from my parents for my birthday (in addition to all the books I received for Christmas). I'm probably good for a while. But I may try to get to Hawksmoor before February 1. Thanks to my no-book-buying resolution, I'll have to check out my local library.