Not Alone

I'm not the only one ready to get this election over with. The results from final election poll by the New York Times and CBS declares, "An overwhelming majority of voters are disgusted by the state of American politics, and many harbor doubts that either major-party nominee can unite the country after a historically ugly presidential campaign." 82% stated the current election has left them disgusted rather than excited. The race is tight, according to this poll, with 45% supporting Clinton and 42% supporting Trump.

Interestingly, they did some analyses of responses based on who the respondent was supporting, and the differences are striking. While 71% of Clinton supporters had "a lot" of confidence that the votes would be counted properly, only 19% of Trump supporters said the same. Relatedly, only 7% of Clinton supporters said vote fraud happened "a lot" versus 42% of Trump supporters. (Maybe because some of the recent cases of voter fraud reported in the media have been by Trump supporters?) 86% of Clinton supporters, and 63% of Trump supporters, said they would accept the results of the election. Trump supporters are also more likely to support poor sportsmanship - only 36% said it was important for the losing candidate to publicly acknowledge the winner, compared to 77% of Clinton supporters.

Despite the fact that the Trump supporters seem to be buying Trump's rhetoric that the election is rigged, and see no reason Trump needs to shake hands and say "Good game," 41% said Trump's candidacy was bad for the Republican party and 85% perceive the Republican party as divided. Only 17% of Democrats felt Clinton's candidacy was bad for the party, and 73% view the party as united. Follow-up interviews were conducted to better understand these results:

Republicans cannot even agree on who is to blame for the division, though they largely believe Mr. Trump has been the impetus for the breach, according to some follow-up interviews.

“I think Donald Trump has definitely divided the party,” said Sheila Wagner, 79, a Republican from Redmond, Wash. She said she had already marked her ballot for Mrs. Clinton, adding: “When he first declared he was going to run, I thought it was a joke. I just couldn’t believe anyone would favor him.”

Yet other Republicans point the finger at Republicans who have refused to support Mr. Trump.

“The old school, quote unquote, the Bushes, the people who have been around a long time, aren’t supporting Trump, and that’s creating division,” said Nora Reinhardt, 66, a farmer from Holt, Mo. “Some Republicans, because of comments Trump has made, which I grant are uncouth and certainly not politically correct, have found they can’t support him, although I think some of those people are coming around at this point.”

She said she was supporting Mr. Trump because she agreed with his policy positions.

Ms. Reinhardt clearly has a word salad translator, because I can't figure out half of Trump's positions from his run-on, stream-of-consciousness answers. So anyone who finds Trump as incoherent as me should give Ms. Reinhardt a call.

I wonder who would be less divisive for the Republican party, Trump, or the shrieking white-hot ball of pure rage (or is this a trick question?):

That Didn't Last Long

I was trying to ride out the Cubs win over stressing about the election a little longer, but around noon today, I failed and began checking statistics on the election and forecasts about who would win. I've been following FiveThirtyEight's election forecast pretty closely, and am really liking some of the figures they use to display the results.

Right now, they give Clinton a 65.9% chance of winning - note, depending on when you check that link above, the figure may have changed. They continually update their models as data becomes available. Their map of the US looks a little too red for my taste, though. Cue momentary tachycardia. Just keep scrolling.

After displaying some data on trends in opinion polling, as well as a chart of state-by-state results, we get to their electoral vote figures. I really liked this particular figure, not just because it reminds me of a board game, but because of how well it communicates a large of amount of data:

At the bottom of the page, they provide some probabilities of different outcomes, including Electoral College deadlock (1.3%), Clinton wins popular vote (77.5%), and map exactly the same as 2012 (0.2%). You can also read more about the methodology used in the election forecast here. Not only does their model aggregate polling data from across the country, and use prediction variables found to be useful in previous elections, they even account for uncertainty in polling:

Our probabilities are based on the historical accuracy of election polls since 1972. When we say a candidate has a 30 percent chance of winning despite being down in the polls, we’re not just covering our butts. Those estimates reflect the historical uncertainty in polling.

So they're factoring in the rate at which polls have previously been inaccurate in other elections. The model also is more reactive to fluctuations closer to the election. So while changes in the polls over the summer change the model very little, changes in the polls now result in more radical changes in the predictions.

Finally, they use a process known as boot-strapping, where the analysis program randomly draws a large number of samples from the data (something on the order of 20,000 in this case) to generate a range of values. This helps minimize the influence of sampling error - drawing a sample that doesn't represent the population well. There's also some interesting info on which probability distributions they use and their rationale. If you're a stats-lover, you'll probably enjoy those details.

On the Cubs, Probability, and a Chance of Rain

I blog a lot about probability here. Statistics is one the loves of my life, and I'm so fortunate to have found something I love this much. Probability comes into play in pretty much every decision we make, and every outcome we consider, whether we realize it or not. Recently, statisticians have spent a great deal of time discussing probabilities of major outcomes, such as who will win the Presidential Election.

And then, of course, there's the Cubs, who have been shown in previous statistical analyses to be the unluckiest team in baseball. There was a lot of evidence that this year would be the change Cubs fans could believe in. And last night, they made history by winning the World Series.

You may remember, though, that a few days ago I blogged that the Cubs had a 24% chance of winning it all. So I'm not too surprised that people have messaged me to say, "I guess the statistics were wrong. I mean, they won!" And yes, yes they did, and words can't express how awesome I think that is. But that doesn't mean the prediction was wrong.

Let's use another example, one we have all encountered. The weatherman tells you there is a 10% chance it will rain that day. It rains. You say, "Guess the weatherman was wrong." But technically, he was not. He said 10% chance, which means, it's unlikely but it could happen. In fact, the way they arrive at those figures is through statistical models, that demonstrate, given these conditions in other places and on other days, it rained about 10% of the time, and did not rain about 90% of the time. So somewhere, sometime, those specific conditions resulted in rain. If there's a non-zero chance of something happening, it could happen.

Whenever we conduct statistical analyses, we are interested in estimating the true population value. I say estimate because we can never really know the true population value, unless we measure every instance in the population. (And even then, there is the strong possibility of measurement error. So really, we need a psychic to divine the true population value.) We accept the fact that there will be some error - that is, some difference between the true value and the value we obtain in our analyses. As long as we conduct our analyses correctly and rigorously, and as long as we base those analyses on data that has been collected correctly and rigorously, we maximize the chance that our estimate will be close to the true population value. But there is a non-zero chance that it is wrong.

Predicting an outcome like rain or who will win the election or whether the Cubs will win the World Series requires complex models, because there are so many variables at play. We use a combination of theory, good guesses, and empirical evidence to determine what variables are good (that should be included) and what variables are irrelevant (that should be dropped). Empirical evidence comes in the form of explained variance. How much of the movement in an outcome (say, between win and loss) can be explained by this variable? Some variables explain more variance than others. Batting averages of Cubs team members explains more variance than say, what flavor Gatorade they have that day. So we choose those variables that explain large proportions of variance, to try to get the total explained variance as close to 100% as possible. The more variance we can explain, the more accurate our model will be at predicting outcomes.

But we can never know what variables will be the absolute best predictor, nor can we say anything about variables we don't or can't measure. So it's probable that the 24% estimate was wrong, and possible that it was very wrong. But simple knowledge of the outcome (they won) is not enough evidence to say the probability estimate was wrong. 24% isn't great odds, but it isn't zero. Unlikely things happen all the time. You might flip a coin and get 10 heads in a row. That's unlikely, but it could (and does) happen. Last night's game may have been one of the best demonstrations of probability in action I can think of.

At least, until next Tuesday. I'm still nervous about the election. But for now, I'm just going to enjoy the Cubs win a little longer.

Evidence in Favor of Blogging A to Z

As I've mentioned before, I participated in the April Blogging A to Z challenge. Each day featured a post about social psychology, with topics from A to Z. Recently, I noticed that my blogging frequency has gone way up, thanks in large part to this challenge. I decided to play around with the data and look at some of the trends. Since I've been spending more time in R lately, I did some time series analyses there.

First, I looked at data by month. You can see that 2016 has been my best year of blogging by far:

For completeness, I included data points for November and December of this year, even though those values are low. They're only going to increase, of course.

Then, I looked at some time series data by day. The first graph is the number of posts per day, which ranged from 0 to 3. You can see the density of the line increase just after day 1500. Blogging A to Z started on day 1651:

And finally, I plotted cumulative frequency over time, which is the black line that shows total posts by day. The blue line is a simple linear trend - what the line would look like if I were posting at a constant rate over time. Once again, you see a large spike just after day 1500:

I thought about doing a time series regression, but I think these data pretty clearly demonstrate the benefit of Blogging A to Z.

And Scene

It's November, which among my writer friends means it's National Novel Writing Month (or NaNoWriMo)! I'll be participating this year - if any of my readers will also be participating, feel free to "buddy" me, username is smlocatelli. I've been writing all my life, but novel writing is something I only got into very recently (as in, the last 5 years or so). Instead, I spent the first part of my life as a playwright. I've written several one acts, a couple of two act plays, and have copious notes for a three act play. I also started writing a musical once, but didn't get as far. So my struggle when I write novels and stories is that I have a lot of dialogue, with minimal commentary about the scene and actions, just as you would see in a play (dialogue and the occasional stage direction or author's note).

I've been thinking recently about stories and what we find believable. And of course, as many have said, if 2016 were a story instead of reality, no one would believe it. So in honor of a month of writing, and as an homage to my previous life as a playwright, here's a scene of a young hopeful author proposing the novel of 2016 to his agent.

Agent: All right, kid, I hear you have a new novel idea.

Author: Oh, yeah, you're going to love it. It's a story of death, political scandals, but at the end, hope.

Agent: I like the sound of that. I mean, you haven't written anything in a while. Your public, they want something new.

Author: Don't worry, this is going to be my best novel yet.

Agent: Okay, tell me more.

Author: Well, first, David Bowie, Prince, Alan Rickman, they all die. And -

Agent: Wait, what?

Author: Bowie, Prince, Rickman, gone. So -

Agent: But... Are they all in a car together? Did they crash in an airplane? I mean, how are they all dead at the same time?

Author: Ha, oh, they don't all die at the same time.

Agent: Good.

Author: Rickman and Bowie in January, and Prince in April.

Agent: So Rickman and Bowie are in a car together?

Author: No, there's no car. Will you stop it with the car? They just die around the same time. By coincidence.

Agent: ... okay... You know, we'll work on that part. Tell me what happens next.

Author: Right, so Donald Trump and Hillary Clinton are running against each other for president -

Agent: Hold on, hold on. Trump? The guy from the Apprentice?

Author: Yeah.

Agent: Is he, like, a third party candidate?

Author: No, he's the Republican nominee.

Agent: Hmm... I mean, I don't want to belittle the excellent work you've put into your story. I just, I mean, the idea is so far-fetched. Why would he run for president?

Author: I don't know. I guess he just likes seeing himself on TV.

Agent: He needs some motivation. That part needs a little more work. Maybe if you -

Author: Oh no, it's great. Trump kind of says whatever the hell he wants and people still like him. He'll even say something and then two days later, deny he said it. Like when he talks about grabbing women. Oh, and he wants to build a huge wall on the Mexican border, and make Mexico pay for it.

Agent: Why would Mexico pay for it?... Never mind. Listen -

Author: No, it'll be great. He's been a really interesting character to write. I just can't figure out how to fit in the scenes with his BFF Putin.

Agent: [Sigh] Putin? So he lies, he's friends with Putin. Maybe we can make that angle work. You know, make that his motivation. Anything else?

Author: Oh, lots of scandals, really. But everyone is all up in arms about Hillary Clinton's email server.

Agent: Wait, you have a guy with ties to Russia, who grabs women, who thinks he'll make Mexico pay for an enormous wall, and people are freaking out about an email server? Kid, I think you should maybe revisit some of this.

Author: Oh, but I haven't even gotten to the best part. At the end, hope.

Agent: ... Okay...

Author: Cubs v. Indians World Series.

Agent: ... Get out of my office.

Author: But -

Agent: Out! Now! And don't come back until you give me something real!

When the Evidence is Wrong

I've blogged before about the issues with memory, particularly when eyewitness testimony is used in trials to convict. That doesn't get at the whole story though; the problem isn't really memory, it's how eyewitness evidence is obtained that introduces issues. Psychologists who study memory in the context of eyewitness testimony have often made recommendations for how police and other legal personnel should question eyewitnesses. In fact, one of these experts, Iowa State University professor Gary Wells, has spent a lot of his time studying police lineup procedures, where they can go wrong, and how to do them properly.

Fortunately, people in general seem to have gotten the message about the issues with memory. Unfortunately, they tend to put too much faith in other types of evidence, which can be just as fallible.

DNA evidence is viewed by many as the gold standard in forensic evidence. But just as there is a right and wrong way to obtain eyewitness evidence, there are strict processes that must be followed in examining DNA evidence, or you run the risk of invalidation.

An article in the APS Observer discusses a cold case that was "solved" with DNA evidence. However, the story needed to explain that evidence is convoluted, and the possibility that the DNA evidence was contaminated seems much more likely. APS Fellow John Wixted explains:

As part of the “Psychology & Law” class that I teach, I set out to illustrate the point that contaminated forensic evidence of any kind (not just contaminated eyewitness evidence) can lead to a wrongful conviction. Because DNA evidence generally is regarded as the gold standard of forensic evidence, I decided to use that as my example. To find an illustrative case, I conducted a Google search using terms like “DNA contamination” and “wrongful conviction.” That’s how I stumbled upon the story of Gary Leiterman.

The story begins with a tragic event: 23-year-old University of Michigan law student Jane Mixer was shot in the head with a .22-caliber gun in the early morning hours of March 21, 1969, and her body was dumped in a graveyard. She had arranged to meet a stranger named David Johnson at the student union on the evening of March 20, 1969, and he was supposed to drive her home to Muskegon, Michigan (about a 3-hour drive). She never made it. She was initially thought to be a victim of the infamous “Co-Ed Killer” (later found to be John Norman Collins), who was murdering women in the area at the time, but no direct link to him was ever established.

Mixer’s murder remained unsolved until a cold-case analysis of DNA from the crime scene was conducted in a Michigan State Police forensic lab in 2002. That analysis uncovered two previously undetected DNA profiles that matched the profiles of two men who were included in a federal DNA database because they had recently committed crimes. One of those men was John Ruelas, who murdered his mother in early 2002. His DNA was found on a blood spot taken from Mixer’s hand in 1969. The other was Leiterman, who recently had forged a prescription for pain medication. His DNA was found on a piece of pantyhose that Mixer had been wearing that day. Because Ruelas was only 4 years old in 1969, he was ruled out as a suspect, so only Leiterman remained a suspect. In 2005, he was sentenced to life in prison without the possibility of parole.

The problem is, that because the prosecution viewed the DNA evidence as accurate, they had to explain how 4-year-old Ruelas's DNA made it onto Mixer's hand. Their explanation was that he was also present at the crime, and bleeding. Mind you, the crime took place around 3am. It seems highly unlikely that a 4-year-old would be there, nor is there any reason to believe Leiterman and Ruelas knew each other. And the plot thickens:

The timing of the DNA analyses is striking. On October 24, 2001, the Mixer evidence from 1969 was pulled out of storage and taken to the Michigan State Police crime lab for processing. It was analyzed in March and April of 2002. Independently, evidence from the 2002 Ruelas murder case arrived at the lab on January 29, 2002, and was analyzed on February 20 and 21, 2002. At this point, it was unclear who had murdered Ruelas’s mother, so the evidence was sent to the lab to help with the investigation. On February 22, 2002, a mouth swab from Leiterman arrived in the lab for initial processing so that his DNA profile could be entered into the federal database.

So these three people, who were previously considered unconnected, all had DNA evidence being analyzed at this lab at the same time. The DNA evidence obtained from Mixer's pantyhose, which the lab said belonged to Leiterman, was not blood or semen, but saliva, the exact kind of sample sent to the lab for processing. Further, Wixted continues discussing the Co-Ed Killer John Collins, who was active in that area, at that time, and many of his crimes matched the details of the Mixer case. It seems much more likely that Collins, not Leiterman, committed the murder. The problem is that, if you can't trust the DNA evidence, what can you trust? What additional evidence could possibly be found to overturn Leiterman's conviction?

On Liberalism, Conservatism, and Media Bias

I still have that email subscription to Glamour's newsletter - see previous posts on why. And while many of the articles are fluff (but hey, who doesn't like a good fluff sometimes?), one article this morning caught my eye as an interesting social analysis. Glamour writer, Aminata Dia spent three weeks reading only conservative news sources and reported back her results.

To give some context, Aminata describes herself as open-minded, non-judgmental, and curious. She recognizes that she is liberal and tends to get her news from "liberal media sources." She also recognizes that she herself works for what would be described as the "liberal media." But she saw this experience as opportunity to look at why conservatives hear the same piece of news and come away with such different opinions.

She offered many great conclusions, and one of the big take aways is that there is no such thing as unbiased media, on either side of the political spectrum. Photos are selected to make a person look bad (or good, depending on who it is), videos are edited to remove some of the context, and there's lots of finger-pointing, with each side of the spectrum blaming the other for the bad, and patting themselves on the back for the good:

So, what happened after week three? Weirdly enough, I realized that the conservative lens is sometimes oddly similar to the liberal one; there's a lack of objective reporting and a lot of finger-pointing when it comes to placing blame. And although aspects of the conservative mentality will continue to conflict with my views, I plan to keep reading and watching my right-winged counterparts to broaden my outlook and keep my sense of reality in check. In the end, I can’t say I walked away a Trump supporter, nor can I claim to have had any ah-ha! moment that made me realize The New York Times is really just a cesspool of liberal propaganda. But I will tell you one thing I learned: Now, more than ever, you are what you read.