Stats Note: Making Sense of Open-Ended Responses with Text Analysis

Good survey design is both art and science. You have to think about how people will read and process your questions, and what sorts of responses might result from different question forms and wording. One of the big rules I follow in survey design is that you don't assess any of your most important topics with an open-ended item. Most people skip them, because they're more work than selecting options from a list, and people who do complete may give you terse, unhelpful, or gibberish answers.

When I was working on a large survey among people with spinal cord injuries/disorders, the survey designers decided to assess the exact details of the respondent's spinal injury or disorder with an open-ended question so that people could describe it "in their own words." As you might have guessed, the data were mostly meaningless and as such unusable. But many hypotheses and research questions dealt with looking at differences between people who sustained an injury and those with a disorder affecting their spinal cord. We couldn't even begin to test or examine any of those in the course of our study. It was unbelievably frustrating, because we could have gotten the information we needed with a single question and some categorical responses. We could have then asked people to supplement their answer with the open-ended question. Most would skip it, but we'd have the data we needed to answer our questions.

In my job, I inherited the results of a large survey involving a variety of dental professional populations. Once again, certain items that could have been addressed with a few close-ended questions were instead open-ended questions and not many of the responses are useful. The item that inspired this blog post assessed the types of products dental assistants are involved in purchasing, which can include anything from office supplies to personal protective equipment to large equipment (X-ray machine, etc.). Everyone had a different way of articulating what they were involved with purchasing, some simply saying "all dental supplies" or "everything," while others gave more specific details. In total, about 400 people responded to this item, which is a lot of data to dig through. But thanks to my new experience with text mining in R, I was able to try to make some sense of responses. Mind you, a lot of the responses can't be understood, but it's at least something.

I decided I could probably begin to categorize responses by slicing the responses up into the individual words. I can generate counts overall as well as by respondent, and use this to begin examining and categorizing the data. Finally, if I need some additional context, I can ask R to give me all responses that contain a certain word.

Because I don't own these data, I can't share them on my blog. But I can demonstrate with different text data to show what I did. To do this, I'll use one of my all-time favorite books, The Wonderful Wizard of Oz by L. Frank Baum, which is available through Project Gutenberg. The gutenbergr package will let me download the full-text.

library(gutenbergr)
gutenberg_works(title == "The Wonderful Wizard of Oz")

## # A tibble: 1 x 8
##   gutenberg_id title   author   gutenberg_autho~ language gutenberg_books~
##          <int> <chr>   <chr>               <int> <chr>    <chr>           
## 1           55 The Wo~ Baum, L~               42 en       Children's Lite~
## # ... with 2 more variables: rights <chr>, has_text <lgl>

Now we have the gutenberg_id, which will be used to download the fulltext into a data frame.

WOz <- gutenberg_download(gutenberg_id = 55)

## Determining mirror for Project Gutenberg from http://www.gutenberg.org/robot/harvest

## Using mirror http://aleph.gutenberg.org

WOz$line <- row.names(WOz)

To start exploring the data, I'll need to use the tidytext package to unnest tokens (words) and remove stop words that don't tell us much.

library(tidyverse)

library(tidytext)
tidy_oz <- WOz %>%
  unnest_tokens(word, text) %>%
  anti_join(stop_words)

## Joining, by = "word"

Now I can begin to explore my "responses" by generating overall counts and even counts by respondent. (For the present example, I'll use my line number variable instead of the respondent id number I used in the actual dataset.)

word_counts <- tidy_oz %>%
  count(word, sort = TRUE)
resp_counts <- tidy_oz %>%
  count(line, word, sort = TRUE) %>%
  ungroup()

When I look at my overall counts, I see that the most frequently used words are the main characters of the book. So from this, I could generate a category "characters." Words like "Kansas", "Emerald" and "City" are also common, and I could create a category called "places." Finally, "heart" and "brains" are common - a category could be created to encompass what the characters are seeking. Obviously, this might not be true for every instance. It could be that someone "didn't have the heart" to tell someone something. I can try to separate out those instances by looking at the original text.

heart <- WOz[grep("heart",WOz$text),]
head(heart)

## # A tibble: 6 x 3
##   gutenberg_id text                                                  line 
##          <int> <chr>                                                 <chr>
## 1           55 happiness to childish hearts than all other human cr~ 48   
## 2           55 the heartaches and nightmares are left out.           62   
## 3           55 and press her hand upon her heart whenever Dorothy's~ 110  
## 4           55 strange people.  Her tears seemed to grieve the kind~ 375  
## 5           55 Dorothy ate a hearty supper and was waited upon by t~ 517  
## 6           55 She ate a hearty breakfast, and watched a wee Munchk~ 545

Unfortunately, this got me any line containing a word starting with "heart" like "hearty" and "heartache." Let's rewrite that command to request an exact match.

heart <- WOz[grep("\\bheart\\b",WOz$text),]
head(heart)

## # A tibble: 6 x 3
##   gutenberg_id text                                                  line 
##          <int> <chr>                                                 <chr>
## 1           55 and press her hand upon her heart whenever Dorothy's~ 110  
## 2           55 "\"Do you suppose Oz could give me a heart?\""        935  
## 3           55 brains, and a heart also; so, having tried them both~ 975  
## 4           55 "rather have a heart.\""                              976  
## 5           55 grew to love her with all my heart.  She, on her par~ 992  
## 6           55 now no heart, so that I lost all my love for the Mun~ 1024

Now I can use this additional context to determine which instances of "heart" are about the Tin Man's quest.

It seems like these tools would be most useful for open-ended responses that are still very categorical, but it could help with determining themes for more narrative data.

The Art of Conversation

There are many human capabilities we take for granted, until we try to create artificial intelligence intended to mimic these human capabilities. Even an unbelievably simple conversation requires attention to context and nuance, and an ability to improvise, that is almost inherently human. For more on this fascinating topic, check out this article from The Paris Review, in which Mariana Lin, writer and poet, discusses creative writing for AI:

If the highest goal in crafting dialogue for a fictional character is to capture the character’s truth, then the highest goal in crafting dialogue for AI is to capture not just the robot’s truth but also the truth of every human conversation.

Absurdity and non sequiturs fill our lives, and our speech. They’re multiplied when people from different backgrounds and perspectives converse. So perhaps we should reconsider the hard logic behind most machine intelligence for dialogue. There is something quintessentially human about nonsensical conversations.

Of course, it is very satisfying to have a statement understood and a task completed by AI (thanks, Siri/Alexa/cyber-bot, for saying good morning, turning on my lamp, and scheduling my appointment). But this is a known-needs-met satisfaction. After initial delight, it will take on the shallow comfort of a latte on repeat order every morning. These functional conversations don’t inspire us in the way unusual conversations might. The unexpected, illumed speech of poetry, literature, these otherworldly universes, bring us an unknown-needs-met satisfaction. And an unknown-needs-met satisfaction is the miracle of art at its best.

Not only does she question how we can use the essence of human conversation to reshape AI, she questions how AI could reshape our use of language:

The reality is most human communication these days occurs via technology, and with it comes a fiber-optic reduction, a binary flattening. A five-dimensional conversation and its undulating, ethereal pacing is reduced to something functional, driven, impatient. The American poet Richard Hugo said, in the midcentury, “Once language exists only to convey information, it is dying.”

I wonder if meandering, gentle, odd human-to-human conversations will fall by the wayside as transactional human-to-machine conversations advance. As we continue to interact with technological personalities, will these types of conversations rewire the way our minds hold conversation and eventually shape the way we speak with each other?

The Evolution of Ghosts

From Hamlet to American Horror Story, ghosts have been part of our entertainment and even our darkest fears. The first (known) ghost stories date back to the Roman Empire, when Pliny the Younger wrote about an old man with chains who haunted his home. People visit grave sites, not just to pay respects to the person buried there, but sometimes to encounter the ghost of that person as well. And people report sightings of famous people who have died, like Anne Boleyn, Benjamin Franklin, and Abraham Lincoln.

But you may be surprised to realize that the way ghosts are portrayed in pop culture has changed over time. And these changes have likely also changed people's reported experiences of the paranormal. (Which, in my opinion, lends some credence to more mundane explanations for supernatural experiences.)

A new book, The Ghost: A Cultural History by Susan Owens, explores how portrayals of ghosts have changed over time:

Susan Owens begins not with the specters of Halloween or some drafty Victorian haunted house, but with this scene where Scrooge is visited by his former partner.

Charles Dickens described Jacob Marley as “transparent,” and laden with “cash-boxes, keys, padlocks, ledgers, deeds, and heavy purses wrought in steel”; otherwise he had “the same face” and garb. Ghosts in the late 18th century and into the 19th century became translucent in part due to new optical shows (like phantasmagoria) and lantern-slides that projected luminous images, as well as the increased use of watercolors in art. “When Dickens made Marley’s ghost see-through in A Christmas Carol, he was drawing on a convention that had only relatively recently been established,” she writes.

Owens is formerly a curator of paintings at the Victoria and Albert Museum in London, and she explains that as she began to research ghosts in art and literature, she found written records dating back to the eighth century.

Owens book explores a variety of cultural and doctrinal forces that shaped our concept of ghosts, such as the English Reformation - rather than considering ghosts to be souls in purgatory, the belief that souls went directly to heaven or hell meant that wandering spirits had to be something else.

As a lover of horror movies and ghost stories, I will definitely check this book out.

He Saves, She Screams

While I was blogging about the ABCs of statistics, Variance Explained was doing some awesome statistical analysis - this time on gender and verb use across 100,000 plot descriptions.

Inspired by collaborator Julia Silge's similar analysis of gender and n-grams in Jane Austen's novels, he used the Wikipedia plots dataset to examine what verb follows the words "he" and "she":

I think this paints a somewhat dark picture of gender roles within typical story plots. Women are more likely to be in the role of victims- “she screams”, “she cries”, or “she pleads.” Men tend to be the aggressor: “he kidnaps” or “he beats”. Not all male-oriented terms are negative- many, like “he saves”/”he rescues” are distinctly positive- but almost all are active rather than receptive.

Women in storylines are not always passive victims: the fact that the verb “stabs” is shifted towards female characters is interesting. What does the shift look like for other words related to violence or crime?

There’s an old stereotype (that’s appeared in works like Game of Thrones and Sherlock Holmes) that “poison is a woman’s weapon”, and this is supported in our analysis. Female characters are more likely to “poison”, “stab”, or “kick”; male characters are more likely to “beat”, “strangle”, or simply “murder” or “kill”. Men are moderately more likely to “steal”, but much more likely to “rob”.

It’s interesting to compare this to an analysis from the Washington Post of real murders in America. Based on this text analysis, a fictional murderer is about 2.5X as likely to be male than female, but in America (and likely elsewhere) murderers are about 9X more likely to be male than female. This means female murderers may be overrepresented in fiction relative to reality.

As always, he gives you all of his code and encourages readers to do some analyses of their own. Hopefully I'll have a chance to play with some of the data myself.

Movie Review: Split

Last night, I went to see Split, M. Night Shyamanlan's new movie. Shyamalan seems to be like cilantro: you either love him or you hate him. And his newest contribution is probably no exception. In fact, I had many mixed feelings about the movie.

The basic premise is this: Kevin is a young man with dissociative identity disorder (DID, previously known as split personality or multiple personality disorder, though these are both antiquated terms). He has 23 confirmed personalities, and there is a potential 24th personality alluded to by the other personalities. At the start of the movie, Claire, Marcia, and Casey are leaving Claire's birthday party and are kidnapped by Dennis, one of Kevin's personalities. He locks them in a room, and though Claire insists they have to fight with every ounce of strength they have, Casey instead says they should watch and wait for an opening. We see flashbacks to Casey's childhood that help show us her approach from what she learned hunting with her father and uncle.

Kevin is also receiving treatment from Dr. Fletcher and sets up emergency appointments with her via email. Dr. Fletcher senses something is seriously wrong in Kevin's life, brought on by a triggering event at Kevin's work, but is met at the appointments by Barry, another of Kevin's personalities (the one who is presumably in control of Kevin and the other personalities) who enjoys fashion design. He spends the appointments instead talking about his latest designs, rather than whatever is bothering him. Dr. Fletcher tries to use her pre-existing rapport with Barry to get at the truth. We also learn that she believes DID is more than simply a mental disorder; it is a way to overcome physical disabilities, and "unlock the potential of the human mind."

In the meantime, Claire, Marcia, and Casey meet 2 of Kevin's other personalities, Patricia, a controlling English woman and Hedwig, a 9-year-old boy who tells them they are going to be sacrificed to the Beast, Kevin's as-yet unseen 24th personality. Casey forms a bond with Hedwig, that she exploits to help them escape. Unfortunately, Hedwig is suspicious and very dependent on Patricia and Dennis (another personality), so this exploitation doesn't get Casey very far. Claire and Marcia make fruitless escape attempts that instead result in them being locked up separately from Casey and each other.

My reactions to the movie are multi-faceted, and really align with my different roles:

As a writer, my response is that Shyamalan is a good storyteller, but as a writer first and filmmaker second, he has a tendency to tell us rather than showing us. He tries to overcome this by telling us then showing us, but it is not always as organic as it should be. For instance, he tells us about a gift one of Kevin's personalities is capable of then shows us for no real purpose in the story other than, "Oh, hey, look what he can do!" That is, his storytelling is primarily dialogue driven and secondarily visually driven, when as a filmmaker, it should really be the other way around. Shyamalan also continues to use his favorite literary concept playbook of red herring and Chekhov's gun.

As a Shyamalan fan specifically, I enjoyed much of the movie, and really liked the tie-in to another work. I won't say more because

But if you're a Shyamalan fan, you'll probably enjoy the movie.

As a psychologist... oh, boy, where to begin. My main issue with the movie is the connection he draws between mental illness and the supernatural. For millennia, people with genuine mental illness were exorcised, and accused of witchcraft or conspiring with the devil, rather than being treated. This was my primary complaint about a recent horror movie, The Taking of Deborah Logan. While Split switches the relationship - rather than the supernatural causing the mental illness, it is the mental illness causing the supernatural - the connection is still problematic.

I also take issue with the fact that Kevin, a victim of horrific child abuse (which is the primary cause of DID), becomes the antagonist. There are so many better examples of how survival of such abuse changes a person that does not result in them being the bad guy. And to be fair, Casey is an example of a victim who becomes stronger as a result. (And yes, the movie passes the Bechdel test.)

So the movie, while trying to be somewhat accurate in its portrayal of DID (but also taking the concept out for a spin) ends up depending on stereotypes and myths. Dr. Fletcher's comment on "unlocking potential" of the human brain had me bracing myself for a "we only use 10% of our brain" myth eye-roll. She fortunately didn't use this myth, but I feel like it was creeping under the surface.

Overall, if you're a Shyamalan fan, I'd recommend checking the movie out. If you're a fellow psychologist (or at least knowledgeable about psychology), you'll probably have many of the same reactions, so be forewarned.

On Star Wars, Leia, and Coming of Age Stories

Not so long ago, in a city not so far away, I watched Star Wars: A New Hope for the first time. There were many viewings after that, and the Star Wars trilogy became part of my regular rotation of movies. Yesterday, after finding out the sad news that Carrie Fisher had passed away, I had many hours in our car ride home from the holidays to think about Star Wars and what it meant to me. Because though I went on to see many movies with Carrie Fisher, that was the first time I encountered her and her badass character - a character I could look up to, who was strong, intelligent, beautiful, and brave. For a little girl who at many times wished she could do "boy" things and felt stifled by many (though not all) of the "girly" things, her character meant a lot to me.

I think Star Wars meant a lot to many people I know, just as so many coming of age stories do. We were all Luke, wishing for bigger and better things, wanting to fight for something, to be brave and heroic. And we all secretly wished we could find out we were special somehow, with some power or ability that made us different from the rest - that gave us a purpose. The story of Luke developing his Force abilities appealed to us in the same way as (to name a more contemporary movie franchise) Harry Potter's magic abilities. I think that's one reason these stories are so enduring. And Luke's story is probably most meaningful to people who were themselves coming of age when they first encountered it - whether as children, teenagers, or young adults, trying to find themselves in the world.

The thing about Star Wars is, there's something in there for everyone, and on repeat viewings I see and experience new things. I have a personal "theory" that while coming of age stories appeal most to the young, the themes that become important to us in adulthood are stories about redemption. While there are many characters in Star Wars with redemptive goals, the best example is of course Darth Vader. At first, Luke wants to fight and defeat him. As he learns the truth about who Vader is, his goal changes to wanting to save him. And in some ways, Vader wants to be saved, but even then, believes it's too late for that. It's unlikely any of us have done anything as awful as what Vader did, but I think most adults have regrets - things they feel bad about and want to atone for in some way. Many of us think we've done too many bad things or gone too far to ever come back from it. But those stories of redemption resonate with us, and help show us that it's never too late to change for the better. It's those messages that appeal to me on repeat viewings of Star Wars now.

What hasn't totally changed over the years is my opinion of Leia. I loved her feistiness, her strength, her willingness to sacrifice for the greater good, and of course, that she had no problem grabbing the gun and basically rescuing herself when needed. I aspired to be like her, and it was wonderful to have a character who "looked like me" that I could look up to. I went on to adore many movies, shows, books, and so on featuring strong female characters. And Leia is also probably why my approach to flirting frequently involves teasing someone; I haven't called anyone a scruffy-looking nerf herder or a scoundrel though (not yet anyway), but you get the idea.

I probably sound like kind of a nerd for saying that when I heard about Carrie Fisher's heart attack, I said a little prayer that she would be okay. No, I don't know her. Yes, I know Leia isn't her only character. And yes in response to a blog post I saw circulating on Facebook (that I won't link to here but really hope the three ghosts of Christmas pay that blogger a visit soon), I realize that she was pretty hard on her body over her 60 years. But I really wanted her to be okay. One of her characters meant a lot to me - as a child and still today - and so she meant a lot to me. Over the years, as I learned more about her, I saw she really was someone I could look up to. Yes, she made her mistakes, if you want to call mental illness and addiction a "mistake," and like so many of us, she was working toward redemption, toward self-improvement. She was a writer who used her words as a way to heal and grow, and also to share her story and connect to others. To show all of us that we could come back from wherever we found ourselves, that our past is not who we are now. And she encouraged us to be proud of how far we had come.

And damn, did she have a way with words:

On Netflix Binges and Emily Dickinson

Yesterday I discovered a series called Black Mirror. It's an anthology show that started on Channel 4, before being picked up by Netflix for its newest season. The most similar show I can think of is The Twilight Zone, which I watched with my dad. Each episode features a stand-alone story with different actors and characters, but they clearly exist in the same universe. One bit of technology that exists across multiple episodes is an implanted device that allows the recording of memories, the ability to see the world through another's eyes through an uplink, and the power to block people completely - when you block someone, they appear as an outline filled in with gray:

You can't see the person; you can't speak to them, in person or otherwise. Even looking at pictures of the person, or remembering events with them, all you see is the gray figure. One character said it was like having your memories vandalized. It's a world that encourages impulsive reactions to others, and those impulsive reactions have lasting consequences.

The current episode I'm watching, Nosedive, explores how people establish their self-worth through social media. In this world, people rate each other - their posts and their interactions. Even total strangers have the ability to rate a person who makes them smile or pisses them off. Your overall rating establishes your place in society, and impacts where you live, what job you can hold, and what medical treatment you can receive. As we follow the main character, Lacie, who is trying to raise her rating so she can get the apartment she wants, she encounters a string of bad luck causing her rating to plummet and everyone around her to shun her. It's much like the block, except now people look at your rating to decide if they're worth your time. They see you, but not really.

Either way, if you're blocked or down-rated, you're nobody. It reminded me of a poem by my favorite poet, Emily Dickinson:

I'm nobody! Who are you?

Are you nobody, too?

Then there's a pair of us -- don't tell!

They'd banish us -- you know!

How dreary to be somebody!

How public like a frog

To tell one's name the livelong day

To an admiring bog!

The message is clear: It's time to stop basing our self-worth on people who don't value us. That may mean accepting that you're nobody to someone. But in the end, what makes the character - and us - happiest are real interactions that aren't based on a momentary up/down judgment.

For Sentimental Reasons

As part of a small research project I'm hoping to undertake (stay tuned!), I did a little research into some of the sentiment analysis packages available in R. Sentiment analysis involves classifying words and phrases as positively or negatively valenced; people conducting sentiment analysis may also want to delve deeper into the precise emotions being portrayed, such as anger or joy. Software conducting sentiment analysis often uses previously existing lexicon databases, that have already done the work of classifying words as portraying different sentiments. One example is the NRC Word-Emotion Association Lexicon, developed through crowd-sourcing on Mechanical Turk; it offers the following sentiments: negative and positive; and emotions: anger, anticipation, disgust, fear, joy, sadness, surprise, and trust.

But as I was looking into details on sentiment analysis for this project, which will include analysis of writing, I stumbled upon this article about sentiment in fiction - specifically how do commercial novels differ from literary fiction (the "greats") in terms of sentiment? They begin the article by offering a summary of what critics and great writers have to say on the subject, which is that to be considered great fiction, you need to limit sentimentality in your writing. To answer the question more fully, though, they conducted a sentiment analysis of 2,000 published novels across 10 non-mutually exclusive categories: Classics, Mystery, Literary Prize-Winners, Bestsellers, Reviewed by New York Times, Science Fiction, Most Widely Held in Libraries Published Since 1945, Young Adult Fiction, Romance, and Victorian novels:

The more a novel contained strongly positive or negative words (abominable, inept, obscene, shady, on the one hand, admirable, courageous, masterful, rapturous on the other), the higher its score. However complex literary sentimentality may be, the assumption is that in order to be sentimental at a minimum you need a sentimental vocabulary.

Below you see a comparison of the varying levels of sentimental vocabulary in each of our 10 categories. The most telling aspect of the graph is the way the novels of the nineteenth century (labeled “VIC”) represent an altogether different world in terms of sentimentality. These are the novels of Charles Dickens, Mary Shelley, Anthony Trollope, Emily Bronte, and their contemporaries. To give you an idea of what this difference is like, the average amount of sentiment vocabulary in the nineteenth-century novels accounts for just under 7% of all words in a given novel. For prizewinning fiction from the past decade, by contrast, that number is about 5.5%. This means that for a given novel of 100,000 words (about the length of Pride and Prejudice), a reader will encounter on average 1,500 more sentimental words, or about seven and half more per page. That’s an enormous difference from a reader’s perspective. In this sense, one could see our current antipathy to sentimentality as a longstanding reaction to a distinct moment in the novel’s history when emotion reigned.

The other noticeable feature about this graph is the way it is not well-sorted according to our distinctions between “high” and “low” categories (“popular” and “serious” might be another way to label them). Some popular genres like Romance, Young Adult Fiction (YA), and Science Fiction (the latter being surprising to us) do use sentimentality to a higher degree than so-called highbrow novels reviewed in the New York Times (NYT) or those that win literary prizes (PW). (Here the difference is significant though less extreme – Romances for example use about 0.75% more sentimental words than prizewinners, or roughly 3-4 more per page.) On the other hand, the most widely held novels in libraries since 1945 (POST45) use levels of sentiment on par with more popular categories like SciFi and YA fiction, just as our high-brow categories like New York Times novels and Prizewinning novels show no difference with more popular groups like the Bestsellers or the Mysteries.

In other words, up to a certain point, sentimentality does not help us distinguish between ostensibly high-cultural things and low-cultural things (or popular things and serious things).

 So what is their overall message for hopeful authors, like myself?

Our advice to writers? Based on the available evidence, if you want to write one of the fifty most important novels in the next half-century, then by all means avoid sentimental language. But if you want to get published, sell books, be reviewed, win a prize or simply make someone happy, then emote away and just write a good novel.

Shakespeare Knows How to Insult Someone

Election Day is almost here! I'm laughing. I'm crying. I'm developing a nervous eye tick. This will be a year for the record books, hopefully for good reason.

Some of my friends have spent countless hours insulting certain political candidates. So much so that they seem to have run out of insults and have decided to just stay off social media until after the results are announced. Honestly, if that's your approach, I'd give it a good week after the election before venturing onto social media again.

But if you'd like to stay in the fray, and want to keep lobbing insults at a candidate, Shakespeare is to the rescue. This site will generate a Shakespearean insult for you. Here's a taste of the insults I received:

Thou spongy plume-plucked boar-pig!

Thou cockered beef-witted dewberry!

[Thine] face is not worth sunburning.

I scorn you, scurvy companion. What, you poor, base, rascally, cheating, lack-linen mate! Away, you moldy rogue, away!

With a Little Help From His Friends

Thanks to research, Christopher Marlowe will now be listed as an official coauthor on three of William Shakespeare's plays, specifically the three dealing with Henry VI:

The new Oxford edition, which will be available in November, was edited by four Shakespeare scholars: Gary Taylor of Florida State University, John Jowett of the University of Birmingham, Terri Bourus of Indiana University and Gabriel Egan of De Montfort University.

Taylor tells NPR the conclusion that Marlowe should be credited as co-author is partly based on a combination of new and old research. In particular, Taylor cited 2009 research by Hugh Craig and Arthur Kinney that analyzed vocabulary from the Henry VI plays and compared it to plays known to have been written by Marlowe, and a 2015 article by John Nance analyzing the prose of Part 2 of Henry VI.

Taylor himself has published scholarly work on Marlowe and Shakespeare, including work from last year titled Imitation or Collaboration? Marlowe and the Early Shakespeare Canon, on which he collaborated with Nance.

This research is a great demonstration of the power of computational linguistics. It's similar to some of the past research I've blogged about, such as this study using sentiment analysis of Trump's tweets. In fact, see this recent post on Natural Language Processing for a more in-depth look.

To put it very simply, every writer, no matter how they were taught or who influenced them, has their own unique style. Some of the unique characteristics are very subtle - simple word choice or sentence length. Others are more noticeable. And of course, writers from the same period and genre will be more similar to each other. So based on the fact that both playwrights were active at the same time in the same location, and dealing with similar content, we would expect Shakespeare's writing to be more similar to Marlowe than, say, Isaac Asimov.

But even after taking that into account, there will be some slight differences. So if the writing of Shakespeare is more similar to Marlowe than we would expect based on their similarity in approach and paradigm, we can start to speculate that the two may have been collaborators, rather than simply rivals. This is hinted at in the film Shakespeare in Love, where Shakespeare discusses his writer's block with Marlowe, and gets some advice and direction for the play that would become Romeo and Juliet. This short scene isn't surprising considering that people have been speculating about Marlowe's role in Shakespeare's plays for a very long time. But while a person may read the work of two writers and notice similar patterns, a computer can really dig into the data. Just as statisticians test against a null hypothesis (that a relationship does not exist) and estimate the probability that if there is no relationship we would observe a difference, these analyses can estimate the probability that, if the two authors did not work together, we would see such similarity in vocabulary and style.

Obviously, there could be alternative explanations for the similarities. One strong possibility is offered in the article:

Carol Rutter, a professor of Shakespeare and performance studies at the University of Warwick, told the BBC, "It will still be open for people to make up their own minds. I don't think [Oxford University Press] putting their brand mark on an attribution settles the issue for most people."

Rutter told the BBC, "I believe Shakespeare collaborated with all kinds of people ... but I would be very surprised if Marlowe was one of them."

As for how Marlowe's vocabulary and style could have made it into Shakespeare's work without direct collaboration, Rutter said: "It's much more likely that he started his career working for a company where he was already an actor, and collaborated not with another playwright but with the actors — who will have had Marlowe very much in their heads, on the stage, in their voices. ... They were the ones putting Marlowe's influence into the plays."

TSMR: Devil

I'm planning to sit down at some point and make a list of all the horror movies I watched throughout October - fortunately, my Netflix viewing history will help fill in a lot of it!  One of those movies was Devil, which I'd seen listed on Netflix before but passed up because the premise was so ridiculous.

Five people are stuck in an elevator.  A police detective on the case must hurry to get them out, because one of the people is the devil.  Yup.

But, it was October, and I knew I wanted to watch some horror movies I hadn't already seen, so I decided to give it a try.

The movie is written (but not directed) by M. Night Shyamalan.  It definitely shows.  Full disclosure: I do like many of his movies, including some that were panned by critics.  But it drives me nuts that he always needs to have this huge twist close to the end.  The twist was really well done in "The Sixth Sense" but felt a little strained in some of his other movies.

Shyamalan also frequently uses two literary elements in combination - these two elements contradict each other, which is how he is able to lead up to the twist.  This might be why some of his other movies, although I enjoyed them, felt much more predictable than "The Sixth Sense" - I've seen enough of his movies to be onto the toolkit he's using.

The first element is the "red herring": I would imagine most people have heard this term.  It is an element of the story that seems to be crucial and allows the characters (and audience) to solve the mystery, when it is actually simply a distraction from the real truth.  In some cases, it is the opposite of what is actually true.  In others, it is simply another potential version of the truth that turns out to be incorrect.  The point is that the red herrings are meant to distract you, not only from the actual truth, but from Shyamalan's other favorite tool: Chekhov's gun.

Anton Chekhov was a writer of both short stories and plays, though he was also a physician. His comment on this dual life was that "Medicine is my lawful wife and literature is my mistress."  He first starting writing as a way to make money, but then began to enjoy it and started honing his craft.  One of his many contributions to literature is what has become known as Chekhov's gun.

Chekhov did not believe in unnecessary details.  He felt that any information given in a story, or any prop on the stage of a play, should be essential to the story.  His example was that if the writer describes a shotgun hanging above the fireplace (or if a loaded gun is on stage during a play), someone had better fire that gun in the story.  If the gun will never be used, there is no point in taking the time to describe it (or show it) to the reader (or audience).

In Shyamalan's work, the "gun" is usually a small element, introduced early in the story, as simply a fact about the character or even an event that happened before the start of the story that later on becomes the key to solving the mystery.

Despite knowing what tricks Shyamalan had up his sleeve, I'll admit that I actually did enjoy "Devil".  I'm not sure if I'd truly consider it a horror movie, though I can see why it was classified that way.  There are a few points in the story where the movie tries to be scary (and sometimes, it is effective).  But I'd consider this movie to be a mystery-thriller with supernatural elements.

Overall, I'd recommend checking this movie out, especially if:
1) you're a fan of Shyamalan's work
2) you want to watch a horror movie (or your friend/roommate/significant other wants to watch a horror movie) but you don't really want to be scared
or
3) you like mysteries, but can also suspend disbelief (especially if you don't believe in the supernatural)

Devilishly yours,
~Sara