Friday, September 29, 2017

Just You Wait: Hamilton, Madison, and the Federalist Papers

Completely by accident, the last book I read and the one I'm about to finish have a common story - a study in the 1950s and 1960s that attempted to answer the question: who wrote the 12 Federalist Papers with disputed authorship, Alexander Hamilton or James Madison? First, some background, if you're not familiar with any of this.

By Publius (pseudonym) [Alexander Hamilton, John Jay, James Madison]. - http://www.americaslibrary.gov/aa/madison/aa_madison_father_2_e.html., Public Domain, Link
In 1787 and 1788, a series of 85 essays, collectively known as "The Federalist" or "The Federalist Papers," were published under the pseudonym, Philo-Publius. These essays were intended to support ratifying the Constitution, and influence that voting process. It was generally known that the essays were written by three Founding Fathers: Alexander Hamilton, the 1st US Secretary of the Treasury; John Jay, the 1st Chief Justice of the US Supreme Court; and James Madison, 5th US Secretary of State and 4th US President. The question is, who wrote which ones?

The authorship was not in question for 73 of the essays; each of these essays had a unique member of the trio claiming authorship in the form of a list shared with the public later (in some cases, following the individual's death). The problem is that for 12 essays, both Hamilton and Madison claimed authorship.

Historians have debated this issue for a very long time. In the 1950s, two statisticians, Frederick Mosteller and David L. Wallace, decided to tackle the problem with data: the words themselves. I learned about the study, which produced an article (available here) and a book, first in Nabokov's Favorite Word is Mauve. In fact, that was Ben Blatt's inspiration for book, which involved analysis of the word usage patterns (as well as a few other interesting analyses) of literary and mainstream fiction.

But it was through the book I'm reading now that I learned their approach was Bayesian. I've written about Bayes theorem (and twice more). Its focus is on conditional probability - the probability one thing will happen given another thing has happened. Bayesian statistics, or what's sometimes called Bayesian inference, uses these conditional probabilities, and allows analysts to draw upon other previously collected probabilities (called priors) that may be subjective (e.g., expert opinion, equal odds) or empirically based. Those prior probabilities are then used with the observed data to derive a posterior probability. Bayes was frequently used by cryptanalysts, including the code breakers at Bletchley Park (such as Alan Turing) who broke the Enigma code.

Mosteller and Wallace started off with subjective priors - they went in with the prior that each of the 12 disputed essay was equally likely to have been written by Hamilton or Madison. Then, they set out analyzing the known essays for word usage patterns. This also provided prior probabilities. They found that Madison used 'whilst' and Hamilton used 'while.' Hamilton used 'enough' but Madison never did. They then examined the disputed essays, using these word usage patterns to test alternative scenarios: This essay was written by Madison versus This essay was written by Hamilton. They found that, based on word usage patterns, the 12 essays were written by Madison, meaning Madison wrote 29 of the essays. This still leaves Hamilton with a very impressive 51.

Overall, I highly recommend checking out The Theory that Would Not Die. I'll have a full review on Goodreads once I read the last 20 or so pages. And I think I'm ready to finally tackle learning Bayesian inference. I already have a book on the subject.

No comments:

Post a Comment