Wednesday, March 7, 2018

Statistical Sins: Gender and Movie Ratings

Though I try to feature my only content/analysis/thoughts in my statistics posts, occasionally, I encounter a really well-done analysis that I'd rather feature instead. So today, for my statistical sins post, I encourage you to check out this excellent analysis from FiveThirtyEight that uncovers what would qualify as a statistical sin. You see, when conducting opinion polling, it's important to correct for discrepancies between the characteristics of a sample versus population, characteristics like gender. But apparently, IMDb ratings also show discrepancies, where men often outnumber women in rating different movies, sometimes as much as 10-to-1. And if you want to put together definitive lists of best movies, you either need to caveat the drastic differences between population and raters, or make it clear that the results are heavily skewed by one gender.
The Academy Awards rightly get criticized for reflecting the preferences of a small, unrepresentative sample of the population, but online ratings have the same problem. Even the vaunted IMDb Top 250 — nominally the best-liked films ever — is worth taking with 250 grains of salt. Women accounted for 52 percent of moviegoers in the U.S. and Canada in 2016, according to the most recent annual study by the Motion Picture Association of America. But on the internet, and on ratings sites, they’re a much smaller percentage.

We’ll start with every film that’s eligible for IMDb’s Top 250 list. A film needs 25,000 ratings from regular IMDb voters to qualify for the list. As of Feb. 14, that was 4,377 titles. Of those movies, only 97 had more ratings from women than men. The other 4,280 films were mostly rated by men, and it wasn’t even close for all but a few films. In 3,942 cases (90 percent of all eligible films), the men outnumbered the women by at least 2-to-1. In 2,212 cases (51 percent), men outnumbered women more than 5-to-1. And in 513 cases (12 percent), the men outnumbered the women by at least 10-to-1.

Looking strictly at IMDb’s weighted average — IMDb adjusts the raw ratings it gets “in order to eliminate and reduce attempts at vote stuffing,” but it does not disclose how — the male skew of raters has a pretty significant effect. In 17 percent of cases, the weighted average of the male and female voters was equal, and in another 26 percent of cases, the votes of the men and women were within 0.1 points of one another. But when there was bigger disagreement — i.e. men and women rated a movie differently by 0.2 points or more, on average — the overall score overwhelmingly broke closer to the men’s rating than the women’s rating. The score was closer to the men’s rating more than 48 percent of the time and closer to the women’s rating less than 9 percent of the time, meaning that when there was disagreement, the male preference won out about 85 percent of the time.

In the article, a table of the top 500 movies (based on weighted data) demonstrates how gender information impacts these rankings - for each movie, the following are provided: what the movie is currently rated, how it would be rated based on women or men only, and how it would be rated when data are weighted to reflect discrepancies in the proportion of men and women. Movies like The Shawshank Redemption (#1) and The Silence of the Lambs (#23) would generally remain mostly unchanged. Movies like Django Unchained (#60) and Harry Potter and the Deathly Hallows: Part 2 (#218) would move up to #34 and #50, respectively, while Seven Samurai (#19) and Braveheart (#75) would move down to #59 and #112, respectively. And finally, movies that never made it on to the top 250 list, like Slumdog Millionaire and The Nightmare Before Christmas, would have rankings of #186 and #199, respectively.

12 comments:

  1. I was surfing the Internet for information and came across your blog. I am impressed by the information you have on this blog. It shows how well you understand this subject. hdfilme.cc

    ReplyDelete
  2. I wish more authors of this type of content would take the time you did to research and write so well. I am very impressed with your vision and insight. 3x aternativas apurogol

    ReplyDelete
  3. Presently, motion pictures can assist you with transforming you, and they can do it while you completely drench yourself in the stimulation. On the off chance that watching films in a performance center doesn't seem like some tea, an ever increasing number of purchasers are going to "home theater" frameworks; which can jump on the costly side rapidly. Latest Hindi Movie Download Website Visit: ShareMKV.com

    ReplyDelete
  4. The top 1,000 movies are available at the terrarium tv application, now people can download movies to watch offline. terrariumtvultimate.com provides users the United States and United Kingdom to trust.
    Download terrarium tv for devices!
    Download Terrarium TV App For Smart TV Free
    Download Terrarium TV App For PC Free
    Download Terrarium TV App For Android Free




    ReplyDelete
  5. I wanted to thank you for this great read!! I definitely enjoying every little bit of it I have you bookmarked to check out new stuff you post. 9xrockers

    ReplyDelete
  6. Pretty good post. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I'll be subscribing to your feed and I hope you post again soon. Big thanks for the useful info. Latest & Updated Version

    ReplyDelete
  7. All your hard work is much appreciated. Nobody can stop to admire you. Lots of appreciation. jalshamoviez

    ReplyDelete
  8. I am fully satisfied with the information you have posted. Good job. Keep posting
    Click here

    ReplyDelete
  9. I enjoy you because of all of the effort on this web page. My daughter loves carrying out research and it’s really easy to see why. Almost all know all regarding the dynamic means you convey both interesting and useful steps through your web site and as well as invigorate participation from other individuals on this area of interest so our favorite girl has been being taught so much. Have fun with the remaining portion of the year. You are always doing a superb job. putlocker

    ReplyDelete
  10. You produced some decent points there. I looked on-line for the problem and discovered most individuals goes in conjunction with with your site. watch hd movie online freemovie7

    ReplyDelete
  11. Hello I am so delighted I located your blog, I really located you by mistake, while I was watching on google for something else, Anyways I am here now and could just like to say thank for a tremendous post and a all round entertaining website. Please do keep up the great work. Dolittle 2020

    ReplyDelete