Thursday, September 27, 2018

All Thumbs

Recently, I was diagnosed with flexor tendinitis in my left thumb. After weeks of pain, I finally saw an orthopedic specialist yesterday, who confirmed the original diagnosis, let me peek at my hand x-rays, and gave me my first (and hopefully last) cortisone injection. It left my thumb puffy, with a rather interesting almond-shaped bruise at the injection point.

If I thought the pain was bad before, it was nothing compared to the pain that showed up about an hour after injection. So severe, I could feel it in my teeth, and I spent the whole day feeling nauseous and jittery from it. Any movement of my thumb was excruciating. I stupidly ordered a dinner that required a knife and fork last night, but I discovered how to hold my fork between my forefinger and middle finger, with my pinky to steady it, much to the amusement of my dinner companion. I'm sure I looked like a kid just learning to use a fork. Who knows? Perhaps that will be a useful skill in the future.

I slept in my thumb brace with my hand resting flat on a pillow, and even still, woke up after every sleep cycle with my hand curled up and on fire.

Today, my thumb is stiff and sore, but I can almost make a fist before it starts to hurt, and I can grasp objects with it for short periods of time. The pain is minimal. It kind of makes yesterday's suffering worth it.

Best case scenario is that this injection cures my tendinitis. Worst case is that it does nothing, but based on how much better I'm feeling, I'm hopeful that isn't what's happening. Somewhere between best and worst case is that I may need another injection in the future. If I keep making a recovery, and feeling better tomorrow, I would probably be willing to do it again. But next time, I'll know not to try going to work after. I was useless and barely able to type.


I'm moving out of my apartment this weekend and I have a draft of a chapter I'm writing due Monday, so sadly, there will probably be no Statistics Sunday post this week. Back to regular posting in October.

Wednesday, September 26, 2018

Moving: A Haiku

I swim in boxes,
A sea of cardboard, where I
lost the packing tape

Thursday, September 20, 2018

On Books, Bad Marketing, and #MeToo

I recently read a book I absolutely loved. Luckiest Girl Alive by Jessica Knoll really got to me and got under my skin in a way few books have. When I went to Goodreads to give the book a 5-star rating, I was shocked at so many negative reviews of 1 and 2 stars. What did I see in this book that others didn't? And what did others see in the book that I didn't?

Nearly all of the negative reviews reference Gone Girl, a fantastic book by one of my new favorite authors, Gillian Flynn. And in fact, Knoll's book had an unfortunate marketing team that sold her book as the "next Gone Girl." Gone Girl this book is not, and that's okay. In fact, it's a bad comparative title, which can break a book. The only thing Ani, the main character in Luckiest Girl Alive has in common with Amy is that they both tried to reinvent themselves to be the type of girl guys like: the cool girl. Oh, and both names begin with A (kind of - Ani's full name is Tifani). That's about it.

While Gone Girl is a twisted tale about the girl Amy pretended to be, manipulating everyone along the way, Luckiest Girl Alive is the case a girl who was drugged and victimized by 4 boys at her school, then revictimized when they turned her classmates against her, calling her all the names we throw so easily at women: slut, skank, an so on. No one believed her, and she was ostracized. She responded by putting as much distance as she could from the events of the past and the person she was, hoping that with the great job, the handsome and rich fiance, designer clothes, and a great body (courtesy of a wedding diet that's little more than an eating disorder), she can move past what happened to her. She becomes bitter and compartmentalized (and also shows many symptoms of PTSD), which may be why some readers didn't like the book and had a hard time dealing with Ani.

I know what's it like to be Ani. Like her, and so many women, I have my own #MeToo story. And I know what it's like to have people I love react like Ani's fiance and mother, who'd rather pretend the events in her past never happened, who change the subject when it's brought up. It's not that we fixate on these things. But those events change us, and when a survivor needs to talk, the people she cares about need to listen, understand, and withhold judgment.

That marketing team failed Jessica Knoll for what should have been a harsh wakeup call in the age of #MeToo, to stop victim blaming. Just like Ani's classmates failed her. And just like nearly everyone close to her failed Amber Wyatt, a survivor of sexual assault who was recently profiled in a Washington Post article (via The Daily Parker).

Given these reactions, it's no wonder many sexual assaults go unreported. Some statistics put that proportion at 80% or higher, with more conservative estimates at 50%. And unfortunately, the proportion of unreported sexual assaults is higher when the victim knows the attacker(s). They're seen as causing trouble, rocking the boat, or trying to save face after regretting a consensual encounter. It's time we start listening to and believing women.

Wednesday, September 19, 2018

The Road Trip That Wasn’t

I was supposed to go to Kansas City today for a visit but sadly it wasn’t meant to be. My car had been sputtering and lurching this morning. When a check engine light came on I pulled off at the nearest exit and headed to the nearest Firestone, which happened to be in Romeoville. I’ve never been though I haven’t missed much, other than a small airport. I got to watch small planes take off and land while I waited for the diagnostic results. It wasn’t spark plugs. Apparently there’s something wrong with the engine. Do not pass go, do not head to Kansas City, go straight to the dealer. Since it’s a Saturn, which they don’t make anymore, that means one of the few GM-authorized dealers that can service Saturns. At least the nice man at Firestone didn’t charge me for the diagnostic reading. They’ll be getting a 5-star review from me.

So I’m staying in Chicago this week/weekend instead of visiting my family. Le sigh.

Tuesday, September 18, 2018

I've Got a Bad Feeling About This

In the 1993 film Jurassic Park, scientist Ian Malcolm expressed serious concern about John Hammond's decision to breed hybrid dinosaurs for his theme park. As Malcolm says in the movie, "No, hold on. This isn't some species that was obliterated by deforestation, or the building of a dam. Dinosaurs had their shot, and nature selected them for extinction."

This movie was, and still is (so far), science fiction. But a team of Russian scientists are working to make something similar into scientific fact:
Long extinct cave lions may be about to rise from their icy graves and prowl once more alongside woolly mammoths and ancient horses in a real life Jurassic Park.

In less than 10 years it is hoped the fearsome big cats will be released from an underground lab as part of a remarkable plan to populate a remote spot in Russia with Ice Age animals cloned from preserved DNA.

Experiments are already underway to create the lions and also extinct ancient horses found in Yakutia, Siberia, seen as a prelude to restoring the mammoth.

Regional leader Aisen Nikolaev forecast that co-operation between Russian, South Korean and Japanese scientists will see the “miracle” return of woolly mammoths inside ten years.
Jurassic Park is certainly not the only example of fiction exploring the implications of man "playing god." Many works of literature, like Frankenstein, The Island of Doctor Moreau, and more recent examples like Lullaby (one of my favorites), have examined this very topic. It never ends well.

By Mauricio Antón - from Caitlin Sedwick (1 April 2008). "What Killed the Woolly Mammoth?". PLoS Biology 6 (4): e99. DOI:10.1371/journal.pbio.0060099., CC BY 2.5, Link

Sunday, September 16, 2018

Statistics Sunday: What Should I Read Next?

When You Need a New Book to Read I log all of my books on Goodreads. On top of that, whenever I hear about a new book I have to read, I add it on Goodreads, so I remember it. Of course, this means my Goodreads bookshelves are a little out of control. Fortunately, I can use R to dig through my Goodreads to-read shelf and figure out the next book to read and/or buy.

If you're on Goodreads, you can easily download your entire bookshelf, including your to-read books, by going to "My Books" then clicking "Import and Export". On the right side of the screen will be a link for "Export Library". Click that and give it a minute (or several). Soon, a link will appear to download your entire library in a CSV file. You can then bring that into R.

If I'm ever stuck for the next book to read, I can use this file to randomly select a book from my to-read list to check out next. Because I own a lot of books on my to-read list, I'd like to filter that dataset to only include books I own. (Note: You can add books to your "owned" list by clicking on "My Books" then "Owned Books" to select which books you already have in your library. Otherwise, you can keep running the sample function until you get a book you already own or have ready access to. You'd just want to skip the first part of the filter in "reading_list" below.)

setwd("~/Dropbox")
library(tidyverse)
books <- read_csv("goodreads_library_export.csv", col_names = TRUE)
reading_list <- books %>%
  filter(`Owned Copies` == 1, `Exclusive Shelf` == "to-read")
head(reading_list)
## # A tibble: 6 x 31
##   `Book Id` Title    Author  `Author l-f` `Additional Auth… ISBN    ISBN13
##       <int> <chr>    <chr>   <chr>        <chr>             <chr>    <dbl>
## 1  27877138 It       Stephe… King, Steph… <NA>              1501…  9.78e12
## 2     10611 The Eye… Stephe… King, Steph… <NA>              0751…  9.78e12
## 3     11570 Dreamca… Stephe… King, Steph… William Olivier … 2226…  9.78e12
## 4  36452674 The Squ… Kevin … Hearne, Kev… <NA>              <NA>  NA      
## 5  38193271 Bickeri… Mildre… Abbott, Mil… <NA>              <NA>  NA      
## 6  20873740 Sapiens… Yuval … Harari, Yuv… <NA>              <NA>  NA      
## # ... with 24 more variables: `My Rating` <int>, `Average Rating` <dbl>,
## #   Publisher <chr>, Binding <chr>, `Number of Pages` <int>, `Year
## #   Published` <int>, `Original Publication Year` <int>, `Date
## #   Read` <date>, `Date Added` <date>, Bookshelves <chr>, `Bookshelves
## #   with positions` <chr>, `Exclusive Shelf` <chr>, `My Review` <chr>,
## #   Spoiler <chr>, `Private Notes` <chr>, `Read Count` <int>, `Recommended
## #   For` <chr>, `Recommended By` <chr>, `Owned Copies` <int>, `Original
## #   Purchase Date` <chr>, `Original Purchase Location` <chr>,
## #   Condition <chr>, `Condition Description` <chr>, BCID <chr>

Now I have a data frame of books that I own and have not read. This data frame contains 55 books. Drawing a random sample of 1 book is quite easy.

reading_list[sample(1:nrow(reading_list), 1),]
## # A tibble: 1 x 31
##   `Book Id` Title    Author  `Author l-f`  `Additional Auth… ISBN   ISBN13
##       <int> <chr>    <chr>   <chr>         <chr>             <chr>   <dbl>
## 1     14201 Jonatha… Susann… Clarke, Susa… <NA>              0765… 9.78e12
## # ... with 24 more variables: `My Rating` <int>, `Average Rating` <dbl>,
## #   Publisher <chr>, Binding <chr>, `Number of Pages` <int>, `Year
## #   Published` <int>, `Original Publication Year` <int>, `Date
## #   Read` <date>, `Date Added` <date>, Bookshelves <chr>, `Bookshelves
## #   with positions` <chr>, `Exclusive Shelf` <chr>, `My Review` <chr>,
## #   Spoiler <chr>, `Private Notes` <chr>, `Read Count` <int>, `Recommended
## #   For` <chr>, `Recommended By` <chr>, `Owned Copies` <int>, `Original
## #   Purchase Date` <chr>, `Original Purchase Location` <chr>,
## #   Condition <chr>, `Condition Description` <chr>, BCID <chr>

According to this random sample, the next book I should read is Jonathan Strange & Mr Norrell. Now if I'm ever stuck for a book to read, I can use this code to find one. And if I'm in a bookstore, picking up something new - as is often the case, since bookstores are one of my happy places - I can update the code to tell me which book I should buy next.

to_buy <- books %>%
  filter(`Owned Copies` == 0, `Exclusive Shelf` == "to-read")
to_buy[sample(1:nrow(to_buy), 1),]
## # A tibble: 1 x 31
##   `Book Id` Title    Author   `Author l-f` `Additional Aut… ISBN    ISBN13
##       <int> <chr>    <chr>    <chr>        <chr>            <chr>    <dbl>
## 1   2906039 Just Af… Stephen… King, Steph… <NA>             1416…  9.78e12
## # ... with 24 more variables: `My Rating` <int>, `Average Rating` <dbl>,
## #   Publisher <chr>, Binding <chr>, `Number of Pages` <int>, `Year
## #   Published` <int>, `Original Publication Year` <int>, `Date
## #   Read` <date>, `Date Added` <date>, Bookshelves <chr>, `Bookshelves
## #   with positions` <chr>, `Exclusive Shelf` <chr>, `My Review` <chr>,
## #   Spoiler <chr>, `Private Notes` <chr>, `Read Count` <int>, `Recommended
## #   For` <chr>, `Recommended By` <chr>, `Owned Copies` <int>, `Original
## #   Purchase Date` <chr>, `Original Purchase Location` <chr>,
## #   Condition <chr>, `Condition Description` <chr>, BCID <chr>

So next time I'm at a bookstore, which will be tomorrow (since I'll be hanging out in Evanston for a class at my dance studio and plan to hit up the local Barnes & Noble), I should pick up a copy of Just After Sunset.

If you're on Goodreads, feel free to add me!

Saturday, September 15, 2018

Walter Mischel Passes Away

Walter Mischel was an important figure in the history of psychology. His famous "marshmallow study" is still cited and picked apart today. Earlier this week, he passed away from pancreatic cancer. He was 88:
Walter Mischel, whose studies of delayed gratification in young children clarified the importance of self-control in human development, and whose work led to a broad reconsideration of how personality is understood, died on Wednesday at his home in Manhattan. He was 88.

Dr. Mischel was probably best known for the marshmallow test, which challenged children to wait before eating a treat. That test and others like it grew in part out of Dr. Mischel’s deepening frustration with the predominant personality models of the mid-20th century.

“The proposed approach to personality psychology,” he concluded, “recognizes that a person’s behavior changes the situations of his life as well as being changed by them.”

In other words, categorizing people as a collection of traits was too crude to reliably predict behavior, or capture who they are. Dr. Mischel proposed an “If … then” approach to assessing personality, in which a person’s instincts and makeup interact with what’s happening moment to moment, as in: If that waiter ignores me one more time, I’m talking to the manager. Or: If I can make my case in a small group, I’ll do it then, rather than in front of the whole class.

In the late 1980s, decades after the first experiments were done, Dr. Mischel and two co-authors followed up with about 100 parents whose children had participated in the original studies. They found a striking, if preliminary, correlation: The preschoolers who could put off eating the treat tended to have higher SAT scores, and were better adjusted emotionally on some measures, than those who had given in quickly to temptation.

Walter Mischel was born on Feb. 22, 1930, in Vienna, the second of two sons of Salomon Mischel, a businessman, and Lola Lea (Schreck) Mischel, who ran the household. The family fled the Nazis in 1938 and, after stops in London and Los Angeles, settled in the Bensonhurst section of Brooklyn in 1940.

After graduating from New Utrecht High School as valedictorian, Walter completed a bachelor’s degree in psychology at New York University and, in 1956, a Ph.D. from Ohio State University.

He joined the Harvard faculty in 1962, at a time of growing political and intellectual dissent, soon to be inflamed in the psychology department by Timothy Leary and Richard Alpert (a.k.a. Baba Ram Dass), avatars of the era of turning on, tuning in and dropping out.

This is a great loss for the field of psychology. But his legacy will live on.

Thursday, September 13, 2018

Don't Upset a Writer

There's an old joke among writers: don't piss us off. You'll probably end up as an unflattering character or a murder victim in our next project.

There's another joke among writers: if you ever need to bump someone off or dispose of a body, ask a writer. Chances are he or she has thought through a hundred different ways to do it.

The thing about these jokes, though, is that writers usually get our retribution through writing. We don't tend to do the things we write about in real life. If someone legitimately encouraged us to cause real harm to another person or actually help in the commission of a crime, the answer would likely be a resounding "hell no." That's not how we deal with life's slings and arrows. Writing about them is generally enough to satisfy the desire and alleviate the pain.

But not in all cases.

If there were any rules about committing a crime (and I'm sure there are), rule #1 should be: don't write about it on the internet before you do it. And yet, a romance novelist may have done just that. Nancy Crampton Brophy has been charged with murdering her husband; this same author also wrote a blog post called "How to Murder Your Husband":
Crampton Brophy, 68, was arrested Sept. 5 on charges of murdering her husband with a gun and unlawful use of a weapon in the death of her husband, Daniel Brophy, according to the Portland Police Bureau.

The killing puzzled police and those close to Daniel Brophy from the start. Brophy, a 63-year-old chef, was fatally shot at his workplace at the Oregon Culinary Institute on the morning of June 2. Students were just beginning to file into the building for class when they found him bleeding in the kitchen, KATU2 news reported. Police had no description of the suspect.

In Crampton Brophy’s “How to Murder Your Husband” essay, she had expressed that although she frequently thought about murder, she didn’t see herself following through with something so brutal. She wrote she would not want to “worry about blood and brains splattered on my walls,” or “remembering lies.”

“I find it easier to wish people dead than to actually kill them,” she wrote. “. . . But the thing I know about murder is that every one of us have it in him/her when pushed far enough.”
It's kind of surprising to me that Brophy's books are romances and yet apparently involve a lot of murder and death. I don't read romance, so maybe murder and death is a common theme and I just don't know it. Of course, as I was thinking about last year's NaNoWriMo book, I surprised myself when I realized that the genre may, in fact, be romance. One without any murder, though.

This year's book will be an adventure/superhero story. And I'm excited to say I'm getting a very clear picture in my head of the story and its key scenes, an important milestone for me if I want to write something I consider good. I didn't have that for last year's project, which is why no one has seen it and I haven't touched it since November 30th. It was by design that I went into it without much prep - that's encouraged for NaNo, to write without bringing out the inner editor. And I'll admit, there are some sections in the book that are beautifully written. I surprised myself, in a good way, with some of it. But all-in-all, it's a mess in need of a LOT of work.

Tuesday, September 11, 2018

Inside the Nike Call Center

One more post before I leave work for the day - Rolling Stone interviewed an employee of the Nike call center, who requested he remain anonymous. His responses are interesting, though sadly not surprising:
So, what has this week been like for you?
Bittersweet. A lot of us have more respect for our company than we have in the past. We feel a big swell of pride that we stood up for something meaningful. But we’ve been getting harassed like crazy.

Normally it’s like whatever, I’m getting paid for it. But this week hit home. With any job you’re going to have to deal with some abuse. Sometimes you can just go with it and apologize, but with this there was no reasoning with it. It felt like it was the proverbial klan’s mask: looking at someone and not knowing their identity, wanting to take off the mask, but you’re getting your ass whooped so you can’t.

What about donating?
Only one group of people called to say they were donating.

A group of republican moms told me they meet up to talk about local news and politics and because of this they have stopped all their support of Nike. They said, “We gathered up all our kids’ Nike stuff and we’re going to go donate it. All our kids play sports but will find another brand.”

I said “Wow, that’s really nice. That’s good. I’m happy for you.”

They basically said, “F*** you, it’s not funny. We’re never going to give another cent, we’re going to talk to our brokers and withdraw our stock.”

No Take Bachs

About a week ago, Boing Boing published a story with a shocking claim: you can't post performance videos of Bach's music because Sony owns the compositions.

Wait, what?
James Rhodes, a pianist, performed a Bach composition for his Facebook account, but it didn't go up -- Facebook's copyright filtering system pulled it down and accused him of copyright infringement because Sony Music Global had claimed that they owned 47 seconds' worth of his personal performance of a song whose composer has been dead for 300 years.
You don't need to be good at math to know that this claim must be false. Sony can't possibly own compositions that are clearly in the public domain. What this highlights though is that something, while untrue in theory, can be true in practice. Free Beacon explains:
As it happens, the company genuinely does hold the copyright for several major Bach recordings, a collection crowned by Glenn Gould's performances. The YouTube claim was not that Sony owned Bach's music in itself. Rather, YouTube conveyed Sony's claim that Rhodes had recycled portions of a particular performance of Bach from a Sony recording.

The fact that James Rhodes was actually playing should have been enough to halt any sane person from filing the complaint. But that's the real point of the story. No sane person was involved, because no actual person was involved. It all happened mechanically, from the application of the algorithms in Youtube's Content ID system. A crawling bot obtained a complex digital signature for the sound in Rhodes's YouTube posting. The system compared that signature to its database of registered recordings and found a partial match of 47 seconds. The system then automatically deleted the video and sent a "dispute claim" to Rhodes's YouTube channel. It was a process entirely lacking in human eyes or human ears. Human sanity, for that matter.
Does Sony own copyright on Bach in theory? No, absolutely not. But this system, which scans for similarity in the audio, is making this claim true in practice: performers of Bach's music will be flagged automatically by the system as using copyrighted content, and attacked with take-down notices and/or having their videos deleted altogether. There's only so much one can do with interpretation and tempo to change the sound, and while skill of the performer will also impact the audio, to a computer, the same notes and same tempo will sound the same.

Automation is being added to this and many related cases to take out the bias of human judgment. This leads to a variety of problems with technology running rampant and affecting lives, as has been highlighted in recent books like Weapons of Math Destruction and Technically Wrong.

A human being watching Rhodes's video would be able to tell right away that no copyright infringement took place. Rhodes was playing the same composition played in a performance owned by Sony - it's the same source material, which is clearly in the public domain, rather than the same recording, which is not public domain.

This situation is also being twisted into a way to make money:
[T]he German music professor Ulrich Kaiser wanted to develop a YouTube channel with free performances for teaching classical music. The first posting "explained my project, while examples of the music played in the background. Less than three minutes after uploading, I received a notification that there was a Content ID claim against my video." So he opened a different YouTube account called "Labeltest" to explore why he was receiving claims against public-domain music. Notices from YouTube quickly arrived for works by Bartok, Schubert, Puccini, Wagner, and Beethoven. Typically, they read, "Copyrighted content was found in your video. The claimant allows its content to be used in your YouTube video. However, advertisements may be displayed."

And that "advertisements may be displayed" is the key. Professor Kaiser wanted an ad-free channel, but his attempts to take advantage of copyright-free music quickly found someone trying to impose advertising on him—and thereby to claim some of the small sums that advertising on a minor YouTube channel would generate.

Last January, an Australian music teacher named Sebastian Tomczak had a similar experience. He posted on YouTube a 10-hour recording of white noise as an experiment. "I was interested in listening to continuous sounds of various types, and how our perception of these kinds of sounds and our attention changes over longer periods," he wrote of his project. Most listeners would probably wonder how white noise, chaotic and random by its nature, could qualify as a copyrightable composition (and wonder as well how anyone could get through 10 hours of it). But within days, the upload had five different copyright claims filed against it. All five would allow continued use of the material, the notices explained, if Tomczak allowed the upload to be "monetized," meaning accompanied by advertisements from which the claimants would get a share.
 

Monday, September 10, 2018

Dolphin Superpods

A thousand dolphins gathered in Monterey Bay, California, hunting for bait fish. While this gathering - like a dolphin food festival - usually happens farther from shore, this gathering happened closer to the coastline, allowing a rare glimpse of the dolphin superpod:

Sunday, September 9, 2018

150 Years in Chicago

This morning, I joined some of my musician friends to sing for a mass celebrating the 150th anniversary of St. Thomas the Apostle in Hyde Park, Chicago. In true music-lover fashion, the mass was also part concert, featuring some gorgeous choral works, including:
  • Kyrie eleison from Vierne's Messe Solennelle: my choir is performing this work in our upcoming season, and I give 4 to 1 odds that this movement, the Kyrie, ends up in our Fall Preview concert (which I unfortunately won't be singing, due to a work commitment)
  • Locus iste by Bruckner
  • Thou knowest, Lord by Purcell
  • Ave Verum Corpus by Mozart
  • Amen from Handel's Messiah (which my choir performs every December)
The choir was small but mighty - 3 sopranos, 4 altos, 4 tenors, and 5 basses - and in addition to voices, we had violin, trumpet, cello, and organ. The music was gorgeous. Afterward, I ate way too much food at the church picnic.

I'm tempted to take an afternoon nap. Instead, I'll be packing up my apartment.

Statistics Sunday: What is Standard Setting?

In a past post, I talked about content validation studies, a big part of my job. Today, I'm going to give a quick overview of standard setting, another big part of my job, and an important step in many testing applications.

In any kind of ability testing application, items will be written with identified correct and incorrect answers. This means you can generate overall scores for your examinees, whether the raw score is simply the number of correct answers or generated with some kind of item response theory/Rasch model. But what isn't necessarily obvious is how to use those scores to categorize candidates and, in credentialing and similar applications, who should pass and who should fail.

This is the purpose of standard setting: to identify cut scores for different categories, such as pass/fail, basic/proficient/advanced, and so on.

There are many different methods for conducting standard setting. Overall, approaches can be thought of as item-based or holistic/test-based.

For item-based methods, standard setting committee members go through each item and categorize it in some way (the precise way depends on which method is being used). For instance, they may categorize it as basic, proficient, or advanced, or they may generate the likelihood that a minimally qualified candidate (i.e., the person who should pass) would get it right.

For holistic/test-based methods, committee members make decisions about cut scores within the context of the whole test. Holistic/test-based methods still require review of the entire exam, but don't require individual judgments about each item. For instance, committee members may have a booklet containing all items in order of difficulty (based on pretest data), and place a bookmark at the item that reflects the transition from proficient to advanced or from fail to pass.

The importance of standard setting comes down to defensibility. In licensure, for instance, failing a test may mean being unable to work in one's field at all. For this reason, definitions of who should pass and who should fail (in terms of knowledge, skills, and abilities) should be very strong and clearly tied to exam scores. And licensure and credentialing organizations are frequently required to prove, in a court of law, that their standards are fair, rigorously derived, and meaningful.

For my friends and readers in academic settings, this step may seem unnecessary. After all, you can easily categorize students into A, B, C, D, and F with the percentage of items correct. But this is simply a standard (that is, the cut score for pass/fail is 60%), set at some point in the past, and applied through academia.

I'm currently working on a chapter on standard setting with my boss and a coworker. And for anyone wanting to learn more about standard setting, two great books are Cizek and Bunch's Standard Setting and Zieky, Perie, and Livingston's Cut Scores.

Wednesday, September 5, 2018

Lots of Writing, Just Not On the Blog

Hi all! Again, it's been a while since I've blogged something. I'm currently keeping busy with multiple writing projects, and I'm hoping to spin one of them into a blog post soon:

  • I'm still analyzing a huge survey dataset for my job, and writing up new analyses requested by our boards and advisory councils, as well as creating a version for laypeople (i.e., general public as opposed to people with a stats/research background)
  • My team submitted to write a chapter for the 3rd edition of The Institute for Credentialing Excellence Handbook, an edited volume about important topics in credentialing/high-stakes testing; I'm leading our chapter on standard setting, which cover methods for standard setting (selecting exam pass points) and the logistics of conducting a standard-setting study
  • I've already begun research for my NaNoWriMo novel, which will be a superhero story, so elements of sci-fi and fantasy and lots of fun world-building
  • Lastly, I'm developing two new surveys, one for a content validation study and the other a regular survey we send out to assess the dental assisting workforce