Yesterday, I talked about beta/Type II error and described things we can do in our study to decrease beta. The things I described were more about study methods though and making certain you're conducting a rigorous study to reduces bias/outside influences. While in many research fields, we discuss methods and statistics separately, they're connected, and methods can and do impact your statistical analyses.
The cleanest study is one where you can control your participants' entire situation - what they see, what they have access to, even what they've seen in the past. This is impossible, of course. But we do our best to recreate this perfect situation and (when it's possible) count on random assignment of participants to groups to even out any pre-existing differences. Once again, because probability. It's unlikely that every person who would react as you expected ends up in one group (say your experimental group) while every person who would not have reacted as you expected end up in another; the chances of that happening are small (but remember, not 0 - it's completely possible, though unlikely, to flip a coin and get 100 heads in a row).
Weird things happen, because probability, and we also cannot control everything. We certainly can't randomize everything. There are some conditions it would unethical or impossible to manipulate in our study. In those cases, we can still take them into account by using them as control variables. Control variables are variables that we think will affect our outcome (dependent variable), but are not the actual thing we're studying (independent variable). We deal with them by measuring them and using them in our analysis. I'll go into more detail later on about exactly what's happening when we use control variables in our analysis, but essentially we're factoring that information out, so we can get a clean look at the relationship between the variables we're interested in (independent and dependent variables).
In my caffeine study, I might want to use some pre-existing information about my participants as control variables. Characteristics, like gender, which might affect their reaction to caffeine are possibilities. Another is how much caffeine they ingest regularly. I wouldn't want them to have any caffeine before my study, and if a participant showed up saying he or she had had caffeine that day, I would probably throw their data out. But I can't control how much caffeine they consume before they even signed up for my study. If they drink a lot of coffee each day, they might react to the treatment differently than a person who drinks very little. I can collect that information from a short questionnaire, then use that as a control variable in my analysis.
People make for such unpredictable subjects! I'm very sensitive to caffeine, my love of it versus my reaction to it are often in conflict.ReplyDelete
Thanks for a very interesting post!
Candidly; The Work of Writing: Two
A Bit 2 Read