On Surveys
Mar. 23rd, 2017 12:26 am![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
dr_tectonicSo the Chapman University Survey of American Fears. It's a thing. Someone on an email list I'm on ranted that the facts that 10% of respondents reported being afraid of zombies and that 32% agreed with a question about the government suppressing information about a fictitious event meant that people are stupid and ignant. My reply turned long, so here ya go.
Dude, come on. That's not even remotely what the poll results mean.
The only thing that you can say with certainty that the first result means is that, if you ask a bunch of random strangers who have no particular reason to give you deep and honest insights into their soul to spend half an hour filling out an incredibly long and detailed online poll (88 different topics, many with followups!) about an intimately personal subject (what makes you afraid), and, after asking about a whole bunch of other potentially scary things, you ask "are zombies scary?", about 10% will click "yeah, sure, zombies are scary."
Does that mean that some people genuinely worry about the dead coming back to life? I suppose there are probably a few such folks out there. But it could mean "I know it's just fiction, but the idea of zombies totally wigs me out." Or "Yeah, zombie movies are my go-to when I want to feel scared!"
It could also mean "Jesus, how long is this damn survey?" Or "Lol, they're asking if people are afraid of zombies. I'm totally gonna say yes to that." Or even "I just click randomly on these polls because the faster I click the faster I get to the incentive."
Similarly, if you ask a question that is predicated on a false premise but has answers that reflect a particular stance or affiliation, sometimes people will pick the answer that reaffirms their identity rather than the one that accurately represents, in a strict sense, their beliefs about the real world.
Sometimes (often, even) people will be confused by the mismatch between the question and their knowledge of current and historical events, and will pick an answer based on misremembered details. Sometimes people will be unsure of their knowledge, and will pick an answer using the question itself as a source of information. Sometimes people will answer the hypothetical presumed by the question.
And sometimes people will get annoyed that the poll is asking stupid questions that are wrong, turn contrary, and say "Yes! Yes I do think the government is hiding fictional things from me! Stupid poll!"
I find none of these things to be cause for shock, alarm, concern, or dismay, let alone weeping.
I mean, think about answering that poll yourself. The first thing I ask is "well, what do they mean by 'afraid'"? Is finding something troublesome or worrying the same as being afraid of it? If I think something is really super-duper-scary, but I think the odds of it actually affecting me are very low, does that make me slightly, somewhat, or very afraid of it? What if I'm only scared of the thing under certain circumstances? I wouldn't even know how to interpret my own answers to this survey, let alone a whole group of people's!
In my opinion, the only way to get useful insights from survey results like these is to look at them relative to one another. People are a lot more afraid of credit card fraud than they are of being mugged. That's possibly informative when it comes to criminal policy. People fear losing their jobs or losing all their data about as much as they fear heights and spiders. That sort of calibrates different kinds of fear against one another, which seems sociologically interesting.
But the absolute numbers are close to meaningless, and basing a supercilious "lookit all these stoopid muggles" rant on them is not a flattering look. Be better than that.
Dude, come on. That's not even remotely what the poll results mean.
The only thing that you can say with certainty that the first result means is that, if you ask a bunch of random strangers who have no particular reason to give you deep and honest insights into their soul to spend half an hour filling out an incredibly long and detailed online poll (88 different topics, many with followups!) about an intimately personal subject (what makes you afraid), and, after asking about a whole bunch of other potentially scary things, you ask "are zombies scary?", about 10% will click "yeah, sure, zombies are scary."
Does that mean that some people genuinely worry about the dead coming back to life? I suppose there are probably a few such folks out there. But it could mean "I know it's just fiction, but the idea of zombies totally wigs me out." Or "Yeah, zombie movies are my go-to when I want to feel scared!"
It could also mean "Jesus, how long is this damn survey?" Or "Lol, they're asking if people are afraid of zombies. I'm totally gonna say yes to that." Or even "I just click randomly on these polls because the faster I click the faster I get to the incentive."
Similarly, if you ask a question that is predicated on a false premise but has answers that reflect a particular stance or affiliation, sometimes people will pick the answer that reaffirms their identity rather than the one that accurately represents, in a strict sense, their beliefs about the real world.
Sometimes (often, even) people will be confused by the mismatch between the question and their knowledge of current and historical events, and will pick an answer based on misremembered details. Sometimes people will be unsure of their knowledge, and will pick an answer using the question itself as a source of information. Sometimes people will answer the hypothetical presumed by the question.
And sometimes people will get annoyed that the poll is asking stupid questions that are wrong, turn contrary, and say "Yes! Yes I do think the government is hiding fictional things from me! Stupid poll!"
I find none of these things to be cause for shock, alarm, concern, or dismay, let alone weeping.
I mean, think about answering that poll yourself. The first thing I ask is "well, what do they mean by 'afraid'"? Is finding something troublesome or worrying the same as being afraid of it? If I think something is really super-duper-scary, but I think the odds of it actually affecting me are very low, does that make me slightly, somewhat, or very afraid of it? What if I'm only scared of the thing under certain circumstances? I wouldn't even know how to interpret my own answers to this survey, let alone a whole group of people's!
In my opinion, the only way to get useful insights from survey results like these is to look at them relative to one another. People are a lot more afraid of credit card fraud than they are of being mugged. That's possibly informative when it comes to criminal policy. People fear losing their jobs or losing all their data about as much as they fear heights and spiders. That sort of calibrates different kinds of fear against one another, which seems sociologically interesting.
But the absolute numbers are close to meaningless, and basing a supercilious "lookit all these stoopid muggles" rant on them is not a flattering look. Be better than that.
no subject
Date: 2017-03-23 09:17 pm (UTC)no subject
Date: 2017-03-27 04:10 am (UTC)Fear is a process that's much more physiological than rational and it's common for certain idea forms in the brain to trigger fears even when the brain has a whole stack of facts that the feared scenario is unlikely or impossible.
I might go so far as to claim that all fear is non-rational. When it's rational, we call it "risk."