1) #4 could overlap with #2 or #3, so it certainly creates error within
the poll. My guesstimate (heavy on the guess) is that less than five percent of those who answered number 4 would overlap with number 3, based on the idea that people answering the question would view the two as entirely different approaches. Your point correctly points out the nuance of an argument (you could want withdrawl in a certain amount of time but want troop buildup to achieve the goal) and most poll respondents probably don't think in such nuanced terms.
My thought: I'll admit that I have a tendency to "overthink" poll questions when I am taking them myself, so I might exaggerate the amount this goes on in the general public. But it is possible to construct a poll without such conflicts.
My friend continues:
2) This is not just a strangely conducted poll, it really is a terrible poll. I hadn't noticed how bad Gallup questions on the war until you pointed it out. #3 is essentially a meaningless question because it allows for people with completely divergent viewpoints to select the response. One person could answer in the positive for #3 because they don't want the US in forever but is uncomfortable with setting a timetable of 12 months or less, even though they really want to withdraw sooner rather than later, while another could answer in the positive because they want the job done right no matter how long it takes. Because it is so ambiguous, or rather that it allows for multiple interpretations of a yes response, it makes an analysis of the grouping of the responses impossible. (The other huge flaw with the question is that what does it mean when they say "take as many years to do this as are needed to turn control over to the Iraqis"? What's control- a weak gov't barely holding off a smaller insurgency? A full-fledged democracy? Something in between?)(And another huge flaw is that the question itself is sort of contradictory- withdraw but take as long as is needed. Well, no shit, people expect us to withdraw eventually- how long is the key question. By placing withdraw in the question, as opposed to asking "Stay until the Iraqis take control" you include respondents similar the first person I describe above.)
I think all of this is correct. In light of that, the most sensible approach to take to such a poll is to ignore it because its results are, from a social science point of view, meaningless. However, they do represent the character of the popular discourse (for what THAT is worth), so I judged it was ok to use the numbers with caveats.
Just so everyone understands, what is being discussed here is not bias. I'm not claiming or implying that in any way, shape or form. We are talking about lousy social science methodology. Someone over at Gallup needs a refresher course. (It is scary to note that the same question has been asked since at least 2005. One wonders what the internal review procedures are over at Gallup. Do they have the equivalent of peer review there?)
No comments:
Post a Comment