There’s one point I didn’t get a chance to get into in the post below about polling, though I allude to it in the part about crosstabs and subgroups. One thing people do when they don’t like what a poll is telling them is to dig into the crosstabs and find things that just don’t add up. Maybe the party mix seems kind of off. Or they’ll show a very Democratic group favoring Republicans or vice versa.
Basically, don’t do this.
It’s just too good a way to fool yourself.
If you really know what you’re doing and can look at polling in general sometimes you can really see issues with sampling. But with a poll of 500 to 1000 people, the number of Hispanics, or African-Americans, or people under 30 is really small. And that means that the margin of error for that subgroup is so big that the number is kind of meaningless. They’re worth looking at, especially if you can see trends over many polls. But the one-off gotcha stuff is something you should avoid or be skeptical of when you’re hearing it from others.
The point works in the other direction too. In the context of extremely negative press coverage about Democrats I often see reporters finding something down in the crosstabs that really looks awful and making a whole story about it. Again, bad idea.
That isn’t to say that people who really understand polling (and I don’t include myself in this group) and are good at avoiding wishful thinking can’t intelligently critique polling. They can. But just be cautious about this.