Blaming the messenger

Democrats are attacking Rasmussen Reports because they don’t like the numbers he’s been producing:

Of course, President Obama’s numbers have been plummeting in nearly every poll, but Rasmussen comes in for particular opprobrium because his polls have consistently shown Democrats and liberal opinions a few points behind most other polls. There are three remarks worth making.

First, as Glenn Reynolds puts it, attacking the pollsters is seldom a good sign. Put more simply, attacking the polls is loser talk. I know this from personal experience. Back in 1992, I and many other Republicans were in denial that George Bush could actually lose the presidential election to someone like Bill Clinton. The talk on the right was all of how the polls were wrong. Of course, they were not and Clinton won the election handily. So when Democrats attack the polls, I think we’re seeing a similar phenomenon on the left.

Second, there is a simple reason why Rasmussen’s polls have been more friendly to Republicans than other polls. Unlike most other pollsters, Rasmussen polls likely voters all the time. (The Politico story gets around to mentioning this point near the bottom of page two.) Most pollsters poll registered voters or even the general public most of the time, and shift to likely voters just before elections, resulting in a bump for the GOP right around elections. I think Rasmussen is right to use the same methodology all the time.

Third, and most importantly, Rasmussen’s national polls are objectively among the best. We can benchmark a poll’s quality by comparing its election predictions with actual results. RealClearPolitics has a table of the final 2008 polls from various pollsters.

Quite a few polls did a good job of producing a good central estimate, but with varying margins of error. To measure the quality of polls against each other, we want a measure that takes both the poll’s accuracy and precision into account. To do this, I calculated the root-mean-square error for each poll, evenly balanced throughout the margin of error. (ASIDE: Basically, I’m assuming a uniform distribution throughout the indicated margin of error. This isn’t exactly right, of course, but the published numbers don’t give enough information to reconstruct the true distribution, and it would be too much trouble to do so anyway.) I computed the error based on Obama’s share of the two-party vote, which eliminates discrepancies due to treatment of minor parties and “don’t know” responses.

The result is a single measure that indicates how well each poll predicted the 2008 election. Rasmussen and Pew come in first, with accurate estimates and small margins of error. ABC/Washington Post, Fox, IBD/TIPP, and NBC/WSJ also did well:

1 Rasmussen 1.34
1 Pew 1.34
3 ABC/WPost 1.76
4 Fox 1.86
5 IBD/TIPP 1.98
6 NBC/WSJ 1.98
7 CNN/Opinion Research 2.05
8 Ipsos/McClatchy 2.11
9 CBS 2.18
10 Gallup 2.19
11 Diageo/Hotline 2.28
12 Fairmodel 2.31
13 Battleground (Lake) 2.36
14 Reuters/C-SPAN/Zogby 2.62
15 Marist 2.64
16 CBS/NYT 2.89
17 Battleground (Tarrance) 3.39
18 Newsweek 3.63

Note that the Fair model came in 12th place. The Fair model is not a poll, but an econometric model devised by Yale economist Ray Fair that predicts the outcome of presidential elections. The Fair model does no polling and doesn’t even know who the candidates are, but it still does as good a job at predicting election outcomes as many major polls. Any poll that comes in behind Fair can be safely regarded as a bad poll — you would be better off not polling at all.

The results speak for themselves. In 2008, Rasmussen and Pew were the best, and several polls were literally worse than no poll at all.

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s