Nate’s preliminary analysis of pollsters’ performance is out. And the headline is what we’ve all known: Rasmussen is biased. Up to 2009, Scotty’s bias was held in check to some degree. Since then, he’s left it all hang out. And it shows. The proof is in the pudding. (And Nate doesn’t even address Rassmussen’s 12 point final generic).
The 105 polls released in Senate and gubernatorial races by Rasmussen Reports and its subsidiary, Pulse Opinion Research, missed the final margin between the candidates by 5.8 points, a considerably higher figure than that achieved by most other pollsters. Some 13 of its polls missed by 10 or more points, including one in the Hawaii Senate race that missed the final margin between the candidates by 40 points, the largest error ever recorded in a general election in FiveThirtyEight’s database, which includes all polls conducted since 1998.
Moreover, Rasmussen’s polls were quite biased, overestimating the standing of the Republican candidate by almost 4 points on average. In just 12 cases, Rasmussen’s polls overestimated the margin for the Democrat by 3 or more points. But it did so for the Republican candidate in 55 cases – that is, in more than half of the polls that it issued.
If one focused solely on the final poll issued by Rasmussen Reports or Pulse Opinion Research in each state – rather than including all polls within the three-week interval – it would not have made much difference. Their average error would be 5.7 points rather than 5.8, and their average bias 3.8 points rather than 3.9.
Who did well? Q, SUSA (in the last three weeks) and, surpisingly, YouGov:
The most accurate surveys were those issued by Quinnipiac University, which missed the final margin between the candidates by 3.3 points, and which showed little overall bias.
The next-best result was from SurveyUSA, which is among the highest-rated firms in FiveThirtyEight’s pollster rankings: it missed the margin between the candidates by 3.5 points, on average.
SurveyUSA also issued polls in a number of U.S. House races, missing the margin between the candidates by an average of 5.2 points. That is a comparatively good score: individual U.S. House races are generally quite difficult to poll, and the typical poll issued by companies other than SurveyUSA had missed the margin between the candidates by an average of 7.3 points.
In some of the house races that it polled, SurveyUSA’s results had been more Republican-leaning than those of other pollsters. But it turned out that it had the right impression in most of those races – anticipating, for instance, that the Democratic incumbent Jim Oberstar could easily lose his race, as he eventually did.
YouGov, which conducts its surveys through Internet panels, also performed fairly well, missing the eventual margin by 3.5 points on average – although it confined its polling to a handful of swing races, in which polling is generally easier because of high levels of voter engagement.
This cycle was mission accomplished for Rasmussen. Next year, when he keeps putting out five times as many polls as anyone else, do you think the media will ignore his efforts to shape the narrative?
Is there any data on that? I remember that Survey USA seemed to be the only pollster consistently finding Republican candidates ahead among young voters.
I know a lot of young voters stayed home, but I wonder what the vote share for the parties was among the young people who did vote.
at how well YouGov fared.
3.8 vs. 3.3 for Quinnipiac with less bias (0.3 towards the Republicans vs. 0.8 for Qunnipiac.) It will now not be possible for the media or anyone else to discount PPP because it is a Democratic firm.
The one they missed on the most was Alaska (not sure if this is included in the analysis since it was a 3-way race). They had it 37 Miller, 30 Murkowski, 30 McAdams, but that was because they didn’t mention Murkowski’s name in the first question.
was about GOP +7. Gallup’s GOP +15 represents an enourmous miss….
The GOP will continue to be able to paint a more bleak picture.
Rasmussen was so prolific that people begin to believe his stuff because its all there is to ehar. Until a Dem firm steps forward and polls 15 races a day beginning in August he can paint the narrative and ti will become a self-fulfilling prophecy.
I grew insanely terrified this cycle at the power that polls seem to have on voters. It can depress turnout on either side, as we all know. But considering how many people will have lost by 5,000 votes or fewer in SO many races, I really think the Dems need to find someone willing to counteract Rasmussen. No one else is near as prolific and/or biased.
I think he’s simply lazy and cuts corners in ways that make the GOP look good. Nate Silver pretty much ran down the list: no cell phones, no callbacks, one day only calls.
Whatever his true intentions, though, the end result is the same: providing a boost for the GOP. Of the eight polls that put Raese ahead of Manchin, seven of them came from Rasmussen in one form or another (PPP provided the only other one and their polls flipped back to Manchin long before the Ras polls did). Manchin’s own pollster says they were never behind in the whole race.
The one good thing about this is that we’ll know Rasmussen’s true intentions soon enough based on whether he makes significant changes in his polling policies to address Nate’s concerns. If he does, great, we’ll get honest polling. If not, then Democrats can easily call bias in the future and Ras won’t be able to hide behind his once-sterling reputation.
That is where we’ve seen the really big outliers and where we see the most media frenzy. Most of the pollsters tighten things up and usually get things in the ballpark right before the election but further out things get uglier.
I just mention this because these were exactly the polls we here as well as the media have been gnashing our teeth over. And they are the ones that have driven the media narrative which often becomes self-fulfilling.