And that pollster is none other than Scott Rasmussen. A story posted at Politico over the weekend discusses the “hardening conventional wisdom among prominent liberal bloggers and many Democrats that Rasmussen Reports polls are, at best, the result of a flawed polling model and, at worst, designed to undermine Democratic politicians and the party’s national agenda.” As a moderate and someone who’s been interested in polling for many years, I try and be as fair as possible when the issue of pollster bias arises. More needs to exist than simply a consistent skewing of poll results from the average for me to believe that the pollster has an ulterior motive or hidden agenda. Who’s to say that the pollster whose results generally swing to one side or the other isn’t, in fact, the most accurate? As Nate Silver explains in his post responding to the Politico story:
What Rasmussen has had is a “house effect”. So far in the 2010 cycle, their polling has consistently and predictably shown better results for Republican candidates than other polling firms have. But such house effects can emerge from legitimate differences of opinion about how to model the electorate.
To his credit, Rasmussen has had some accurate results in recent, high-profile contests (2008 general election and NJ governor’s race, for instance). However, even though Rasmussen Reports has had some good final poll results, evidence seems to continue piling up that more is going on than the utilization of a polling model that normally features a more conservative electorate.
It’s not just the data that Rasmussen’s critics object to — they also have a problem with the way the firm frames questions in its automated polls, which are the staple of its work.
In August, for example, Rasmussen asked respondents whether they agreed or disagreed with the statement “It’s always better to cut taxes than to increase government spending because taxpayers, not bureaucrats, are the best judges of how to spend their money.”
“I think they write their questions in a way that supports a conservative interpretation of the world,” said [Democratic pollster Mark] Mellman. “In general, they tend to be among the worst polls for Democrats, and they phrase questions in ways that elicit less support for the Democratic point of view.”
Nate Silver expands on why Rasmussen should be questioned:
Likewise, Rasmussen recently produced a poll in which they purported to describe the Democratic health care plan to their respondents. Several other pollsters have found that support for the plan increases when it is actually described to respondents, but Rasmussen showed no such increase. However, the second sentence in their description reads: The plans before Congress would prohibit people from choosing insurance plans with lower premiums and higher deductibles. I don’t particularly know where this comes from; Rasmussen claims that its questions came from a ‘summary of the legislation provided by the New York Times’, but such a depiction of the health care policy appears nowhere in the New York Times article. But there it is in the Rasmussen survey, where it appears to be designed to build a relationship in the respondent’s mind between the Democratic plan and higher premiums.
[W]hen they do use unorthodox question wording, nine times out of ten it favors the conservative argument.
Silver also discusses Rasmussen’s choice of subject matter, and explains that “they have a knack for issuing polls at times which tend to dovetail with conservative media narratives.” So, there are clear examples of why Scott Rasmussen shouldn’t be considered the most independent pollster out there. However, I’m sure his polls will become more frequent in 2010, and conservatives (like our Right Coast friends) will continue to cite to these polls disproportionately to argue that President Obama and/or the Democratic Party are in trouble. I get a kick out of how conservatives over-utilize Rasmussen polls, and many times his is the only poll cited in a particular blog post. Conservatives particularly like to talk about Rasmussen’s daily presidential tracking poll, which always has President Obama’s approval numbers significantly lower than the average of other pollsters. This poll has always seemed a little weird to me, because Rasmussen also compares people who “strongly approve” of the president’s job performance with those who “strongly disapprove.” He deems the difference in the numbers the “Presidential Approval Index.” Sounds fancy, but when you think about it, isn’t there a significantly higher number of rabid right-wingers than their left-wing counterparts? In this country, self-described conservatives always outnumber self-described liberals. While a Presidential Approval Index of -18 seems dire for President Obama, should we really be surprised. In any case, the main approve/disapprove figure can be (and has been) called into question, as well. Rasmussen likes saying that he always uses “likely voters” in his polling, as opposed to all voters, or registered voters. This may be the best way to go when polling just before an election, but presidential approval is another matter. Unless I’m missing something, we don’t vote on whether or not we approve of President Obama’s job performance. Why should only likely voters be used, then? The Politico article discusses this:
But critics note that the practice of screening for only those voters regarded as most likely to head to the polls potentially weeds out younger and minority voters — who would be more likely to favor Democrats than Republicans.
Alan Abramowitz, an Emory University political scientist, said there was “huge concern right now” that Rasmussen was polling a universe of largely conservative-minded voters.
“How is Rasmussen selecting likely voters almost a year before the election? And why would you even screen for likely voters in measuring presidential approval?” said Abramowitz. “My guess is that it’s heavily skewed toward older, white, Republican voters.”
There you go. Sure, Rasmussen may be using this voter model, and there’s nothing wrong with doing so, but people should just be sure and think twice before viewing numbers from Rasmussen Reports as the most accurate around, and conservatives in the blogosphere should think twice before citing to Rasmussen 95-100% of the time. I expect a decent amount of bias coming from the other side, but utilizing Rasmussen over and over again gets to be a little ridiculous!
Edit: Nate followed up with another post, entitled, Putting the [R] in [R]asmussen? He starts:
ThinkProgress has discovered, by way of a cool new invention known as The Internet, that Scott Rasmussen has in fact been conducing polling on behalf of partisan clients, in particular the RNC and the Bush re-election campaign, both during 2003-04.
This appears to contradict all but the most absurdly lawyerly readings of a statement on Rasmussen Reports’ website, which reads: “Scott [Rasmussen] maintains his independence and has never been a campaign pollster or consultant for candidates seeking office.” The statement was also repeated word-for-word in a Politico article without any qualification.
OK, so we’ll score this The Internet 1, Politico’s fact-checking department 0, and Rasmussen a negative something for posting a blatantly misleading statement on their website.