What works better in public opinion
polling: (1) “live” phone calls with a
trained interviewer asking questions,
or (2) ‘Robo-Polls’ where a computer
plays pre-recorded questions, and
the voter presses a phone button
to give their answers?
San Diego-based John Nienstedt
believes ‘live’ is best, and spends
the money to train and employ
skilled interviewers. He has clients
all over the USA, so it is working
for him!
Today he reviews Scott Brown’s 2010 Senate win in Massachusetts, and concludes the ‘live’ pollsters got the more accurate results.
___________________________________
Head to Head in Massachusetts
by John Nienstedt, CERC
“The date is January 19th 2010; the setting is Scott Brown beating Martha Coakley with 52% of the vote in the most watched Senate race of the year. For this analysis we are only using public polls conducted after the first of the year to avoid any confounding by the holidays.
In the robopoll corner we have eight polls. The technique’s most visible practitioner, Scott Rasmussen weighs in with two polls, PPP has two, Pajamas Media/Cross Target has two, MRG has one and Insider Advantage has one. On the “live” side are two ARG surveys and a Suffolk/7News survey.
Here is how we are evaluating the polls: We first back out the small number of votes for third party candidate Joseph Kennedy. We also back out the undecided voters. Polls whose results are then within the margin of sampling error (MoSE) of the actual election outcome and whose results had Brown winning are judged to be accurate polls. Let’s look at the robopolls.
* Rasmussen had Brown losing in both polls, so he had two misses.
* The first PPP poll had a MoSE of +/- 3.6 and had Brown with 51%, well within range and the second PPP poll had a MoSE of +/- 2.8 and had Brown with 52%, so PPP had two accurate polls.
* The first PJM/Cross Target poll had a MoSE of +/- 3.2 and had Brown with an astounding 58% of the vote. Although it had Brown winning, this poll is inaccurate. It was such an outlier among other polls that Pajamas Media commissioned a second one the day before the election. That poll had the race 55% for Brown and, with a MoSE of +/- 4.1, that gives PJM/Cross Target one accurate poll out of two. We should note that, for practical campaign purposes, a poll conducted the day before the election is worthless.
* The Inside Medford/MRG poll had a MoSE of +/- 4.1 and had Brown with 57%. Inside Medford/MRG was therefore inaccurate with its only poll.
* The Politico/Insider Advantage poll, another survey taken the day before the election, had a MoSE of +/- 3.5 and had Brown with 55%. So this firm was accurate
with its only poll.
So four of the eight robopolls were accurate.
Now let’s turn to the live polls.
* The first Suffolk/7 News poll nailed the outcome right on with 52% for Brown, so its only poll was extremely accurate.
* The first ARG survey also nailed the exact percentage for Brown and its second one, with a MoSE of 4.0% had Brown at 54%. ARG was therefore accurate with both its polls.
So the final tally in this head-to-head matchup is a clear victory for the live methodology, with a batting average of 1.000 versus a .500 success rate for the robopolls. Obviously, although these three live polls were accurate, live polls will not nail the outcome every time. However, what is disturbing is that the automated polls were only as good as coin flip.
Clients paying for those polls were throwing their money away half the time . . . alas, if they only knew — ahead of the election — which half were the good ones!
Another interesting point to come out of this analysis is that these robopolls tended to lose their accuracy when they were conducted more than a day before the election. As cell phone-only households become more pervasive, we believe robopolls will become even less reliable. CERC will stick with live surveys.”
________________________________________________________________
Jim Sills is a San Diego political consultant. If you have questions about your future in San Diego and California elections you can contact Jim at this e-mail address: YourElectionVictory@hotmail.com
He has aided the campaigns of Rep. Darrell Issa, Assemblywoman Shirley Horton, Senators Joel Anderson and Tony Strickland, Rep. Devin Nunes, and Assessor/Recorder Greg Smith, among others.