That PPP poll is bogus. After reading up on what was going on with the trolls, I can start to see why Nate Cohn has problems with the poll.
Here's Cohn:
Yesterdays North Carolina survey brought signs of the problem. For some reason, possible Republican Senate candidates like Mark Harris, Heather Grant, and Greg Brennon received higher favorability ratings among very liberal voters than conservatives.
I asked PPP if they had an explanation for this phenomenon. They had an answer: Looks like there were [a] dozen respondents pressing 1 on every question so they would be favorable toward everyone and very liberal.
Is that possible? Yes. PPP is an automated polling firm, which means the respondent listens to an automated recording and responds by dialing a number (if youre 18-29 years old, press 1). So its not impossible to imagine that a few respondents might decide to troll PPP by dialing 1 in response to every answer. For todays North Carolina survey, those voters would be classified as white, 18-29 years old, female, very liberal, Democratic, supporting Kay Hagan for reelection, and having a favorable impression of every candidateincluding the Republicans.
So, essentially, people exactly like me. What effect does it have on the outcome?
At first glance, this might seem like a minor error. But its not. It doesnt just influence favorability ratings, and its a bigger deal than a dozen respondents makes it seem.
Why? The all 1 respondents are treated as 18-29 years oldsand 18-29 year olds get more weight. The need for weighting young voters is simple: young voters have lower response rates, and, just for good measure, PPP doesnt call voters with cell phones. In a randomly selected survey from October 2012, 18-29 year old voters were just 6.7 percent of PPPs final sampleeven after PPP randomly deleted old, white voters. To compensate, PPP weights its 18-29 year olds by 2 foldso PPP probably doubled the weight of these 12 all 1 respondents. In a poll with a base of 600 respondents, thats not small.
All in all, the trolls could have representated about 4 percent of PPPs weighted sampleand they unanimously voted for Kay Hagan, who was also option 1. Given that Hagan held a lead of no more than three points against three of her Republican challengers, its quite possible that the trolls were decisive.
...
Disengagement is a mistake, because there are many unanswered questions in addition to the questions implicit in the critiques I've already offered. For instance, only 9 percent of 18-29 year olds think the ACA rollout was "very successful." If there were 12 "all 1s," I'd expect no fewer than about 14 percent of 18-29 year olds to get on board. So what's going on?