by itself, it shouldn't. Over time with enough of them it should make it a better indication of quality, but we also need to look at the failures and see why it failed over a much longer period of time.
Electoral-Vote.com - FiveThirtyEight: An Assessment
- Thread starter brianmcdoogle
- Start date
- Status
by itself, it shouldn't. Over time with enough of them it should make it a better indication of quality, but we also need to look at the failures and see why it failed over a much longer period of time.
Exactly. That's why we have uncertainty attached to these things.
The models always have some reasonable chance of being wrong, and it's impossible to fully estimate confidence. This is because we don't know what we don't know. The models are just based on past data and things we've observed, so at best we can say that given our history, X is likely.
And to respond to the person I'm the other thread saying it was the nowcast in this graph, it wasn't. The nowcast is even more volitile, him thinking the polls only was a nowcast actually shows how it's a problem how volitile Silver's model is. A model should not swing from 50% chance to 80% chance in a relatively short span of time unless something major happens, which is not the case.
Then why not just have the nowcast? The whole point of polls only and polls plus is to predict November, which thr model clearly does no do. A model of the election should not be this volitile. Not in a race like this.
Actually, it's been a remarkably stable year. Clibton has always led, by about a 4%, though the margin has changed from 2% to 8% over time. Every election since the 2000's has been far more stable than those before, because of hyper-partisanship.
Then that means these aggregates can never be wrong. You can literally have any % chance for Hillary to win other than 0/100% and it would be correct.
How do you judge how good a poll aggregate is then?
TheLaughingStock
Member
The haters cover Nate Silver like he covers Trump. They follow everything he does with hyperbole and name drop his brand for clicks. Allow the data to stand on its own and let the man make a living off the masses.
obviously if the prediction is only "Hillary wins" or "Trump wins" it is hard to measure them given there is only one experiment to judge from, ie: the actual election
but all these polls aggregators predict results state per state, and not every model gets every state right
once you have aggregators giving you the same results on every or almost every state, it becomes harder to differentiate and I think that is indeed a problem
It's a hard problem!
The first thing you need to do is let go of "correct" as any kind of metric when it comes to forecasting. "Correct" isn't a thing.
But yeah. I've already said this but you can't evaluate a forecaster on a single prediction. You can first check the methodology and throw out everyone who's doing nonsense things. But after that, you gotta judge a forecaster on their track record. On a long horizon, unlikely events will wash out. It's easy to imagine if we keep the numbers constant: let's say I predict a 10% of rain ten times. If I'm a good forecaster, one of those times will see rain. But really, I should get ~10 in 100 or ~100 in 1000 before we declare me good.
People designing models will use this intuition. You could, say, build a model based on data from every other election, then use your model to forecast the remainder and check how often you're right. If your data set of election results is big enough, you can get a feel for how accurate your model is. But even this is tricky because the time horizon is so big here. A model that works well in the '50s might not work so well today as more and different data become available and underlying facts change. (For example, massive fundamentals changes like Women's Suffrage or Civil Rights could require totally different assumptions be built into a model.) And we just haven't had that many Presidential elections. You might not want to biforcate your data to make a testing set if doing by chance means train without many no-incumbent races, for example.
The big secret of "data science" is that the science is easy but there's a ton of art to doing it well.
DR2K
Banned
She got a huge boost in campaign funds since this FBI debacle
Brexit taught me that polls are interesting but the worst thing you can do is start thinking any of them are infallible. No matter how scientific the method, at the end of the day it's still using a few thousand peoples stated voting intention to project that of 120+ million actual voters.
Best thing you can do is accept that you don't know who will win, no one does. All you can do is vote (and campaign if that's your thing) and let the results fall where they will.
Best thing you can do is accept that you don't know who will win, no one does. All you can do is vote (and campaign if that's your thing) and let the results fall where they will.
Because she has the resources to do so.
It would be stupid to be complacent, when you can both spend time and money in states you need to keep, and the ones you might be able to flip.
She also has a lot of heavy hitters that can target both kind of places, from Potus and Flotus, to Sanders, VP and Warren.
She has a ton of campaign money, and getting more dems in offices benefits everyone including her.
Clinton and Kaine have both held events in Arizona in the last 2 days, a state that hasn't gone blue since 20 years ago. It's been considered her most-realistic long-shot flip, but still a lofty goal. I think the latest tightening took off the table the truly far-reaching states like Georgia/Utah/Missouri/Indiana, but there's no sign of panic from the Clinton team at all. They aren't going back to Virginia for example as far as I know, which was a surprising flip in 2008 and still a major battleground in 2012.
The final week both camps have been focusing on Pennsylvania/Ohio/Iowa/Florida/North Carolina/Nevada/New Hampshire, and in no way whatsoever does Hillary need to win ALL of them. Trump basically does though.
The recent attention from the Clinton team on Michigan/Wisconsin is a liiitle bit eyebrow-raising, but Dems always fret about those states and then do fine (and polls seem...fine). And I think once the "what totally crazy states can we target?" idea becomes not worth it you might as well invest a little back in a better-safe-than-sorry thing with some of your "weaker" states. But I really don't see that as a retreat or a worrisome move at all.
I've been waffling between nervousness and confidence ever since the FBI/tightening stuff last week, but as far as the vibe the campaign itself is giving off I see no signs of alarm. And they have more accurate polling info than we see.
Polls showed the two outcomes for Brexit would flip-flop with each one leading at various times.
Clinton's pretty much been in the lead for for the entire election (it's by how much that's changed and it's only a few outliers that have Trump in the lead).
So they aren't really comparable.
Nothing1016
Member
Clinton is pretty safe. Early voting has been good for Hillary in states like Florida and North Carolina. She also has poll leads in Wisconsin, Michigan, and Pennsylvania. If she captures four of these states, the race is practically over. If she captures all of them, you can expect the senate to flip democrat and there will probably be a closer house of reps.
PollyVote, which has predicted the past three elections with incredible accuracy, their margin of error has been less then 1%, has Clinton winning the popular vote 53 to 47.
Here is their last three election predictions
Obama 2012:
Actual - 52.0%
Prediction - 51.3%
Obama 2008:
Actual - 53.7
Prediction - 53.9
Bush 2004:
Actual - 51.2
Prediction - 51.5
http://charts.pollyvote.com
Please stop bringing up Brexit. If you don't know the reasons why you should look up what the Brexit polls actually looked like, and compare them to what Hillary's poll's look like. There a huge differece. Britain leaving the EU is shocking like Trump winning the nomination was, in the sense that the polls said one thing but our guts another. That's absolutely 1000000000000000% NOT the case here.
I agree that I have some nervousness about polling error, but I reject the reasoning for your argument. Although sample sizes in individual polls may lead to somewhat large margins of error, this is exactly what poll aggregation is designed to combat. The aggregate sample sizes are more than sufficient to make accurate estimates for the population.
The area where there is room for doubt is if there is a systemic error in likely voter models being used by the polls. For instance if turnout among some demographic group is greatly different than expected by the bulk of pollsters. This sort of thing can be partially validated by early voting data, but it only goes so far since early voters are not exactly a representative sample.
balladofwindfishes
Member
She has more money for her campaign than she could ever hope to spend.
superdeluxe
Member
whats interesting is that most major polling place projects a 85% to 99% probability that Clintoin wins. But 538 is at 60% right now? I wonder why 538 is on a such a outlier.
66%, not 60%. Upshot is about midway between 538 and PEC.
http://fivethirtyeight.com/features/election-update-why-clintons-position-is-worse-than-obamas/
It's pretty well-explained.
- Smaller margin than Obama.
- Higher undecideds than Obama.
- Worst electoral map/math than Obama.
Common Knowledge
Member
Wow, that site has Florida clearly in the blue. While I certainly hope Florida ends up going to Clinton, putting it as squarely Democrat right now seems a bit too optimistic.
http://projects.fivethirtyeight.com/2016-election-forecast/florida/
Florida is close, and there's a pretty insane spread in the recent polls. So that leads to a more uncertain position even off of a similar average lead.
gutter_trash
Banned
the thing is that 60%+ of Floridians have already voted, leading up to 70%+ this coming weekend
leaving a 30% to 25% left to vote on actual Tuesday
http://elections.huffingtonpost.com...residential-general-election-trump-vs-clinton
She's led all year in the polling averages. The EV indicates that she might be out performing them because a large proportion of it is from new or lapsed women and Hispanic voters, who don't make it through likely voter screens. 55% of the EV is from women.
I think the spread we're seeing in the polls reflects the increase in polls conducted as election day nears, rather than increased uncertainty.
Nate Silver has talked about it on their podcast. 538's model accounts for a larger potential systemic polling error than other models. That is the main reason. It may also be more aggressive about extrapolating perceived trends than others.
I think the big dilemma for this year is disagreement on the composition of the electorate. Poll variance is generally quite a bit higher this year than last cycle, which implies that the results depend more greatly on assumptions than usual.
The flipside is that the spread may have been artificially low previously, due to herding.
Yeah. Correlations between states and state polls and national polls is a really tough topic to model. I think part of Wang's overestimation of odds his simulations, and his mid-year change in expected volatility based on this year's polling volatility. I don't think that assumption is justified.
Wang doesn't use National polls in his model to estimate state totals. A better direct comparison to 538 would probably be The Upshot, anyways. It's probably part of the reason why PEC has remained at 99 percent for a while now.
More polls. There's been a shortage of good quality polls the last week, which put more emphasis on their trend adjustments.
Also, PPP's batch of state polls for states that have been under polled helped a lot? Maybe? Wisconsin data has been sparse.
Yeah I noticed that as well. They're still incredibly conservative compared to most polls, but they're correcting in states where they clearly appeared to have a weird trend.
e: This is why I find the polling data infinitely more entertaining than the election itself (much how I love advanced stats in baseball more than watching games). The way things get calculated and the models and tools used to adjust and forecast is just so interesting to me. When did the PPP polls come out? E: Quick tweet with the hard numbers. This probably has a lot to do with it.
New @ppppolls dump of battleground state polling:
NH: Clinton 48-43
NV: Clinton 48-45
WI: Clinton 48-41
PA: Clinton 48-44
NC: Clinton 49-47
Like an hour ago. Trump recently had a great round of state polling in those states. PPP has softened some of those fears, despite them not being a great firm.
QuantumZebra
Member
If any of the early voting polls are true - then Republicans are actually voting Clinton.
Every report I've seen thus far shows women at much higher participation %'s in this election than past ones.
It is not a stretch to assume that Republican-registered women are going out and voting Clinton.
In fact it would quite well explain that 28% of Republicans voting for Clinton thing we saw earlier this week in FL.
Netherscourge
Banned
If Hillary wins North Carolina, it'll be an early Thanksgiving miracle.
I just don't believe it. They will stop at nothing to rig it for Donald. They already have.
Those polls are not yet included in their model. You can see the explicit adjustments here:
http://projects.fivethirtyeight.com/2016-election-forecast/updates/
By the way, I wouldn't rack my brain looking at that page and trying to figure out why each update moved the percentage slightly one way or another. It is a stochastic model, so you can probably expect random movement of +/- 1% even if you were adding a null poll.
Glass Shark
Banned
Wasn't aware of this site, thanks for sharing.
Things are still way too close for my comfort, though.
Voter supression isn't that effective. It can make a huge difference in local and statewide elections, but not enough for a national elections where Hillary leads by the ammount she is.
I'd give it a 75% chance NC goes blue on Tuesday.
hodayathink
Member
Why is PPP not a great firm? I thought they had a pretty good rep.
Basileus777
Member
They do, which is why they get a ton of work from campaigns and other interested groups.
Agreed. I'm looking forward to the post-mortems on the effectiveness of different LV screens. My hypothesis is they are missing the new and reactivated voters we're seeing in places like FL, but we'll see.
Ah, I was wrong, I thought they were in the C range, not the B range. They've mentioned irritation with their joke push-polls, but their core work is good it seems.
Polioliolio
Member
I'm glad you guys make it sound positive, but locally, things sound neck and neck.
I'm also super disappointed that Ohio is proving itself to be majority bigot.
I'm also super disappointed that Ohio is proving itself to be majority bigot.
I completely agree with that.
However, electorate selection is beyond the scope of poll aggregation and modeling, so aggregation sites will have a systematic miss if it is true, unless they're biasing their own results or heavily judging and weighting polling firms for reasons beyond their prior performance and contact methodologies.. The campaigns themselves are probably doing extensive modeling that hasn't been made public.
MisterR
Member
Yep, they are B+ rated on 538. Very good group.
https://twitter.com/ppppolls
PPP Polls is done for the cycle, along with one of the best political twitter accounts around. RIP.
PPP Polls is done for the cycle, along with one of the best political twitter accounts around. RIP.
Nope. At 64.2% now, continuing her downward trend.
FL and NC both red, at R+0.3
The current map on 538 is disgusting. I can't believe that much of the country wants Donald Trump to be fucking president.
I think I'm gonna go vote tomorrow, and then turn off the news until Tuesday.
This is a good advice. But I can't take it. See you on the other side.
- Status