I listened to 538's podcast where Silver is defending his model and it's just not addressing what strike me as the serious concerns people have. Now, probably he is getting bombarded with a lot of tweets making terrible criticisms, but you can't only address those. He characterizes his critics as not believing that the polls have moved, but the issue is not with 538's now-cast but that polls-only looks almost identical to the now-cast. It's great that you see a lot of volatility in the polling, but then shouldn't you suspect that the polls will continue to be volatile such that you could have predicted that they were probably going to move a lot from wherever they happened to be in mid-August?
At one point he even criticizes other models for not moving a lot after each convention. But of course your model shouldn't move much after the conventions. Conventions aren't surprising! You knew months before the convention that the convention would happen and the candidate would get a polling bounce, so you could have included it in your model beforehand. It's true that if you're trying to build a model from only polls with no information about the timing of important events like this you're going to have issues distinguishing unexpected movement that signals a change in the dynamics of the race from expected movement due to things like conventions. That's a weakness of a polls-only model, and it means that if you're making a polls-only model you should probably not make it very reactive because if you do you're going to see a lot of artificial movement where your model thinks that a totally normal event is actually a big shift in who's likely to win.