I wouldn't worry too much, unless, by some miracle, he is massively successful in his first 2 years and soars to a 90% approval rating, then you can panic.
That assumes he could even win in the general election. People underestimate the broader US electorate when discussing Trump. As of right now, he's a "Republican" problem. The GOP's fear comes from the fact that Trump is taking advantage of an image they have been trying to change since Obama was elected. They saw the numbers and knew if they didn't gain broader appeal they would have a much harder time winning the Presidency in the future. But the "cancer" was already there and Palin was one of the first signs of it. Since then they've been powerless to stop it. Mostly it's because of their magnificent ability to control their message and stay in lockstep, which is great in an unchanging world. But the world does change and so the GOP ran smack into that cancer they created.
So the question is; Will Trump actually appeal to the broader electorate? My answer is no, I do not believe his antics will sell there. But I do hope he is nominated. The GOP needs to hit rock bottom, for their own good as well as the nation's, and the only way that will happen is in a very harsh rebuke from the general electorate. Which I believe will happen.
Now, if I'm wrong (and I might be) and Trump is elected, he will have to be enormously successful right out of the gate to keep the pendulum from swinging back in the mid terms. At the very least he has to not make too many waves so as to keep liberal voters at home during the mid terms.
Trump's success has more to do with the GOP than the US as a whole. The GOP set out to create a hardened block of voters that will turn out and win them elections, and they did that. Problem is, that block of voters are an immovable object, so when the the GOP saw a need to appeal to Latinos, that block went for the guy calling them rapists. Oops.
America is a big, diverse place. We have our faults and crazy people just like the rest of the world, but we aren't bat shit insane. Trump is really nothing to be concerned about. His supporters don't represent all of America and the world has changed since the turn of the century. The likelihood of a Hitler 2.0 is really small.
I also think the prospect of having Trump's legacy being a ridiculous fucking wall, will put a stop to those plans. It'll be Trump's Gitmo. Something that will be way more complicated in reality.
TL;DR
Trump won't win and if he does, America is a big lumbering global machine that doesn't turn on a dime. Honestly of the Republican field right now, Trump is the one I am the least afraid of. Cruz is fucking evil and smart enough to be super damaging. Rubio will probably give up when it gets tough or just be a puppet.