I don't know where you lean politically so this statement may be completely unrelated to you: my opinion is that for people on the left, projection is at the heart of this issue.
People know right-wing extremists are dangerous and can fuck shit up, so they have to believe it must be a possibility on the left. I mean, if they dared say, "I don't think it really is", someone might call them biased or partisan. This is another version of the "both sides" myth.
And at the heart of that is a profound ignorance by Americans, including those on the left, of what progressives have done and fight for around the world, as well as just profound ignorance of what they've done and fight for in America. We constantly harp on about how "both parties about the same", then blame them for voting along party lines. That doesn't make fucking sense, but the latter is true: the Democrats typically vote for things that preserve reproductive rights, expand education and healthcare and serve social welfare in general, are backed by science and have been demonstrably effective practiced elsewhere in the world. Republicans typically vote for things that aim to take us back to the 1950's when there weren't any Muslims and black people, women, and other minorities knew their place. Also, we all prayed and had guns. Right-wing groups across the world follow this same mantra of xenophobia, homophobia, mass incarceration, capital punishment, and even violence against perceived criminals with no legal process. This is the modern right wing at its very best.
In summation: what you're worried about is baseless.