We're all anti-fascists, but we should not be antifa at all. Antifa is the left fascist movement and given enough power, they will terrorize everyone like any violent group will.
That being said: Self-defense is our right. But Antifa is in attack mode and aiming at basically anyone who opposes their views. And that sounds strangely familiar to some groups we detest.
Despite your fantasies, antifa has existed for decades and has not actually attacked everyone on sight. But I guess if you wanna defend nazis, you have to build a few strawmen to scare the people.