It is a valid argument, because after a few decades there will be many areas in Europe where the amount of Muslims far surpasses secularists or any other groups. If there is no ideological reform from their side, then our society has to change and take into account the wishes of this growing community. That means many things, building more places of worship and not insulting holy figures, and also making some other cultural adjustments (more modesty).
That's just a reality and there should be an open discussion about the realities of immigration and cultural dynamics. Right now a lot of politicians are just living in a make believe world where everything will just work itself out in time and that creates frustration and the rise of these far right parties.
You're arguing for tyranny of the majority, which is prevented by legal rights. You may as well argue in favour of eliminating legal rights and broader human rights.