Would you apply that standard to those who come from other EU countries but do not speak fluent German?
It strikes me that this broader issue is not about speaking German. All EEA countries readily accept other Europeans to live and work there without any test for fluency. I promise I won't take it as an insult if you tell me I should not be allowed to live or work in Germany, but I am, and I'm not sure how we can square this with the expectation here.
It strikes me that this is more about the cultural foreignness and "incompatibility" of those coming. Unfortunately trying to draw a line based on "compatibility" based on religion, country of origin, cultural values etc. is unpalatable, and so people find proxies, like language fluency, because those seem more rational and less discriminatory of a signal about willingness to integrate.
Like I said, it's very understandable to me that we would connect language to employment services, encourage it, make it easy to learn the language, teach the language in school for children, generally pipeline people into learning the language. I just don't see it as a sensible bright line because of how things are already set up.
(I also understand that the volume of new migrants poses a challenge that individual, voluntarily migration from other EU countries does not, but we generally establish rules and regulations that apply to individuals equally irrespective of their broader group membership.)