This is argument #2 expressed in the article in the OP.
The form of the argument is as such: color was historically used in an unfair way to create divisions between people. Ergo, the best way to make amends for that division is to discard color as a salient factor in identity. As a result, racism means not seeing colour. This also helpful resolves any anxiety the speaker might feel about their own "lack of identity" (because individuals in the majority do not see their majority characteristics as salient to their identity, in part because criticism over past abuses by those in the majority, and in part because white-ness, male-ness, straight-ness etc are seen as "normal" in culture, so differentiation only occurs through being a minority).
However, this has the impact of denying others dignity. When you argue that "we are all the same", you deny the richness that comes from difference. Historical oppression is bad, but that oppression led to interesting acts of resistance and interesting cultural forms which came from the oppression. Those do not deserve to be co-opted or discarded in an effort to unify us all into a "neutral" identity.
To be fair this is also an argument within progressive movements. For example, the gay rights movement has always struggled between whether to adopt a "normalcy", white collar, middle-class, rights-oriented approach (emphasize parity and access to previously straight-only institutions) or to also celebrate difference (to emphasize differing cultural practices that have emerged in the gay and lesbian communities as a result of historical othering by straight people). Or in race issues, the existence of things like awards celebrating diversity; there's always something torn between emphasizing the positive ways that those awards can call attention, while also recognizing that those things imply in some ways that the recipients could not compete in broader categories. In part resolving this tension means recognizing that although equality is the ultimate goal, equality doesn't magically appear overnight and in part a recognition of difference is important to help close gaps in equality caused by the lingering effects of historical divisions, even as laws today nominally reflect equality.
Perhaps a better approach for you personally, as someone who feels anxious about the use of color language, would be to find someone who is of a minority and ask them why they identify with color language, or what role they think it plays for them.