Discrimination is more than burning crosses and banning people from restaurants. It's also making assumptions about how other people live as being like how you live. When you say "I'm colour-blind, I don't discriminate", what you lose is the way that ethnicity is not just people of colour being equal to white people, but also the way in which ethnicity has contributed to that person's sense of identity in a positive way; the sense of connection, the shared heritage, the way your family and country's history contributes to the way you feel today. It's the different cultural and historical things--the names, the cooking, the music, the heroes, the triumphs, and the tragedies. Certainly this kind of passive discrimination is less bad than openly causing other people harm, but it's still a problem. In other words, diversity doesn't just imply equality, it also implies a capacity for pluralism or difference.
Clearly hurting someone or banning them from having rights is WRONG. But just because you give someone equal rights, doesn't mean you are respecting the ways in which they're different.
One concept related to this is privilege. Privilege means that we were all born with certain benefits. Like, by being born in a developed country, people like you or me don't really know what it's like to starve on a daily basis, right? Well the same is true for your gender. If you're male, you are not the victim of sexism. If you're straight, you're not the victim of homophobia. If you're part of the majority ethnic group, you're never seen as "different" or "foreign". If you were born without a disability, you've never felt unable to achieve your potential because of barriers based on disability. If you haven't struggled with mental illness, you might not know what it is not to be able to get out of bed because of crippling depression. If you do well socially or are neurotypical, you might not know what it's like not to understand a social situation and be paralyzed by missing social cues. It goes on.
But besides all these things, the fact that you don't feel those stings of discrimination also limits your ability to understand when people do. Basically even really empathetic people who really care about others only know the life they've lived themselves, they can't ever know what it's like to be someone else.
So in this context, the privilege that occurs is that even if someone who is white doesn't believe in legal discrimination against people of colour, because they don't understand other aspects of people of colour's lives, they tend to assume that there aren't other things going on, that since discrimination is solved, race no longer means anything. Because for people in the majority, many parts of their identity they just take for granted because those parts are expressed in the world around them, so they assume that other people share those parts even if they don't.
The short version here is that the worst kind of discrimination... the hate, the laws, the violence, the racial slurs... that's just the tip of the iceberg. Ethnicity is much more complicated. And when you treat it like it doesn't matter, you're missing the ways it does for other people.