Mmm, I don't think this is a question of legality. It's a question of morality, and what is deemed to be "right". It can be an entirely personal position, where the line to serve should be drawn. Maybe a predominately black bakery doesn't want to make a "kkk 4 eva" cake. Maybe that same bakery also doesn't want to make a wedding cake celebrating an interracial couple. Let's pretend both of these actions were legal - do we, collectively as a society, find them both morally right?
That's where stuff like "voting with your wallet", community engagement, and going to the press comes in.