You could say that about literally any ability.
No. Most abilities only affect the person using them. Swim speed up makes me faster. It doesn't also make my enemy slower. So my enemy doesn't need to stack swim speeds up to cancel that out. How stupid would that be, huh?
Most abilities give the player actual, interesting choices based on how they want to make their personal build. It doesn't matter if the enemy is stacking quick respawns. Good for them. That's not going to influence my choice to stack special charge up because that's what I think is best for my build.
Dmg up and Defense up aren't like that. I don't know if I should use dmg up because I don't know if it's best for my build because I don't know if my enemy is going to run something to cancel it out. Same thing for defense up. There's too many unknown variables to account for. I can't make any real choices, only hope that the enemy I go up against has abilities that don't cancel out mine. And the only weapons that they're really useful for don't actually need them anyway because they're already strong weapons. Yes, as a player it's nice to have a charger that kills at half charge, but as a designer, I wouldn't want that because I purposefully made the charger take that long for a reason.
If they were gone tomorrow, 90% of the game would remain unchanged (except that it would be a cleaner experience without needlessly complex choices), and the other 10% would just go back to normal, possibly being better balanced.
I'm just surprised they were even put in the game in the first place.