There is no real right or wrong answer, but I personally hate the ignorant is bliss defense. Too much shit happened in history because of people being ignorant. Case in point, look how people deal with racism in the US. Put their head in the sand and pretend it doesn't exist, but that's another matter.
Again, there is no assumption in my statement. My statement is purely on what the show have shown and told us, not the assumption that some random character that the writers haven't created may feel this way.
It was shown through Gowasu and even then he never claimed his life was shortened by Goku's actions. And we already now the future isn't a sure thing in Dragon Ball, so them seeing the future is always a flawed argument.
So, because the story doesn't want to blame Goku too much, it doesn't show people getting mad by their existence being cut short and in theory some character the writers didn't create can have these feelings? This isn't real life, this is a story and the characters feel whatever the writers what them to feel. As of right now, there is no character that feels cheated by Goku, not even the person who rightful should of had these feelings.