For late-comers who don't really understand the frustration on on-timers' part:
When I'm invited somewhere, I always ask the time, because when I make plans, I want to make the most of them. So I want to be sure to space out other things in my schedule to make those plans feasible. I time out the other things I do that day so that those plans will work. When I hear, "be at this place at 8PM," that means that I've got to be out of the door by 7:30 (if not earlier), which means I should be ready to go by 7, which means I should take a shower at 6-6:15, etc etc. I'm particular about this, because I respect another person's/peoples' time, and so I don't want anyone to need to wait around for me or be inconvenienced.
So when I do that, and feel like the other person takes zero consideration for my time, it's annoying. Now, I do think all plans have like a 15-minute grace period where you're not really late. 15-30 minutes late deserves a quick text, which I think is also fine. More than 30 minutes late borders on rude, especially if I'm already there. If it's 45+ minutes, I'm usually just gonna leave unless there's a valid excuse.
Recently, my girlfriend and I were invited to a housewarming party, and when we got there and I texted the host so that he could let me into his apartment building, he told me he was still on the way home from work. When I asked why there weren't any other people standing outside his door, he texted back, "sry, didn't text everyone."
I thought this was rude as fuck, and I wanted to go home at that point, but my girlfriend (perhaps reasonably) advised us to just get a drink at a bar nearby and chill out. The host ended up being very apologetic later, and I probably overreacted, but that, to me, is a real sign of disrespect, and that's core to the reason some people hate late-comers. It's that they feel like the respect they have for you isn't reciprocated, especially when it's downplayed as if someone shouldn't take offense.
I'm probably more of a tight-ass about this than I need to be, but I'm chronically early, so maybe I'm just wired differently.