Getting somewhere on time shouldn't be stressful.
I've been late to work (I count 5 min past start time as 'late' here) about 2-3 times in 6 years, and this is with public transport, sometimes up to 3 buses. It's not difficult, just leave earlier. Don't assume optimal conditions. If there's a bus that will get you there just on time, get the previous bus. Sometimes I get the bus before that to make sure I don't get somewhere late.
If you're finding it stressful getting somewhere on time, leave earlier. You shouldn't need to rush, or drive fast, just leave earlier than you absolutely need to and you can take your time. It is far, far less stressful to get to places early than aim to get there on time.
If you're worried about being early (this is crazy to me, I never knew this was a concern until reading this thread), just get a kindle or a 3DS or put a podcast on and go for a walk around the block until you're on time. If it's an appointment, maybe you can go in early.
I think a lot of people have difficulty judging how long things takes, especially getting from point A to point B and all the little things that you forget about (grabbing keys, finding a jacket etc). My suggestion is to actually time how long these things take. Start a timer on your phone the second you decide to leave, then stop the timer when you get to the destination. Did it take more or less time than you expected? I actually tend to overestimate how long things take due to taking a 'worst case scenario' point of view (i.e. what if I miss a bus?), which helps me to get to places on time.
In my case, I do feel something that I can only describe as anxiety when I am late for something, especially if others are waiting on me, so that does kind of help drive me to get to places on time or early. If I'm late for something it's just about always due to me waiting for someone else, and you can be sure that I'm not happy about it.
I generally don't mind if someone else is running a few minutes late though, but if it's something like a movie's start time or something that starts at a designated time, I don't like waiting for someone to arrive past the start time.