I get a few annoyances every now and then. It seems to be random videos for me though though, not the whole site. The errors are random too - videos not playing at all, loading slowly, and some loading half way before the red bar jumps to the end, making me have to refresh the whole page.
Weirdly, with sites like Gametrailers and Giant Bomb I can just push play, and watch whole videos without having to wait at all. The videos on those sites load far ahead of the speed they play, and almost never catch up.
I am guessing it is due to the high amount of visitors on YouTube - maybe they got an influx of users recently?