Seems to be a few things for him. The rabid fanbase, think he didn't care for the older group of fans hounding him everywhere he went and the crew. I forget what it was, that the crew wasn't respectful or he wanted the higher ups to respect the crew. I remember reading about the crew and how that was an influence.
Here's some choice quotes from the man himself, draw conclusions as you wish:
"I thought to remain, which would have made me a lot of money and given me huge visibility, the price I would have had to pay was to eat a lot of shit. I’m not being funny about that. I didn’t want to do that and it comes to the art of it, in a way. I feel that if you run your career and– we are vulnerable as actors and we are constantly humiliating ourselves auditioning. But if you allow that to go on, on a grand scale you will lose whatever it is about you and it will be present in your work. If you allow your desire to be successful and visible and financially secure – if you allow that to make you throw shades on your parents, on your upbringing, then you’re knackered. You’ve got to keep something back, for yourself, because it’ll be present in your work. A purity or an idealism is essential or you’ll become– you’ve got to have standards, no matter how hard work that is. So it makes it a hard road, really.
You know, it’s easy to find a job when you’ve got no morals, you’ve got nothing to be compromised, you can go, ‘Yeah, yeah. That doesn’t matter. That director can bully that prop man and I won’t say anything about it’. But then when that director comes to you and says ‘I think you should play it like this’ you’ve surely got to go ‘How can I respect you, when you behave like that?’ So, that’s why I left. My face didn’t fit and I’m sure they were glad to see the back of me. The important thing is that I succeeded. It was a great part. I loved playing him. I loved connecting with that audience. Because I’ve always acted for adults and then suddenly you’re acting for children, who are far more tasteful; they will not be bullshitted. It’s either good, or it’s bad. They don’t schmooze at after-show parties, with cocktails."
If I had to put it down to three things it is:
1) Disagreements with elements of the production and the BBC itself - the BBC's attitude towards a blockbuster show is very different than towards, say, the lower-key dramas he usually does with them. As soon as the show began to air and became this national phenomenon again, I think he was uncomfortable. He was already unhappy, the press/public reaction to the first 2-3 episodes, which was basically hysterical, sealed his decision. This stuff, which has reached a new zenith with the way Capaldi was announced for the role and then paraded around the world, is not what he wants at all.
2) His feeling that he's a "proper actor" and it's the sort of role that might not be befitting. A bit like how being James Bond is at once hugely prestigious and hugely looked-down-upon. He'd rather be on stage doing Shakespeare.
3) In line with 2, his dislike for the adult fanbase of what he absolutely perceives as a children's show. In interviews, he's said he feels like adults should grow out of things like Doctor Who and seek out 'deeper' drama. He likens Doctor Who to a 'gateway' drug for kids - get them with good stuff young, and they'll graduate to liking proper drama instead of soap operas and reality TV. He sees the adult fans as people who aren't 'graduating'.