Varys says he wants the good of the realm but he destroys any semblance of stability it has. Also i don't buy the whole Littlefinger Varys rivalry, obviously they're adversaries but they probably don't see each other as the enemy, that was just played up by the show.
It's made pretty clear what Varys considers best for the realm in ADWD -- Short-term stability isn't it. He's on team Aegon.