Actually managed to defeat my first invader, got him down to barely any health, he hid behind an enemy (one of those huge knights in Heides' Tower) and healed, I killed the enemy while evading the phantom, and took him out too. Felt good, but I still really hate getting invaded. Need to find the covenant that will protect me from that.
that reminds me, I really wish I'd taken the seed as my starting gift. It's supposed to make enemies hostile to invaders, as well.