Interesting. I've done both with no success, or at least I think I have.. I'll try harder then.
Well, they can still hit you as long as you're still in range of their attacks while they're trying to get to you, but they will stop. You can also just hope that they don't attack your character. Sometimes you get unlucky though.