Scenario time: You're out somewhere shopping/partying, a random woman you've never met walks up to you and says "Hey, are you such and such?" and you say "Yes? Do I know you?" and she replies "No, but I heard from somebody that you raped them"
Pray tell how you're to react to that?
Do you really think a woman is going to confront a rapist like that?
Usually woman sort of care about this thing called self-preservation and if they hear about a dangerous rapist they aren't going to go and poke the goddamn bear.