I have been listening to hip-hop since I was like 4 (I'm 39 now) and still listen to it. I don't think it makes you a misogynist or anything in of itself just from listening to it. However, it can contribute to potential dispositions, biases and perceptual frames if they are already there. But this applies to all media and just about everything in your external environment.
Every day most of us get hypnotized, entrained, micro-shaped by what we expose ourselves to. When we're younger this impresses our psyche to a greater degree whereas when we're older it's more of neural reinforcement of what's already there. Because of this at times I'm torn on how to handle and view certain content and experiences when it comes to exposing the youth.
A big problem is while it does affect us, it varies greatly on how much at least on the individual level. With some people the impact is negligible, yet in others, the effect is more pronounced. (other factors and thought patterns cross pollinate each other which creates amplification of said perceptions and frames) But the impact on larger groups and culture, in general, is easier to see on the whole.