Will I be a poser if I didn't?
I did. Started with Blue.
I think those who read the manga first would probably be a bit disappointed by the games.
I'd say a lot of people picked up manga after playing Pokemon as it was probably their introduction to Japanese media.