I realized long ago that the chances of meeting someone through daily life who cares about or even knows a fifth as much about games as me is close to nil so gaming tends to be my personal hobby and I just drink/hike/whatever with friends. Bit of a bummer but such is life, plenty of other things to enjoy with them! Think gaming is becoming more accepted though, haven't had anyone speak shit about me when they come over to hang out and see all of my gaming shit in a long time. Although I keep it neat of course.
Also has a lot to due with who you hang out with I suppose. When people are tripping in their free time they're unlikely to judge you for spending time on games, lol.