I think it's like a type of unconscious validation for a lot of people. They started watching these shows under the pretense things would be connected in a two-way street, and it never was. Now they're invested all this time in a show under a false pretense, and it sucks when you get into something expecting a result and it never happens.
Of course, ideally no one would care because at the end of the day you should be satisfied by just watching a good show. But things aren't like that, especially when you're peddling a "connected universe" to your audience. It's obvious now that whatever happens in TV-land is separate, but for a while it wasn't. It actually still isn't that clear unless you're familiar with studio politics and interviews tidbits.
Anyway, I used to care about references and connections, but now I don't give a rat's ass. It's cool if they manage to do it, but otherwise, whatever.