You might want to read the work of the French sociologist Pierre Bourdieu. He talks about class differences in behavior and mentions the exact same thing you're doing (The attitude of "If I had that kind of money, I'd know much better ways to spend it"). If he's right, I'd have to guess you've not exactly grown up wealthy...
Or don't read him, his writing is awful. But he has some interesting ideas and observations.
I most definitely did not grow up wealthy but I have managed to do well for myself. I do think there's a difference between growing up wealthy vs. having created wealth in terms of how one's outlook and the value they put on things. Not that I'm anywhere close to being the the league of private jets or 100k evenings at clubs in St. Tropez.