I suppose the barrier (at least for me) is the lack of VR games genuinely worth buying a headset for. Sure Alyx is meant to be amazing but really buying a headset just for that? There aren't many. I have a Quest 2 and my favorite VR game so far has actually been Super Hot VR running natively on Quest 2.
If you're not into indie games I can totally understand not finding much to play. In my case it invited me to try and play games even in genres I don't normally like. I hate roguelites with a passion, but in VR I had a blast playing In Death Unchained and Until You Fall, and now I want to check out Synapse and The Light Brigade. That's already 4 games. This is how, after a bit, 50-game backlogs start without realizing.
Forgot about Yuki, damn that's 5 already.