If people baulked at the price of the XB1, what makes you think they'll jump on a headset that'll cost more than the console?
HoloLens would have way more applications than a game console. It's a stand-alone computing platform for productivity, communication and entertainment.
You can't really compare it with a game console, because people would buy it for completely different reasons.
And don't get me wrong, I think it is likely that PS4 will end up outselling the XB1 by quite a margin. What I mean, is that it isn't set in stone and there are a lot of upcoming factors (AR/HoloLens, Windows 10, Morpheus, games, marketing etc) that are not really predictable so that it isn't far-fetched to think that it's possible that XB1 can still come close to PS4's sales. Like if Minecraft 2 was only on XB1, that could make a huge impact on XB1's sales/popularity. It's largely hypothetical though.