That's a matter of developers not knowing how to budget and not understanding the market. Nothing wrong with making the game of your dreams with niche as hell gameplay and visuals, just UNDERSTAND that what you're trying to make is niche and budget accordingly so you can still profit from it.
If you're going to take a chance at getting a mass audience, you better bring what you think they'll like. The games you mentioned have some very glaring drawbacks that prevented them from getting the massive sales you may think they deserve.
Demon's Souls PS5 - It's a remake that absolutely butchered the original's art and music direction. That plus they didn't change the gameplay enough, it's still the first Demon's Souls, which just isn't as good as some of the other from titles that came after and that are readily available.
Returnal - I'm ready to get roasted on this one. I played the game from start to finish, it has great gameplay and boss fights, buuuuuuut...the visuals are unappealing as fuck. When it comes to both the main protagonist (what were they thinking) and how dark/drab everything looks, it just pushes some people away. Bullet hell games are not known for being mainstream. I'm convinced that if this game had a different art direction, it would've done a lot better numbers.
Alan Wake 2 - Loved by critics, but taking a look at GAF's OT, it's a divisive as fuck game. A LOT of people complaining about the gameplay, how the game is very boring, the story goes nowhere etc. That is not what you want for a game that is meant to sell millions.
Helldivers 1 is probably one such case where they probably deserved better sales, but the game being a top down shooter just makes it a more niche proposal. They evidently made enough back that allowed them to change that with the 2nd and hit critical mass.