So are you mad too when you use your sledgehammer and it can't bust open a shrine door? Your complaint makes little sense.
No, because they're clearly not doors, and you have a sheikah slate activator right next to them. The thought never occurs. Besides, there's tons of stuff you can't break in the game by just hitting them. Most things, actually. Climbing, on the other hand, is something you can do absolutely everywhere BUT in shrines/dungeons.
But to be fair, it's mostly annoying in beast dungeons because of the mechanic in those places. You feel like you're not in control - which is ironic, considering they're the only place in the game where you can actually control the environment.
What you're missing is that climbing has nothing to do with the giant puzzles designed in the beasts and shrines.
The amount of freedom given to solves shrine puzzles is awesome so I have zero clue what you mean by restrictive.
I'm not missing anything. I understand that the puzzles wouldn't work if you could just climb.
The bigger point I'm making is that, by the way they're designed and (mostly not) integrated into the world, the shrines and beast dungeons feel out of place, like they belong in a different game. They're separate areas, they require loading, they have a different esthetic from the overworld (I'm well aware of the visual cues with the guardian designs, but stlil), and you can't use any of the game's key gameplay mechanics in them. They feel tacked on. I enjoy a good puzzle every now and then, and I actually like that they're now byte-sized and distributed over the entire map, but, either through design and/or because of technical limitations, they break the flow of the game. If Nintendo could at least find a way to remove loading times in the next games, or integrate them in the overworld, that would be better already. I'd also love it if they managed to design puzzles that can work with the climbing mechanic.
If they create Zelda style temples in future BotW DLC or game climbing will be removed overall or at certain parts. A lot of the puzzles have to do with landscape in temples.
I won't miss climbing if they design dungeons in a way that doesn't make you wish you could climb and if those dungeons are more combat-focused, like a mix between a maze and a gauntlet or something. Kinda like Hyrule castle, which I hear is designed a bit like that. Haven't been there yet, so don't take my word for it.