I'm well aware of those details, I just don't find them compelling because I don't feel like they are handled in any sort of consistent way. There don't seem to be any clear guidelines to the reader about when there is any sort of consequence to using it, and It only seems to cause Guts problems when it is convenient to the story (which has been barely ever so far). Because the reader doesn't know the rules, it can be used as a deus ex machina whenever Miura needs it.
Well then you don't seem to understand details or connect them together. We've been informed from multiple sources what the Berserk Armor will do to someone both a knowledgeable magic user in Flora and a one time wearer in the Skull Knight, both their warnings are dire despite the power the armor imbues. We've seen that direct effect on Guts both physically and spiritually and Flora had to mute the Armor's pull on its user with that extra magical seal because its so dangerous to its user when unrestrained. Its basically meant to be suicide armor and not to be used over and over by the same user over an extended period of time.
We've seen what its doing to Guts, how it damn near kills him to keep him fighting or do you not remember the spikes grown by the armor that pierce a limb and crudely and painfully put it back together so the user can keep fighting away, not that they feel any of this pain or really feel anything but a torrential flow of the most negative and dangerous emotions and feelings. Its like getting sucked away by the undertow of a huge wave, there is nothing you can do and it should drown you, except you know its your soul. Then there is the nerve damage, the fact he's literally losing feeling in his body or the fact that he's sometimes barely able to walk because the armor plus the wounds from Slann are literally destroying him physically and spiritually.
It needs someone like Schierke to keep him from tearing his friends apart as well as from consuming him entirely and worse yet he has to literally make a deal with his own personal devil and work "hand in hand" with the Beast to make sure he can use the armor without it killing him or searing his very soul away completely. He has to rely on the most horribly, negative and disgusting part of himself to avoid the most horribly, negative and disgusting part of his mind consumes him as its unleashed by the armor.
Then there is the fact that you're also neglecting that Guts, even in Berserk armor mode, is still using trickery, applying his many weapons, and just being clever. The very fight with Grunbeld he disabled his shield by using his cannon arm and catching Grunbeld by surprise. Or the fact him and his worst enemy had to work together to defeat Ganishka in Vritrannis, or that he nearly drowned in the battle with Daiba, and so on and so forth. He gets out of many of these fights by being both insanely skilled, experienced and out right crazy. The armor just allows him to fully utilize this while also damn near ending him in of itself.
The armor is insane but Guts is now taking on things like the Seagod. Him and a small group of people, skilled ones but none the less mortal beings, killed a god. Guts has to eventually take on the God Hand and he has to have the ability to actually hurt him. Which means you neglect what is really Guts' true trump card, the Dragon Slayer. Its literally becoming an anathema to astral beings and cuts through them with far more force than just a huge hunk of iron shaped into a parody of a sword ever could, even swung by a beast like Guts. It wasn't the Berserk armor that took out Ganishka or the Sea God and so forth, it was the Dragon Slayer.