You're looking at it wrong, or at least in a weird way. The numbers you get from armor are the same as the numbers you get from leveling up. If you want to wear good looking, but weak, armor: go ahead. You can just level up a bit to make up the difference in values. Game plays the same whether you're a highly equipped level 50 or a fashionable level 80. My level 130 dresses like a hobo cause I like the look, not because it gives me any kind of advantage.
Once you complete the main story boss the world changes a lot. There's a lot of new bosses, a giant new dungeon and even some quests to do. This is where you get ready for BBI.
BBI is a giant 3 part dungeon with 3 mega bosses, each unlocking a shortcut once defeated. Several mini shortcuts are available (doors that only unlock one way) and if you're immune to fall damage you can skip entire areas as well. BBI is loaded with cool shit to fight, and is really the game's finest moment. Everything up until BBI doesn't really test you. When you first start BBI it will be frustrating, everything will be highly lethal, hp will seem bloated and the new enemies will be overwhelming. Stick with it. You'll learn and adapt, you'll get skill upgrades and new weapons. After spending a few days in BBI I'm convinced it's the best part of the entire game.
I'm very much story driven in these kinds of games, as well as cool gear driven. Witcher 3 was perfect because while a bunch of gear was ugly as fuck, I was always striving to reach the levels I could upgrade my witcher school gear of choice to look great in, as well as keep playing the story.
Even in Souls titles which is all about the combat, atmosphere, boss fights, etc... I tend to have fun finding new gear I really like and then putting it on.
This game, as fun as it was then as it was now, has burned me out. The gear to put it lightly was uninspired and ugly. I was actively trying to get through story quests and reach new story stages to see if the vendors had unlocked better looking gear for either my pawn or myself. But so much of it were recolors or shit that I don't know who designed it but bleh and dorky, particularly for light armor.
The game felt unsatisfying in this regard because it actively killed my joy of getting cooler gear as the game progresses as it genuinely seemed the more gear I unlocked the more bland it was. I saved up so much money and when I spent it I realized I essentially looked the same.
The challenge might be there in post-game, but after 40 hours of being disappointed stage after stage in this regard only to learn that now I gotta go through a whole separate area to possibly get sweet looking gear I can't even buy myself with all of the money and points I have but instead gotta deal with random loot? Nah.
Maybe I'll change my mind later on and will give BBI a try, but to even get there and go through it proper i probably gotta go grind and do sidequests. And these things are meaningless when I'd only be doing all of this to get a cool look that I've wanted throughout the whole base game. Even the elemental daggers look the same only with recolor.
tl;dr I play these games for cool gear just as much as story and fun, and playing this again reminded me just how disappointing the gear is in looks and use.