I can't go into whether the games necessarily need combat or not because that just goes into personal opinion territory, but I will say that partly what made the older games exciting was not only that sense of adventure, but the other aspect you're forgetting was the thrilling feeling of careful platforming. That feeling of traps being around every corner, judging a jump to make sure that you could nail it, and nailing those more difficult jumps leading to secrets that helped you progress through the game better. The level design in addition to this made them pretty special experiences.
I'm not the best at explaining, but Tomb Raider 1-4 weren't like other platformers that tried to copy Mario in everything they did. There was a specific pacing to them that made you feel the pressure and weight of messing up a single jump. That was lost in the I.P.'s transition to the PS3/360 generation and then it was fully gone when the series started to become an Uncharted-like with the current big trilogy. There's an alternate reality here where the developers should have sat down and really looked at what made Tomb Raider what it was and improved on that specific formula, but now that space is open for someone else who might want to claim it one day.
You're right about the clunky and slow controls, and the reason you feel that way is because the subgenre never had a chance to improve and evolve past the PS1/Dreamcast generation(the generation of 3d games infancy) aside from two Legacy of Kain Raziel games, because no one else gave it a shot to do so. Again, it needed a dev to come in and stick to it and refine it, like Fromsoft did with Kings Field leading up to Souls games, or like what the new Doom devs did with Doom 2016, until it revived a subgenre and was solidified as it's own lane of difficult/precise platformers.