It's not so much the availability of choice and consequence, as much as it is the reactivity of the world and the dynamism of the dialogue. It's a key part of a lot of dialogue heavy RPGs and helps build out the world and create a sense that you're part of a much bigger universe. There are moments in Horizon where someone will respond to something you've done in game, and that's pretty cool. I want more of that.
I know it seems like a minor nitpick but it's something that really adds to the overall experience and the feeling of a large and dynamic universe. And Horizon has such a well built world and lore that it's ripe for that sort of thing.
I found conversations to be pretty good and natural, granted not all of them of course. For instance, I'm playing Andromeda now and something I don't like is that pretty much every conversation is an interview, talking to people doesn't flow organically, is just a question after another... I think ME was always like that and I forgot but maybe things changed.
Another thing: Someone said that characters in Horizon aren't memorable, I do agree that most npc aren't unforgetable, although I consider Aloy, Rost, and Sylens pretty great characters. The think is, take The Witcher III for instance, I won't use the "oh but it has three games!" card because I didn't play 1 or 2, just 3. It has a lot of good to great characters while Horizon range from inconsequential to good and I think this is because The Witcher has like 300hs of content and you spent much more time talking to people than battling in comparison with Horizon. Horizon does has good characters is just that they don't have as much screentime and development to go beyond good. I think the idea on this game was to focus on the main ones because of the compact story they wanted to tell and make the rest fill the world and provide context.
There's always room for improvement but I don't think many games handle things as well as Horizon (in this gen at least). You mentioned the exposition dump, I agree is not the best way to tell a story but honestly I loved the story and setting so much and how they revealed key information little by little that I was hooked, how many people can you read here that said "I normally don't read anything but I couldn't stop reading every piece of info in this game!"? Again, I enjoyed a lot but at the same time thought that it was a card that could only be played once... or maybe not, I don't know how a sequel will turn out. In any case even if I don't consider it the ideal case I can't deny I loved it.