I've thought about this before. You touch on a very good topic.
I think the line can be blurred but at the risk of making both sides weaker, i.e. "walking simulators" where the interactivity is borderline nonexistent. The alternative is TLoU's compartmentalized approach, which is split into three forms: combat scenes (practically devoid of story), cutscenes (pure story, no interaction), adn walking simulator (the loop of ladders, pallets and loops where characters can talk because you're barely doing anything). The risk with that approach is it feels too piecemeal, too fragmented -- a constant shift of mental states: "Now I'm in this mode," "Now I'm in another mode."
It's not continuous, like you say. And like you, I'm not sure what the right balance is, if there even is such a thing.