Does turning listen mode off break the game? i.e. was the game designed around listen mode and is incredibly difficult without it?
Nope. You can upgrade listen mode and what not. But the game was not build around this feature. It's more, if anything, a handhold thing for casual players.