I respect your take, but I disagree.
The world of Harry Potter has a lot of potential game like. First theres 4 houses, that means alot of opportunities to see different takes on the storyline(for instance what happened in a Gryphindor happens differently from a Ravenclaw). Another point that its crutial is the core aspect of the house, both goods and bads. For instance, Slytherin is well know for people who ends up being "evil", so this could influence in the dialogues of characters through the story(like for instance a character not trusting you because of your house).