The families wouldn't be at the castle. They would be living in the lands.
Hard to say whether people would maintain the thing after Stannis and his forces were completely eradicated. He bought most of his army and why leave a pitiful force to defend an old, dead castle when your plan is to conqeur Winterfell and install yourself there for a season.
How would any of them know that his forces were eradicated? Did they check BBC News on their smartphones?