I was unaware this had derived so nicely into a Scouring battle, I would have arrived sooner.
I just got finished a semester in a course about Tolkien, and I finished the term by writing an essay on how, at their core, Tolkien's story is one of hobbits. It starts with hobbits, it ends with hobbits, and much of the story's fate revolves around hobbits. Tolkien begins the book with hobbits in the Shire, peaceful, and then through the hobbits the reader enters into a world where everything is foreign and everything is a new experience. Without the Scouring of the Shire, however, the hobbits do not really get to complete their story as a people within the books. While the four individual hobbits no doubt showed extremely heroic properties within the context of their journey, the hobbits as a people were unable to truly show their ability to rise above.
It's a little tacky that Gandalf just kind of leaves them to fight it out themselves despite clearly knowing the evil that lies ahead, but that's really what the scene represents. It represents the hobbits saving their own people, rallying them together and finding their own place in Middle-earth.
However, the problem was that in the case of the films, I really don't think the scene was possible. I think it's less about length, and more about the logistical nightmare that is the deneoument. Jackson was creating cinema, not a novel, and I think that their structures are fundamentally different. I think that the Scouring would have made an extremely interesting scene, and would have closed out things nicely...but I do not believe that most movie viewers would look at it that way at all. It would rather be yet another ending, and one that seems to drag at that.
But it's in place in the books...as a hobbit story, it ends with the hobbits.