I definitely agree with you on the first part. Halo should have stayed on its original course. The thing we'll never know it if Halo would have stayed as popular as it was from 2007 till 2010 if it had kept the original formula.
The thing is that first and foremost they have to sell a game, which already has had many iterations, to the general public and to some extent a fleeting userbase. They need some kind of hook, mechanic, original idea to set as bulletin point to sell the game. Which is why games are tinkered with, remixed to death and in most cases loses some of its core originality in the process. It is really tough rope to balance, because stay too much with the old gameplay, it will be criticized as being outdated (Remember that one reviewer criticizing Halo 4 for not introducing ads?). On the other hand, tinker to much it loses some its core principles in the processin the end some praise it, others despise it.
What annoys me about that way of thinking is that often the solution is to only go with the new and improved gameplay, instead of giving the player base a wide variety of options. Yes, people often fall into old patterns, so they go back to player what they were used to without trying the new gameplayhowever pushing and forcing them into that direction can be detrimental in the long run. Instead of looking at compromises and streamlining, give the tools and options to the players. In case of Halo 4, we got a highly comprised game, that pushed in only one direction and the rest, was a whole pile of nothing to fall back on. I mean betting big on one thing is always risky, so do not be surprised when you lose it all and on top of that act surprised that it failed.
Now lets say that they gave us the vanilla Halo 4, but also packed in multiple options for a strip down classic mode and really worked on community features such as Forge, Theater, Spectator modes and morethe eventual blowback would have been less, because people have much more to fall back on, they are willing to give more feedback and could potentially grow and sustain a community. The other big plus is that people will still try the new gameplay and be a tad more forgiving, because it becomes a element in the overall packageinstead of the only element overall.
Pew, I kind of went on rant mode. Sorry about that.