For your first point, I addressed it briefly in the second half of my previous post. Business is not static and trends change. What worked in the past does not work now, what works for Sony will not work for Microsoft or Nintendo, there are too many factors that goes into growing a brand. Increased competition and dominance from major third parties, install base, the XONE online debacle, the MCC debacle clearly damaging the Halo brand; these played a part in the decreased sales of Halo 5 and Gears 4.
Regarding Diversity Microsoft can't blow money on a bunch of exclusive games no one will buy (which is what happened this generation for MS outside of their core franchises). Project Spark underperformed, D4 underperformed, Crimson Dragon underperformed, Happy Dungeons underperformed, Quantum Break unperformed, Sunset Overdrive underperformed, Scalebound would've likely underperformed and so forth. A vast majority of those titles are single player focused games, what good business sense would it make to keep investing in things that are costing you money? Especially as the cost of game development rises.
Now let's look at Valve, look at how successful their properties are, what's the common trait among them? Multiplayer, long term support, eSports, recurring revenue, etc. all built around a few key franchises. You can't build these initiatives around one-and-done single player products. Even further, let's look at Halo 5 and Gears 4, even if they don't sell the front end numbers their predecessors did, the post game support and microtransactions ensures more profit over the long term from a smaller but more dedicated base that knows your game(s) is a known quantity and will buy the next installment. I am not sure what you mean by MS games forcing the multiplayer service model, Halo and Gears are proven multiplayer commodities not new IPs which is why they are leveraging those brands for their new potential new direction.
Can diversity help their lineup? Yes, it absolutely can and did for Microsoft in the past. If in the future they announce a bunch of new games I certainly won't complain, however with changing market realities Xbox doesn't have the luxury of an unlimited budget to throw money at whatever they want, they have to maximize their ROI with what they do have.