I tend to think that instead of viewing the console market as "shrinking" (as defined by total number of hardware units sold), I think the industry is going through massive consolidation this generation, but the active userbase isn't necessarily decreasing. And this goes for hardware as well as software.
What I mean by consolidation is that it was a lot more common last generation to own multiple consoles. As the core demographic begins to age, there's less time in general to devote to multiple devices. So we're seeing right now that many are opting just to get a PS4 as their main console, to the detriment of competitors. Whereas, in the past it may have been far more likely to own all three systems.
In one space, you have Nintendo which has basically relinquished control of the casual market to mobile, and similarly (but not nearly as much) the Xbox One has lost every single territory they once had. I think Halo 5's lack of success is in large part something that has to hugely concern Microsoft even though they'll remain relevant in a few large regions -- there's very little to differentiate the Xbox One from the PS4. There's no standout set of exclusives that gravitates people towards their platform anymore.
A big failing of theirs is resting on their laurels and key franchises, many of which have never been wildly popular and are now encountering a large decline. Microsoft should have used the console transition to innovate and develop new IPs, but they're doubling down on their core audience which is shrinking. Destiny is basically the new Halo, and they decided to let go of Bungie which was a big, big mistake since they wanted them to turn into a Halo factory rather than allowing them to innovate and expand.
I think Sony has the right approach with their first party studios -- in general, let them do what they want EVEN if it means that you sometimes fail. Sure, Sony Santa Monica had a huge project canned after a few years of work. Millions of dollars wasted. On the other token, Naughty Dog took a risk with The Last of Us and it was hugely popular. I think as a platform holder you have to be willing to take those risks in order to re-excite the core audience. Microsoft has simply failed to do that.
Multiplatform titles have taken over, and Sony has seemingly secured that market with strategic partnerships on nearly all major titles. Their first party initiatives are icing on the cake, and the consumer has followed towards the PS4 platform while Microsoft isn't really presenting a quality case for why this consumer should also buy an Xbox One. Sony managed to make a decent comeback in the US despite a horrible start with the PS3 due in large part to making a compelling case for the consumer to ALSO buy a PS3, and I feel Microsoft should adopt this strategy but there's no evidence that they currently are.
The same consolidation is happening in the software space. Slip up once and you have a title like AC:S that bombs at retail. Meanwhile, Fallout 4 ships 12 million in its first week. Absolutely ridiculous numbers. There's clearly TONS of active core users, but they are all vying for limited time across fewer titles that feature TONS of hours of playtime.
So it's really difficult to quantify a "shrinking market".