Interesting read OP, thought I'd chime in. I've moved a few things around to make it easier to reply.
...This is, in fact, completely antithetical to the industry at large. Everything is improving yearly. New computers, new Blu-Ray players, new cars, new TVs, new blenders, and yes, new smartphones and tablets. But not game consoles...
While certainly true, the question that should be asked isn't "why?", because the answer is obvious in a smart-phone-iteration world. The question we need to ask is: "why now?" The fact is that the rate of improvement for the technologies that the video game industry uses is largely slowing down, as you note below. During the PS2 generation, a mid-gen revision would have made sense in terms of staying relevant, because the sheer degree of technological improvement available within that time was enormous. Instead, we got the PS2 Slim, to prevent user-base fracture, while dropping manufacturing costs. Contrasted with today, the improvements available are so small, the only thing we're really able say that the new hardware will provide are slightly better resolutions and slightly better frame-rates. I don't believe Sony are trying to keep their hardware relevant, or adapt to a change in the market. They're simply scrambling to find reasons to get PS4 owners to buy more hardware while still netting new consumers, and a small hardware upgrade is the best they could find. Sony's financial situation is well known, and Playstation is currently its crown jewel. I believe Sony is doubling down on Playstation, because it is currently it's best proven chance of profitability. The PS4K makes perfect sense from Sony's point of view, but little sense from anyone else's.
Generational Loyalty
...In a cycle that encourages players to slash ties with where they were coming from to a new platform that has everything they're looking for, how do you make sure that customers are tied down to a brand, or at least, have less of a reason to abandon it in favor of the other guys?
The way to do that is to blur the line of product cycles...
... So what does that leave us with? A brand new console, coming way sooner than anyone has expected, that changes a generational cycle into a constantly refreshed cycle. No longer are there gaps in these product generations that allow for consumers to jump to other platforms, at least not one that's easily discernible. You probably didn't know it at the time, but that copy of Knack you bought in 2013 will be playable in 2023, on new hardware...
I understand your point, however I largely disagree with it. "Generational loyalty" is a by-product of limited means. People buy one console, because most households don't have enough disposable income to purchase multiple boxes. The PS4k is not responding to this market. The best example of my point is the PS2 Slim, which was a perfect response to the market. The "slim" edition was cheaper to manufacture, and thus Sony could price it quite cheaply, enabling more people to buy in. House holds that couldn't afford two consoles now could, and Sony sold a significant number of them, driving software sales. In contrast, a hardware upgrade keeps the price high without the drop in manufacturing costs. Dropping the current PS4 in price to fill the 'slim' gap while releasing objectively superior hardware damages good will for short term gains. This is because, if console manufacturers are hoping for multiple hardware purchases more often, consumers will quickly find the path of least resistance. It's the nature of the market. And that's pretty simple in this case: its better value for me to buy an NX over a PS4K if looking for new hardware, because Sony have guaranteed that my PS4 can do everything a PS4K can do. Because a PS4k will not provide improvements to the games I own and play today, whereas an NX provides a new library and potentially superior multi-plat versions, it's the smarter purchase. In my opinion, Sony are actually
banking on brand loyalty to drive their mid-generation revisions amongst their fans, while using the strong word of mouth from the PS4, and promise of objectively better hardware to entice new people to buy-in.
... And that's the whole point of the PS4K: in an age where tech advances are slowing to a point where a technological leap is impossible without a high cost or a long wait, companies have to change up this cycle...
Will consumers actually wait until 2022 to get a generational leap in graphics at a consumer friendly price? What if they didn't have to wait?...
You're point here is largely confused, I feel. Consumers aren't after "new hardware" when buying a new console. They're after new games, and a clear, demonstrable generational leap in them when buying in. The hardware is the gateway to that. We've seen clear, demonstrable impacts of this, with the PS3 being the most recent, and the 3DS being a solid second example. The PS4k provides literally none of this. It provides the same games, only with improvements that are going to be largely difficult to spot short of extensive YouTube breakdowns, and largely misunderstood by mainstream consumers. The PS4k isn't filling a need, and it isn't responding to the market.
...Think about how unfriendly the current generational lifecycle is for the average consumer. They can either come in too early at high adopter prices and a slow start of game releases, or right in the middle where sales peak and deals are good, or come it at the end, where support will soon be ending for the system and mere years or even months are left for game releases. If they don't hit that sweet spot, they can be left holding the bag on a system that has been essentially abandoned in favor of the all new console...
Again, I think you're point is largely confused. You're painting a picture that of a situation that doesn't exist, where virtually no one is happy and the poor consumers need a change. The demonstration that is false is that literally hundreds of millions of people buy in over the course of a generation, and have been doing so for nearly eight full generations now. Clearly, something works, and consumers like it because they keep coming back. This is because buying a console is the single most cost effective way to play video games, bar none. A single hardware purchase can set you up with prime support for AAA releases for six years, with a library of games numbering in the thousands. It's simply a fantastic value proposition. Consumers simply choose when the proposition is the best value for them. Buying in at the start of the PS4 was a good decision for me, because I've spent literally thousands of hours with my PS4. Buying a PS2 at the end of its generation was a good decision for me, because I got cheap access to entire generation of incredible games at bargain bin prices. The PS4K works to counter this, with my hardware purchase costing the same, but only receiving prime support for three years. After that, I may be supported, but if I want the best support, I need to buy-in again at the same high price, which only buys me another three years of prime support. The value proposition halves. As I mentioned above, this is a great deal for Sony, but a bad deal for everyone else.
...We're starting to pivot into the idea of games as a service... It is dependant on one thing, really: a healthy userbase to get users from, which is easily disrupted by a generational gap... If the PS4K does not exist, and the PS5 creates this chasm as all generation leaps do, then what can you do? Develop for the PS4, and have tools in place to bridge the gap to the PS5 when it comes out? This is essentially what Destiny is doing, but it comes at a major cost, something that Activision and probably other major publishers like EA and Ubisoft can support, but few others can... With this new cycle, you don't have to worry about that anymore. The PS4 is also the PS4K, and is probably the PS5 too. When the PS4K launches, it will already have a userbase of over 40 million players. It already has all of the developmental tools that have matured and strengthened over the last few years. There's no risk to building the next big online game late in the PS4 future, because the transition over to the new system is built into the ecosystem. This is a huge, huge benefit to game development...
I'm going to have to question your experience with game development here, because the PS4k creates more problems than it solves, that have far longer impacts than a simple PS5 generational leap would. The examples you used to highlight your point actually demonstrate this issue, but you appear to have missed them. You're arguing that only the biggest publishers can afford to do handle multi-generational development, while missing that only the biggest publishers can offer games like Destiny and The Division.
In order to demonstrate the issue that the PS4k creates, let's use Destiny. This was a multi-platform, multi-generational title. The game worked, with scaled back visuals, on the PS3 and Xbox 360. This held back the next generation versions from using the new hardware to its fullest, required four full QA sweeps, and required additional development houses to actually craft those versions. But they sold additional copies on the 360 and PS3, which offset the expenditure. And herein lies the problem: the PS4k is expensive new hardware that can't be used but must be accounted for without creating new revenue for the developer. Assuming a similar situation with the PS5, developers and publishers QA costs double instantly, and they can't even use the new hardware properly, because the older hardware is a platform-holder-mandated anchor around its neck that must be taken into account during the software feasibility, frame budgeting, and software engineering phases. Sure, you might save Bungie a headache or two, but you create a permanent, unmoving, costly headache for every other developer in the industry, without creating a pay off. Once again, the PS4k is a wonderful move for Sony, and a terrible move for everyone else.
Wave of Anger
Certainly there will be anger to come with this news; it's already happening, but then again, it always happens with everything. The internet amplifies everything to a degree that it becomes somewhat impossible to measure the actual consumer response to things until they're released...
I don't believe this is true at all. Multi-billion dollar corporations do not launch products with the R&D costs of a console and just "hope". They're calculated moves. The internet only confuses a consumer response if you don't actually move outside of closed circles such as NeoGAF. The response for the PS4k is pretty clear. If you want to know if a consumer response will be positive or negative, you just need to understand if the product represents a positive or negative proposition for the consumer. The PS4k straddles the line: its a negative impact on current PS4 owners, but a terrific proposition for potential new owners. Sony are calculating that the amount of new business they can generate will outpace the disgruntled customers they'll incur. Microsoft made similar calculations with their original Xbone plans. Hopefully Sony get the math right.
... But the time for disruptive product cycles is over. There's too much risk involved, with game budgets more expensive than ever, and other markets increasingly eating away at each other. There was a lot of debate before this generation launched on whether or not there was even a market for game consoles anymore as the PS3 and 360 quickly plummeted in sales. The PS4 answered that question with a resounding yes. I wonder what answers the PS4K will bring.
You might not have heard of VR, which promises fast iterations and hugely disruptive technologies like foveated rendering. You're point is demonstrably false.
The arguments you've put forth seem to boil down to: current situation is bad for consumers, new situation is better for consumers. However, as I've hopefully demonstrated, the exact opposite is true. This is why there is a negative push against iterative hardware models in the console space: you're asking consumers to pay double the money to get what they've always had, and providing literally zero incentives to consumers to sweeten the deal. The best you've been able to offer is "its better for developers", which I've shown to be false. The PS4k is a great deal for people who don't own a PS4, and a great business opportunity for Sony. For everyone else, be they current PS4 owners, developers, or publishers, the PS4k creates a headache that isn't responding to market forces or consumer demand. It's only reason for being is for Sony to sell more hardware. And it will certainly sell, because its an objectively superior piece of hardware. But the problems it introduces could up burning bridges with Sony's most loyal, dedicated fans, that showed up in droves to support Sony when it was "for the players". Microsoft imploded the Xbox brand with a similar gamble. Time will tell, but currently, I'm not seeing the case for the PS4k.