Rob Wyatt:
"One thing to remember when it comes to the SPUs is they were made for a completely different purpose to what they were used for. When I started working on the PS3, I was in Japan, it was probably 2002 and I was splitting time between the PS3 and working on Ratchet on PS2. For the PS3 the RSX wasn't in the picture for maybe another year or so.
The original PS3 design had a Sony designed GPU called the RS but it only did pixels, it was also kind of complicated as you had to schedule all the threads yourself. The SPUs were intended to feed the RS with transformed vertices, in a similar manner to how the PS2 worked, and if you look at how the SPU DMA works then processing vertices is an almost perfect use case. The intended design was you'd be able to do fantastically complex vertex processing, with programmable nodes for skeleton joints, because the SPUs were not just stream processors (like vertex processors still are). There was a device called the LDPCU and to this day I'm 100% sure how it worked, it had 1500 pages of documentation in Japanese and only Mark Cerny could read it. It was basically a gate keeper and synchronization system that would allow the SPUs to process and complete vertex batches out of order but still have the RS/GPU render them in order. We never really used it because we didn't know how, to got it to work from what Mark told us and by the nature of how simple our tests were - I'm pretty sure it would have a total nightmare..
So what happened was the RS was too big, in silicon terms, to make and it wasn't really possible to optimize down to a reasonable size without significantly gutting it, if they gutted it then it wouldn't have competed with the XBox. At this point Sony were stuck between a rock and a hard place, they looked putting multiple cells in the console and software rendering (I actually wrote a prototype software renderer, in 100% hand paired asm, that would run across multiple SPUs - ultimately it was a proof of concept of what not to do), they look at stacking a bunch of PS2 style GPUs together to make a pseudo programmable blend stack. Ken Kutargi did not want to give up and go to Nvidia or AMD/ATI but in the end he had no choice, its a good job he did because how terrible would the PS3 have been if the SPUs were used for graphics and games had just the single PowerPC core??
Once the RSX showed up and it could do vertex processing the SPUs had no job. This is when the ICE team started looking at using the SPUs for other tasks, it was a massive exercise in data design. If you started from scratch you could design a system for physics, audio, AI, particles - whatever and it would be very fast because you could factor in the constraints of the SPU memory. However, if you started with existing code or cross platform code, then it was next to impossible to get the SPUs to do anything useful. Initially this resulted in huge variance in quality between first party and third party games. This was also the time frame when fewer and fewer studios were willing to write an engine from scratch and things like Unreal engine were getting very popular, it was UE3 at the time, and it ran like crap on the PS3 but ran awesome on the Xbox and PC. Ultimately, the negative developer feedback cut through the arrogance that was present at the time within Sony (and Ken himself) and the PS4 was intentionally designed to be PC like (and was done by Mark)."