The PSP started off pretty good; it had great support from Sony and 3rd parties alike, with tons of exclusives. I think Sony's biggest mistake was waiting too long to release a successor, especially once it became increasingly clear that the DS would be far ahead in sales. They should've tried to do what Microsoft did to them with the 360 (or what Sony themselves did to Nintendo with the PS1 and PS2), namely releasing their new system a year or more before the competition. Instead they did the opposite, and released the PSP successor a year after the 3ds. I still don't understand how they thought that would be a good idea.
Instead of trying to prolong the PSP's lifespan with the PSP Go in 2009, Sony should've released a proper successor around that time. It wouldn't have been as powerful as the Vita, but that wouldn't have been necessary anyway. A PSP successor roughly on par with the 3ds in terms of power, with dual sticks and touch screen, but released in late 2009, more than a year before Nintendo launched the 3ds, could have been a much more competitive product than what the Vita turned out to be, especially since its price could've been much closer to that of the 3ds. In fact, since a PSP successor would lack both a 3d screen and a second screen, Sony could've theoretically undercut Nintendo in terms of price. And without the need to render in 3d, games on Sony's handheld would've probably still looked better than 3ds games despite being on a system that launched earlier and possibly had a lower price.
An earlier launch of such a PSP successor would've also given Sony the opportunity to release many of their later PSP titles on this new system, and with better graphics (and controls): LittleBigPlanet PSP and Jak and Daxter: The Lost Frontier could've been launch games, while God of War: Ghost of Sparta could've been a great visual showcase in 2010. There are also tons of 3rd party games that came out late during the PSP's life cycle that would've greatly benefited from being on a more powerful system with dual stick controls (though most of them would probably still have PSP versions): MGS Peace Walker, Monster Hunter Portable 3, Final Fantasy Type 0 etc. Most of these games would've come out between late 2009 and 2010, giving Sony's handheld a strong software library before the 3ds had even launched.
I also think Sony should've come out with a high-quality Pokémon clone a long time ago, and I still don't understand why they never even tried this. Unlike their attempts to make a 'Halo-killer' (Killzone), or a 'GTA-killer' (The Getaway), making a 'Pokémon-killer' should've been much easier, and less costly, considering the relatively low production values of Pokémon games.