PSP is amazing but if you want to play P4G anyway and have access to the biggest library possible then might as well go for a Vita.
If you don't care for portability then yeah, a PS TV sounds like the better deal, and yes you can play all those PSP games that way too (there are restrictions to which PSP games you can play on Vita and PS TV, but I'm confident all Persona games are compatible). However you'd be missing on Devil Summoner, only way to go about that is either emulation or a PSP with CFW.
If you don't care for portability then yeah, a PS TV sounds like the better deal, and yes you can play all those PSP games that way too (there are restrictions to which PSP games you can play on Vita and PS TV, but I'm confident all Persona games are compatible). However you'd be missing on Devil Summoner, only way to go about that is either emulation or a PSP with CFW.