I don't see Persona as a case of loyalty but rather of demographic and partnerships. There is really no such thing as "loyalty" when it comes to publishing games because it makes no sense from both a creative and business perspective. You want to release games on the best platform for the game, and meet the demands of the audience wanting that product.
Modern Persona is a much more otaku-centric series than pretty much anything else that Atlus makes. It has anime FMVs, dating elements, lots of anime tropes, stylish designs, vocal music tracks, etc. The very definition of "pop". In Japan the audience for this sort of stuff prefer to consume them on Sony platforms because they are usually also tech geeks who like the latest cool gadgets with good technology. That is the demographic part.
As for the business and partnership aspect, ever since Persona 3 blew up beyond just the games, the franchise has been actively supported by Sony Music and Aniplex (a Sony Music subsidiary). The music in Persona 3 and 4 is promoted by Sony Music, and they sell the CDs along with organizing live concerts and so on. Meanwhile Aniplex produces all the anime content supporting Persona - starting with Trinity Soul and continuing all the way to the upcoming Persona 3 movies.
I don't think it's hard to see why Persona has remained on Sony platforms.