I really liked Parasite Eve 2 (Resident Evil: The RPG). Slowly playing through the first game on my Vita off and on and it's okay (almost done with the first playthrough). Combat's not all that interesting (dodge, shoot, heal, repeat) and though the equipment tinkering system is interesting, so far, there have rarely been any interesting traits to transfer over. Story and setting is cool, but it's plagued by an early "functional but stilted" Squaresoft translation.
Third Birthday was somewhat interesting as an attempt to do a blockbuster third-person shooter on a portable system but otherwise is a hot mess.
Other games worth checking out in the horror/RPG sub-genre would be Vagrant Story, Breath of Fire: Dragon Quarter, Shadow Hearts (the first one, the sequels got less and less horror-ey), and the Shin Megami Tensei series in general. Transistor is also worth checking out as it has a combat system that seems inspired by Parasite Eve & Vagrant Story.