PS3 encoded video by taking some CPU power back from games. It was not fast, and developers did not want to give that power to feature that few would use. Also, PS3 had 802.11g WiFi module, which adds some latency.
On the other hand, both Vita and PS4 were built from the ground up for remote play. PS4 has dedicated hardware for fast video encoding, vita has dedicated hardware for fast video decoding, and both have fast 802.11n WiFi modules. For best performance, PS4 and Vita need to talk directly, without use of home router.
And it's using Gaikai's streaming protocol which I imagine is low latency/overhead etc.