I've tried it with both direct WiFi and over my router and it feels kinda laggy either way. It's a huge improvement from the PS3's but compared to the WiiU it's noticeably worse. AC4 felt great on it though and looked stunning, but faster paced games like Resogun were definitely laggy and I couldn't even imagine playing something like Virtua Fighter or Bayonetta/DMC on it.
I'm fairly latency sensitive though, I run most of my games at 100+ FPS without VSync on PC to minimize latency... I was really impressed with how fast the WiiU was though and was hoping this would be the same. It's definitely not.
The WiiU runs on 5GHz, so most likely what you're experiencing with the vita is the congestion on the 2.4GHz band. Try changing channels on your router, but If you live in an area with lots of other WiFi ap then you might be out of luck.