Mine is perfect when it comes to the core hardware: Performance, storage space and Ram have not been a problem yet. However, outside of the core hardware, I had a couple issues:
1) This battery sucks. And i'm not talking about duration: The actual battery is poorly constructed. The original one got swollen and, luckily, didn't crack anything, but it unglued the entire glass+screen+digitizer combo. I was in the bus one time, looking outside of the widow and I just felt it pop in my hand. I replaced the battery for an original one at a service shop and 1 year later I'm having the same issue (as a comparison, my original Galaxy S battery from 2010 only got bad enough that needed replacement 2 months ago). The screen (which was never glued back on, to avoid cracking if it ever happened again, which was a common issue, as described by the technician) started rising up and not being "flush" with the external frame anymore.
2) The digitizer (maybe because of the faulty batteries, maybe because of the rise on the screen or maybe because of the third fall I had in four years) finally presented the same "phantom touches" followed by a bottom strip the size of the soft navigational buttons not being responsive. I temporarily solved that issue by installing an XPosed module that allows me to increase the size of the navigational soft buttons strip to 115%, which allows the buttons to go to ta section of the screen that still works.
Other than those two issues (the battery one is a real fucker. The digitizer one was probably my fault for tangling the charger cable in my feet and pulling it from the night stand) and the Camera Flash thing, I didn't have any problems with it - i'm even using Pure Nexus mod to get 6.01 and it runs wonderfully.
I bought a new Battery and a new set of Frame+Glass+Digitizer+Screen to replace the faulty ones on my device. If they work and are good quality (which I can't be sure, because they're not official replacement parts), I can see myself using this Nexus 4 for another year or two. Unless you want to jump on the VR wagon, there's no need to replace it... unless it finally breaks down.
In either case, I'm also buying a used Moto X 2nd gen (2014) from a friend for real cheap, in the case that the replacement parts are not good.