Got my Moto E today.
Very pleasantly surprised by the hardware, it's very well built and even though it's a bit generic, I really quite like the design.
It's snappy throughout the interface, though apps lag and judder like fuck (see
https://vine.co/v/OrH3PW6j7QA, Facebook is even worse but I won't vine that for obvious reasons), not something you'd see on low end WP hardware. Nice screen, horrendous camera. You don't expect much from the camera for the price, but you get a less than that, imo (pretty sure that the 620's camera is better than that).
Checked out some of the apps, and while they are generally more feature rich, the judder destroys any advantage, I think.
The keyboard is god awful. I tried SwiftKey and I'm not really impressed by that either, particularly its design is very, very ugly.
One thing that's fantastic in Android is the LastPass integration. That's something Microsoft need to look at and enable going forward. There needs to be more options for integration.
Overall, I don't think you can moan considering the price and I think on high end hardware Android can really shine, when all the judder and lag is (hopefully) removed. I had a few app crashes as well (the store crashed several times), don't know if that's hardware related. From a UI point of view, I still think it's fairly messy, but when you're used to it, you can live with it. No L update for the E yet though, maybe it's a bit better there.
All that said, I think that for the low end, WP is a superior choice. Yes, your app choice is more limited, but at least the apps are useable.
Will play around with it some more and bore you with more impressions in due course.