The problem I have with the Xbox One's UI is that, whilst it's very, very impressive at what it does, I can't shake the feeling that it's all coming at the expense of how well the games themselves actually run.
I'm not a huge tech-guy, but I believe the UI eats up quite a lot of the system's memory and resources? Stuff like being able to suspend the game, watch a movie, have a Skype chat, then hop straight back where you left off is impressive in a technical sense, but isn't really something I'll ever actually want to do, so I can't help but think it's a pretty huge waste of resources.
It is what it is though, and I don't want to detract from the incredible work the team has done. It looks like it all works really well.