So, I design TV apps for a living. Currently on my desk at work, I have a PS4, an XBO, Nvidia Shield, Google Nexus, ATV4, OG ATV, FireTV, Fire and Roku sticks, and a couple others I'm sure I've forgotten about.
OP is right. The Apple TV 4 remote is terrible. Or, to be more accurate, the horizontal on-screen keyboard is terrible. In our testing, the keyboard grid, as seen on PS4, XBO, FireTV, and others, makes entering type significantly faster and more efficient for users.
The ATV4 remote is great for large swipes, but terrible for precise movements. The touchpad is way too sensitive-if I even look at the touchpad funny on my home setup, it turns on, and switches away from whatever I was watching. It got so bad that I have a Harmony remote at home, and use the combination of the old Apple remote and bluetooth keyboard at work, because I have to enter passwords a bunch. And the OP is right-telling him to download additional apps or add hardware to make the device work well out of the box isn't really the best solution.
But the Apple TV hardware and software itself is aces. And the quality of the media streaming apps on the device have dramatically improved-a lot of things we're starting to implement on our apps have already started to hit the market (which really fucking sucks, for me, anyway). I actually also really like the Shield device, and Android TV, but the apps right now are sub-par compared to the best of ATV apps.
In my experience, purely considering media streaming capabilities: ATV 4 > Nvidia Shield > XBO > Fire TV > Roku > PS4 > OG ATV > Roku/Fire Sticks