Genuinely, I'm not sure. I'm sure someone much more numbers-savvy knows. That's not a deflection of course, but my understanding was that even Nintendo's weaker-selling hardware has generally helped them make enough to be worthwhile, between hardware itself, royalties, and not having to pay royalties to another publisher. So I'm speculating here and I'd be glad to be corrected if I'm mistaken. I do think a portable you can play on your TV doesn't have to be any significantly more powerful than a Wii U is (but is that already shooting for the moon?) as I don't believe graphics are of utmost importance to most Nintendo fans.
Modern Tegra parts are very competent and cost effective. It's probably the best bet currently if someone wanted to do that and have something that at least looked pretty similar to a Wii U. If we're talking a couple more years down the line then we'd be a lot closer to boot.
As for the other part of the question, I would say it depends on how much they can sell versus what they're selling on Wii U. There's no third party royalties on Wii U as it stands (since there's basically no third party games), so they'd have to make up the $12 a unit in licensing fees in terms of unit increase at minimum if we assume the hardware is break even between R&D and sales.
Nintendo dropping home console hardware is as much a work of fantasy dreaming as it is for them to publish games for Playstation
I think we would need that new QOL arm to get cut off first before those guys wake up and see that their home console business is really this piss poor mess
they seem to be doubling down on WiiU support
That's why I started the conversation as I did, which was assuming that Nintendo viewed launching a home console as an immutable fact, even if it's not necessarily an optimal business decision.
However, I think there's room for going "Well it would make more sense to do A, but assuming they're not doing A, here's what I feel is their best option for B."