I agree that trying to do the palaces in one go would obviously tire someone out. But at the same time, it doesn't change the fact that the dungeons are quite lengthy. Like if I take a break and finish the palace the next day (in real life), I'm still going to be ultimately spending the same amount of time finishing that palace regardless if I did or did not do it in one day. I'd still end up complaining about most, if not all, of them are. Doesn't help things either that the combat at some point just had me feel like I'm going through the motions, which is understandable since I played for 90+ hours, but if the palaces weren't often so long and tedious, it would've helped somewhat.
The part that I enjoyed the most was the social links and progressing the story, so in that sense, the palaces were an impediment to that. I have to get it out of the way so I can have as much time as possible hanging out with people, as well as have the story progress. Generally I'll see the palace, think it looks really swell and like the little puzzles they have within them, but before long I can't wait to leave the damn place.
As for the anxiety, well, you only have so much time. I didn't use a guide or anything, but you can't blame someone for wanting to make sure they max out the social links they've become most interested in.