I don't know why you guys have some much trouble with the companions.
Personaly, I change frequenly my party members outside of the story while doing quests.
They're all at the moment lvl 20 with at least 1 or 2 affinity hearts except for Elma and Lin, ofc.
I only do the Blade gathering quests when I've the items on me, then go hunt the ones on monsters (thanks Caww database) or buy them with tickets.
There're some workaround the flaws of the game, so it's okay with me.
I can progress smoothly without scratching my head.
Anyway, I juust finished chapter 5 (after 45 hours, lol), I liked it, the twist was nice.
I would recommand to do it at lvl 20+ at least because it can ecome pretty mobish sometimes.
So new chapter fnished mean new outfits for my characters, I like to change regulary my look <3
My main team (in the pic) is level 24+.
Personally, I don't found that game need to grind at all.
Most of my time I spend exploring, while killing monsters and doing npcs quests and I'm already over level up for the story, lol.