You can swap in for no penalty during battle so it doesn't matter too much. The only caveats are if you want all of your characters to act early in Turn 1 you probably want your faster characters in, and if you don't have the Open Audition Radiant Skill for a character (which you won't have access to yet)
Am I correct in assuming that a member who doesn't participate in a battle doesn't get experience?