I think you're right. There's a certain convenience factor that is lost; it's how I initiate our raids every week (you won't be able to "Join Session" on me if set to Invite Only) but it's a fair reading of the systems to interpret a Friends Only status as "accepting random joins." I will stop being jesus lazy and set my status on login from now on.There is no excuse for not setting it to invite only if random drop-ins are a big deal to you. None. It's lame and needs to be addressed that you can't save your preference right now but jesus do you have to be extremely lazy to not make this very quick change before setting out on whatever activity you're going to begin. There are multiple privacy settings for a reason people, use them! It only makes sense that if you have it set to "Friends Only" you're saying it's ok for anyone on your FL to jump in. The permissions set the edict for us and you don't have to deal with awkward situations.
edit: *sigh* But I also don't want to send the signal that I'm not happy to group with people. Invite Only will probably make it look like I'm waiting on a specific group or don't want to be bothered whereas I am almost definitely down to do stuff. Hopefully people will still shoot me a message rather than assuming I'm not available.
The PSN smartphone app can help if you have a lot of trouble with messaging.