I thought within 3 it was based off the rank of your highest player. Not the average of your party. IE if a 40 paired with two 20's they would match with an opposing team that had a high rank of 30 or above.
Rank splits > 10 apart were considered Mixed Skill parties. I don't seem to recall it being based off the highest player, probably better to say that it was a weighted average though. If I went in to RP with GAF or PA guys, the randoms would be fairly competitive. If I played with my brother and his friends who don't play that much Halo and are more casual in how they play, it was like a feeding frenzy for me.
However, add another one or two high skilled players and the randoms would be competitive.
It was a problem for me in Social BTB since people would be having fun all night then invite me, and I'd start pulling higher ranks. If we invited my buddy Ratsult, we'd start pulling 45-50s regularly (at that point I was at or near being a playlist General in BTB, so my invis-TS had to have been pretty solidified). If I went in by myself it was a straight up Brig/Generalfest.
They didn't mind losing, but they did mind getting obliterated, so I only played BTB when srsbsness clanmates were on.
I'm not sure this problem has a solution I can think of From what I've seen in various communities, it does tend to shape who plays with each other, even if they don't overtly realize why.