I can think of several systems, partly based on experience and partly based on K/D. So players who have 1) cleared a lot of missions while 2) not dying often and 3) also killing a healthy number of enemies (to discourage guys from hanging back and getting a good K/D by just not participating) could be paired with players with a similar record. This may not be optimal, however.
In Firefight, with very few exceptions, I finish games with both the highest score and the lowest number of deaths. When I play Limited, it's often very lopsided. I think I'm probably the Firefight equivalent of a 45-50 player, so I realize it is probably hard to find many of the same skill set (not trying to boast, just reality; I play a LOT). But more often than not it's me and a member of the Bozo and the Banana Brothers clan. I think there are ways to improve this, both by factoring in something into matchmaking, and by encouraging a different style of play in the game settings.
Given the importance of a good connection in co-op modes due to the asynchronous networking model, I don't think adding a skill based filter would be ideal, unless it was secondary to connection speed (sort of like the Social Settings in Reach). I think game settings will be the best way to achieve this, by encouraging team play, or at least more thoughtful individual play. Again, stuff like non-pooled lives and pooled points (during the game). Perhaps have a player preference for difficulty level, as another possibility.
I can honestly say that I've never actually welcomed other players in Reach Firefight, other than playing with friends. In every single game, ever, I've wanted the other players to quit. But I love co-op games!