Support NeoGAF

[Draugoth]

Activision has historically stayed on the sidelines during these debates, but that changed earlier this year when the publisher finally lifted the veil on how matchmaking works in every modern Call of Duty game, and explaining its reasoning for choosing SBMM. Building on that, today the company published the first of a series of white papers diving deep (and I mean terminology tables and multi-format graphs deep) on matchmaking.

The new paper, a 25-page document titled "The Role of Skill In Matchmaking," is as impressive as it is jarring. On one hand, it's a little funny that Activision is treating the topic with tact and seriousness, posing such questions as "What is Skill?" and mapping "kills per minute" and "skill buckets" on graphs, when we all know that naysayers will probably just say "SBMM still trash lmao" and move on. But it's also a wild document for a videogame: I don't think we've ever seen a major studio go this in-depth on the intricate criteria and details that its algorithm uses to judge us.

Some of this ground has been covered before. We finally learned for sure back in April that Call of Duty's matchmaker, above all else, values finding matches quickly over ensuring it's perfectly balanced. What players might find most interesting is the discovery that Activision ran a secret test to see what CoD would be like with less SBMM (but not none). Those sneaky devils.

The experiment ran in Modern Warfare 3 in early 2024. Discreetly, Activision toned down the "skill" in skill-based matchmaking for 50% of the North American MW3 population.

The results? Over 90% of players with toned down SBMM played less Call of Duty after the change, with the top 10% of skilled players not as affected. The paper goes on to confirm what has, to me, always been the most obvious reason to have SBMM: random matchmaking primarily benefits really good players, and disproportionately sucks for average-skilled players.

via PCGamer

 

[th4tguy]

Reminds me of how with every early patch of counter strike, the community would complain about lag getting worse even though nothing in the code had been touched related to network.
Finally the devs put a fake update out saying that they fixed lag even though they again, hadn’t done anything and everyone celebrated saying it was fixed and so much better. People are collectively dumb.

 

[MayauMiao]

First, what the hell is SBMM?

 

[Draugoth]

First, what the hell is SBMM?

Skill Based Match Making

 

[feynoob]

First, what the hell is SBMM?

Skill-based matchmaking (SBMM), also referred to as matchmaking ranking, is a form of matchmaking dependent on the relative skill level of the players involved.

 

[March Climber]

First, what the hell is SBMM?

Smash Bros Melee Modded

 

[ChorizoPicozo]

weren't the gamers complaining about it?

 

[Diddy X]

As a casual COD player I'm ok with it and I understand tryhards will be upset as they want to be paired against weaker enemies which is kinda lame.

 

[Men_in_Boxes]

Common sense. The only people who wanted SBMM turned off are weird high skill players who like dunking on little children.

 

[CLW]

What people prefer roughly level competition?

 

[Black_Stride]

I was literally reading the first two papers earlier today.
Followed a couple of developers on Black-Ops and even they said gamers dont know what they are angry at, and swear the older games didnt have SBMM, they did.
Said gamers either got better, or the pool of good players is larger now so they are getting matched with players at their skill level.

High level gamers hate SBMM because they get matched against people of similar skill.....so they call the system sweaty.
Average gamers get a good mix of tough and okay matches.
Low skill players just have a good time.

On Average the system they use is a net benefit.

weren't the gamers complaining about it?

High skill player hate it because obviously they are likely to be matched against other high skill players.
They are also the ones most likely to make a big stink about it.
Average to low skill players have a great time and dont complain or praise the system, so the online discourse ends up looking like gamers dislike the system.

 

[kikkis]

Cod influencers have already gotten their tin foil hats out

 

[Miles708]

I am seriously out of the loop so I'll just ask: why would anyone want skill based match making disabled?

 

[kikkis]

I am seriously out of the loop so I'll just ask: why would anyone want skill based match making disabled?

It reduces the amount of feeling of progression as you get more skilled. And big swings in difficult when you are a tweener.

 

[realcool]

I guess we'll have to take Activision's word for it.

 

[ReyBrujo]

What people prefer roughly level competition?

Nobody but on the contrary, bullies spend quite a lot of time and money building skills to bully and SBMM puts them against people that might be even better than them. They want to bully, they want easier targets.

 

[analog_future]

Angry gamers don’t actually know what they want!?

 

[HeisenbergFX4]

Common sense. The only people who wanted SBMM turned off are weird high skill players who like dunking on little children.

Which still happens as its super easy to "reverse boost" for a few games and those high skill players get dropped into much easier lobbies then

I think its a fairly easy fix there are so many different queues already just make one with zero SBMM and leave the rest and actually let the players decide which one they want to play it

 

[Cornbread78]

Common sense. The only people who wanted SBMM turned off are weird high skill players who like dunking on little children.

Hey, they dunk on us old guys too.... big jerks just want the social media clips pawning noobs for epein

 

[Cornbread78]

I think its a fairly easy fix there are so many different queues already just make one with zero SBMM and leave the rest and actually let the players decide which one they want to play it

Destiny has this and all the avg & lower skill players run away from those modes, then the top players bitch that the mode is dead...

 

[Spukc]

First, what the hell is SBMM?

Skill based match making you fucking noob
It fucks the streamers so i love it

 

[simpatico]

mid FPS players just like pub stomping kids who got the game a couple days ago. Make SBMM match making permanent, so these guys actually improve.

 

[Icymanipulator]

Reminds me of how with every early patch of counter strike, the community would complain about lag getting worse even though nothing in the code had been touched related to network.
Finally the devs put a fake update out saying that they fixed lag even though they again, hadn’t done anything and everyone celebrated saying it was fixed and so much better. People are collectively dumb.

Wasn’t this the case of them just displaying a lower ping regardless of what it actually was, and then ya, players ate it up wholeheartedly lol thought it was amazing.

 

[clarky]

weren't the gamers complaining about it?

Mostly you'll find the COD community moaning that it swings too wildly in one direction or the other. For example I can have 3 or 4 good games, with the playing field fairly level and having good games then it'll chuck me in games that are way above my skill level and i get dunked on. Then after 3-4 games like that i'll get an easy lobby. Followed by a few decent ones. Rinse repeat. Its like clock work and i'm not having this is working as intended.

I take it they don't address this in the article, its the main complaint from players. No one, well not me at least, is against fair SBMM.

 

[Dirk Benedict]

Smash Bros Melee Modded

I caught a quick whiff of the basement as I read this.

 

[clarky]

I Dont see why they don't use the ranking system but hidden. For example if i'm ranked Platinum II, only match me with players who are plat I, II & III. Maybe the odd Gold III or Diamond I . Sure that can be abused with reversed boosting but that happens already with the current system.

 

[ManaByte]

People are collectively dumb.

 

[Rentahamster]

It seems obvious that matching players of relatively equal skill makes for a more compelling gameplay experience. If it's too easy, it gets boring. If it's too hard, it gets demotivating.

The only people opposed to proper skill matching are scrubs who want to punch down and overinflate their numbers to make it look like they're better than they actually are.

With proper skill matching in ranked game modes, rank actually means something and I can more reliably tell who is actually a good player.

 

[HeisenbergFX4]

It seems obvious that matching players of relatively equal skill makes for a more compelling gameplay experience. If it's too easy, it gets boring. If it's too hard, it gets demotivating.

The only people opposed to proper skill matching are scrubs who want to punch down and overinflate their numbers to make it look like they're better than they actually are.

With proper skill matching in ranked game modes, rank actually means something and I can more reliably tell who is actually a good player.

Which is great in theory but this SBMM swings too wildly

You have a couple of decent games you know you are getting thrown into super sweat lobbies to intentionally get knocked down again

Rinse and repeat and you have CODs SBMM

I think Rocket Leagues SBMM system works much better gradually moving you in better or worse lobbies

 

[Men_in_Boxes]

Which still happens as its super easy to "reverse boost" for a few games and those high skill players get dropped into much easier lobbies then

Saying "it still happens" ignores the fact that it happens at significantly reduced rates. It's like saying we shouldn't have seat belts because dying in car wrecks "still happens". Making it harder for the high skill players to get into matches with low skill players reduces toxic friction considerably.

I think its a fairly easy fix there are so many different queues already just make one with zero SBMM and leave the rest and actually let the players decide which one they want to play it.

They don't do this because no one plays the mode with zero SBMM. The rabbits don't like going 2 - 18 at the end of every match so they leave. Then the weaker wolves become the new rabbits and they soon leave. The cycle continues until no one is in that queue.

 

[Men_in_Boxes]

Hey, they dunk on us old guys too.... big jerks just want the social media clips pawning noobs for epein

I'm with you.

The fact is that going 22 - 2 isn't significantly more fun than going 12 - 7. Or rather, the increase in fun isn't worth losing the low skill players over. It's a balance.

 

[S0ULZB0URNE]

As if a super low tick rate wasn't reason enough to skip.

 

[TehGoomba7s]

The only people who have an issue with SBMM are people who think they are way better than they actually are because they play against people who've picked up a controller first time. It's a proven way to improve at a game, it retains players significantly and keeps it a less frustrating experience overall.

I don't know anyone still playing XDefiant, because of how random it is without it. I dropped it quickly, not only because of how boring it is playing against mostly bots, but when you do meet the opposite end of it, you'll get absolutely steamrolled. No in-between.

 

[HeisenbergFX4]

Saying "it still happens" ignores the fact that it happens at significantly reduced rates. It's like saying we shouldn't have seat belts because dying in car wrecks "still happens". Making it harder for the high skill players to get into matches with low skill players reduces toxic friction considerably.

They don't do this because no one plays the mode with zero SBMM. The rabbits don't like going 2 - 18 at the end of every match so they leave. Then the weaker wolves become the new rabbits and they soon leave. The cycle continues until no one is in that queue.

At least give people that option imo, don't blanket force it either way

I have said this before but something that makes me scratch my head on this SBMM is on the rare occasion I do get on a roll and hitting my shots and killing people this SBMM puts me in lobbies where I am a step behind, I get killed by people who see me before I see them when turning corners

I will use a term someone used who once upon a time was really high up the food chain at Blizzard and knows a ton of people at Acti he didn't call it SBMM he would call it Experience Based Match Making as it would dictate your experience you had with the game

Put me in better lobbies thats fine don't make me feel like youre tweaking my experience like giving people an edge

 

[clarky]

The only people who have an issue with SBMM are people who think they are way better than they actually are because they play against people who've picked up a controller first time. It's a proven way to improve at a game, it retains players significantly and keeps it a less frustrating experience overall.

I don't know anyone still playing XDefiant, because of how random it is without it. I dropped it quickly, not only because of how boring it is playing against mostly bots, but when you do meet the opposite end of it, you'll get absolutely steamrolled. No in-between.

Simply not true. Its not black and white.

 

[ChuckeRearmed]

It is understandable really - high skilled players will destroy the low skilled players all the time (making playing uninteresting) or will carry the whole team alone (making it boring too).
The only reason for highly skilled players to play together with low skilled is because they want to destroy noobs.

In the past (years years ago) it was fine because you literally did not have much to play. Now you can just switch to another game.

 

[clarky]

It is understandable really - high skilled players will destroy the low skilled players all the time (making playing uninteresting) or will carry the whole team alone (making it boring too).
The only reason for highly skilled players to play together with low skilled is because they want to destroy noobs.

In the past (years years ago) it was fine because you literally did not have much to play. Now you can just switch to another game.

Was there a drought of games i didn't hear about years and years ago?

 

[a multicultural team of v]

I'm not sure if this is the proper study, it needs to be SBMM vs old school lobbies.

 

[Dorfdad]

Activision has historically stayed on the sidelines during these debates, but that changed earlier this year when the publisher finally lifted the veil on how matchmaking works in every modern Call of Duty game, and explaining its reasoning for choosing SBMM. Building on that, today the company published the first of a series of white papers diving deep (and I mean terminology tables and multi-format graphs deep) on matchmaking.

The new paper, a 25-page document titled "The Role of Skill In Matchmaking," is as impressive as it is jarring. On one hand, it's a little funny that Activision is treating the topic with tact and seriousness, posing such questions as "What is Skill?" and mapping "kills per minute" and "skill buckets" on graphs, when we all know that naysayers will probably just say "SBMM still trash lmao" and move on. But it's also a wild document for a videogame: I don't think we've ever seen a major studio go this in-depth on the intricate criteria and details that its algorithm uses to judge us.

Some of this ground has been covered before. We finally learned for sure back in April that Call of Duty's matchmaker, above all else, values finding matches quickly over ensuring it's perfectly balanced. What players might find most interesting is the discovery that Activision ran a secret test to see what CoD would be like with less SBMM (but not none). Those sneaky devils.

The experiment ran in Modern Warfare 3 in early 2024. Discreetly, Activision toned down the "skill" in skill-based matchmaking for 50% of the North American MW3 population.

The results? Over 90% of players with toned down SBMM played less Call of Duty after the change, with the top 10% of skilled players not as affected. The paper goes on to confirm what has, to me, always been the most obvious reason to have SBMM: random matchmaking primarily benefits really good players, and disproportionately sucks for average-skilled players.

via PCGamer

the reason that people play less than before isn't based on SBMM its because the game gets old after 3-4 months of constantly rinse and repeating and all games drop off. SBMM when handled correctly would be a blessing, but the versions / algorithm they use is flawed IMHO.

It's heavily skewed when you have parties or not even utilized more than likely based on some matches.

If they truly wanted SBMM they would force the top tier players their base to play against each other and since they are the outliners they would run into problems finding matches, and they would compiling the loudest.

Personally I don't care for it and would appreciate them either asking me if I want to join a game that's already over 100-15 score wise or at least when you trow me into a game were destined to lose give me double XP for that match so I have some reason to try.

Just make the SBMM adjust based on your collective KD ratio average alone would be better. <1.00 >1-2 >2-3. > Hackers

 

[Men_in_Boxes]

At least give people that option imo, don't blanket force it either way

Here's why I don't think they want to do that. Developers operate under the following pretense...

"Every time a player opens up our game and plays, we want that person having fun. If a player is miserable for a length of time...be it 10 minutes or 50 minutes, we risk losing that player and the potential cash they bring."

They probably believe there would be a number of rabbits who open up that NO SBMM lobby, play it for a length of time and be miserable doing so. They're trying to avoid that as much as possible.

I have said this before but something that makes me scratch my head on this SBMM is on the rare occasion I do get on a roll and hitting my shots and killing people this SBMM puts me in lobbies where I am a step behind, I get killed by people who see me before I see them when turning corners

I don't like it either. I've played a lot of The Finals and Valorant recently and they seem to do it as well. I've thought about your past comments on this while playing those games and I've reached a slightly different conclusion.

I will use a term someone used who once upon a time was really high up the food chain at Blizzard and knows a ton of people at Acti he didn't call it SBMM he would call it Experience Based Match Making as it would dictate your experience you had with the game

It's definitely a little creepy how they've figured out how much fun they're producing in our brains.

Put me in better lobbies thats fine don't make me feel like youre tweaking my experience like giving people an edge

I think the best version of this system looks like this...

In one match, I feel like I'm our teams Lebron James. Best player on the team, but it's still challenging to a degree. In the next match they make me feel like Mike Conley. Not as important but if Mike Conley goes 5/8 from the field and gets 14 points with 8 rebs, he impacts the game.

This system falls apart when you feel like Bronny James. I actually think the developers are correct in having us bounce between having us be the best player on our team and the worst player on our team. Once they make it feel like we don't belong on the court it falls apart.

 

[IntentionalPun]

At least give people that option imo, don't blanket force it either way

You'd be splitting your player base by doing this. You can't effectively "turn off' SBMM and then be in the same lobby as someone with it on.

 

[Dorfdad]

You'd be splitting your player base by doing this. You can't effectively "turn off' SBMM and then be in the same lobby as someone with it on.

Let's just fix this by splitting the base in anyone under 2KD plays together. and above in different lobbies. If you play with friends above 2K and your 1.1 you are chasing ti play in higher lobbies and that's it. This keeps casuals in games they can play and enjoy some wins and the sweats in their own lobbies trying to rank up. You can't tell me that would be bad because they already split the player base based on game modes.

 

[Pegasus Actual]

The fundamental problem of matchmaking is that people are losers who want to "win" but will look at every option other than "getting better" as the preferred means to do so.

The real win condition for the average COD lobby is to max out your killstreak. Getting an AC-130, MOAB, Nuke, or whatever you got going. You generally don't get those streaks playing the objective or by playing in balanced lobbies.

Perfect or even good matchmaking for COD isn't a solvable problem.

 

[Three]

I am seriously out of the loop so I'll just ask: why would anyone want skill based match making disabled?

Some people don't like SBMM because it's designed to keep your K/D ratio at 1 and W/L ratio at 1 as best as it can. Instead some prefer to have a free for all where you get put with randoms and sometimes you get slaughtered and sometimes you do the slaughtering. It depends if you prefer extremes or not and how good you are.

 

[Killjoy-NL]

People are idiots. What's new?

Gamers always complain but never really know what they actually want.

 

[Dorfdad]

The fundamental problem of matchmaking is that people are losers who want to "win" but will look at every option other than "getting better" as the preferred means to do so.

The real win condition for the average COD lobby is to max out your killstreak. Getting an AC-130, MOAB, Nuke, or whatever you got going. You generally don't get those streaks playing the objective or by playing in balanced lobbies.

Then what's the objective for the game? I play modes to win I will never be that 3.0 KD guy but at least objectively I do very well and get my points. However many people play objective modes with zero intentions of playing the mode but to camp and get K/D. I honestly wish they removed it as it proves nothing to me.

 

[IntentionalPun]

Let's just fix this by splitting the base in anyone under 2KD plays together. and above in different lobbies. If you play with friends above 2K and your 1.1 you are chasing ti play in higher lobbies and that's it. This keeps casuals in games they can play and enjoy some wins and the sweats in their own lobbies trying to rank up. You can't tell me that would be bad because they already split the player base based on game modes

That would create a MASSIVE jump up once you hit 2KD and then you'd have a massive jump down when you inevitably (since it's a zero sum game) go back down below 2KD.

Once you start messing with any matchmaking based on any skill, you have to do it in a balanced way that's using a lot more complex math than cutting people off at 2KD.

These people do this for a living.. study the math.. study the results using data they track and analyze. You will NEVER please everyone, you mathematically can't.

 

[Dorfdad]

That would create a MASSIVE jump up once you hit 2KD and then you'd have a massive jump down when you inevitably (since it's a zero sum game) go back down below 2KD.

Once you start messing with any matchmaking based on any skill, you have to do it in a balanced way that's using a lot more complex math than cutting people off at 2KD.

Would it though also Im using those numbers as a simplistic discussion. It can't be much different than winning 2 games and than getting rolled by higher skilled players 3-4 games in a row. The seesaw is there already

Also they are always going to follow where they are told the money is so it really doesn't mean anything for us to bitch about stuff we can't change.

 

[IntentionalPun]

Would it though also Im using those numbers as a simplistic discussion. It can't be much different than winning 2 games and than getting rolled by higher skilled players 3-4 games in a row. The seesaw is there already

Also they are always going to follow where they are told the money is so it really doesn't mean anything for us to bitch about stuff we can't change.

It absolutely would. If every single player in a lobby literally averages above 2KD vs everyone else mixed together it’s like looking at the tip of an iceberg. The 2KD group would be tiny and elite.

 

[Three]

I have said this before but something that makes me scratch my head on this SBMM is on the rare occasion I do get on a roll and hitting my shots and killing people this SBMM puts me in lobbies where I am a step behind, I get killed by people who see me before I see them when turning corners

I've had this too. Sometimes I get shot from straight ahead on a corner and the person isn't even there at all. I feel as though they go in and out of a corner quickly where they get to see me based on their ping to host only and before my game updates with their new position they're back around the corner and not visible to me. I think it's too dependent on ping.