Matchmaking-Improvements

There's a lot of conversation about matchmaking in Predecessor. Today we're talking about what's worked, what hasn't, and what we're doing to make it better.

Developer Diaries 2023年7月24日

Matchmaking is hard .

So let’s talk about it a bit.

What is Skill-Based Matchmaking?

Skill-Based Matchmaking is the process by which we assign teams for a match of Predecessor. When you hit the “Play” button, a message is sent to our Matchmaking Service with information about which region and Game Mode you’re trying to play, how many people are in your party, and your MMR ( Matchmaking Rating). MMR is the number our system uses to assess your skill and experience in Predecessor based on your past wins and losses against stronger and weaker opponents.

Everyone starts at the same baseline MMR value, and from there your number constantly adjusts as you win and lose games, as the system attempts to find you the fairest matches for your skill level. You gain more MMR for beating opponents with a higher MMR than you, and drop more when you lose to opponents with a lower MMR than you. Eventually you’ll end up at a point where you’re winning about as often as you’re losing against players of a similar MMR.

But isn’t it hard to assign a single number to your skill level?

Absolutely! This is one of the great unsolved challenges in competitive gaming, especially for a team- and role-based game like Predecessor. Competitive performance in Predecessor is a mixture of natural skill, experience, your comfort with a particular team role or hero, the playstyles of teammates and opponents, your mood, your ability to communicate with others, your desire for intense competition or silly fun, or even how much sleep you got the night before! Assigning a single number to your chances of winning a match will likely never be an exact science, but the goal of our matchmaking system is to get close enough to create a fair, balanced game with every single match.

So does it all work?

Well, yes and no. The Glicko Matchmaking Rating system is a tried-and-tested method for assessing the likelihood of a team winning against any given set of opponents. Variations of this system are used in many of your favorite online competitive games.

We currently use our own modified system based on Glicko, and we’ve found that we can predict a team’s actual probability of winning a match with over 93% accuracy for the vast majority of all Predecessor matches.

So, what are the issues?

Well, there are a couple.

Grab your net, we’re going on a bug hunt.

First and foremost, we need to discuss a significant bug that up until recently had silently plagued our matchmaking system. The matchmaking process can be broadly split into two distinct stages:

1. The service finds 10 players with comparable skill levels based on similar MMR.

2. It balances those players into two fairly-matched teams of 5.

Occasionally, the matchmaker would overwrite the MMR value for a player with a bogus default value while trying to update the “ matchmaking request ticket” and pass it between those two matchmaking steps. When that MMR value was overwritten for even a single player, the entire process of rebalancing teams to make sure those 10 players were evenly sorted was thrown out the window. Sometimes, the random sorting was reasonably fair, but often a match that could have been nearly perfectly balanced if appropriately mixed ended up with differences of several hundred MMR.

What made this bug particularly nasty to catch was that our own Predecessor game clients and servers continued reporting all the correct MMR values to our own data pipeline, while that bug lived deep within thousands of lines of external backend code.

Thanks to the collaboration between our own internal teams and the matchmaking provider, we were ultimately able to track down the bad code and develop a fix. This change was rolled out by the matchmaking provider less than two weeks ago (on July 12th), and the improvement to match fairness was significant.

Inflation hurts

Beyond that significant team-sorting bug, we’ve also been slowly combatting something we’ve been calling MMR Inflation. Every player started at the same initial MMR when we launched Early Access, but over time the average player MMR has shifted several hundred points above where new players begin their Predecessor journey.

However, much of this inflation was the result of accidental introduction of “extra” MMR into the system due to an error in the initial version of the post-match MMR adjustment calculation for parties. We were able to detect and fix this error, but while the improper calculations were out in the world quite a few players were moved into MMR brackets that were well above their skill level.

3’s a crowd, 4’s a party

The Party Multiplier has been another hot topic in the community. Our data has shown a strong correlation between party size and the odds of winning a match at a given MMR, where larger groups of partied players gain a significant advantage over ungrouped players of the same level.

This is due to a number of factors including faster and more accurate communication, advantages in coordinated role or hero selection, and team chemistry that comes from playing with your friends.

We rolled out a Party MMR Multiplier system that inflates your team’s MMR for matchmaking purposes as a function of how large your party is. Essentially, a bigger party gains a bigger “boost” to their MMR to find stronger opponents.

Practical limits

Lastly and as you might expect, there are regional and time-of-day population limitations that we need to design around. While we’d love to configure our systems to make the absolute best matches during peak weekend evening hours, the reality is that we also need the system to function at 3am on a Tuesday in our smaller regions.

There is a constant balance between matchmaking queue times, matchmaking quality, and player latency as determined by the number of matchmaking regions, and optimizing all three of those factors takes time and experimentation.

Developing and Testing

To tackle something as complicated as matchmaking, we put together a task force of developers from our Game Design, Analytics, and Backend Engineering teams.

Turning player feedback on matchmaking quality into meaningful solutions first required a stronger understanding of the system as it previously existed. We began by extracting and analyzing data on every aspect of matchmaking quality we could find, including team-to-team average MMR differences, queue duration, and the effects of party composition.

Once we felt we had a comprehensive understanding of our problems, the next step was designing and testing solutions.

New and Improved

As mentioned above, we’re thrilled to announce that the first improvements have begun rolling out. With the primary team rebalancing bug addressed on July 12, the median MMR difference between teams has dropped from 60 MMR points to less than 20. These improvements aren’t sufficient to satisfy our goals, but this change represents a big step forward for overall match quality.

To combat MMR Inflation, we ran a database change of everyone’s MMR to move those stats closer to the starting value.

The last immediate improvement is the unveiling of our new Party MMR adjustment with Version 0.9.1. This is a formula that accounts for both the party size and the party’s average MMR when calculating the adjustment factor, and is scaled to better align with the match data we’ve collected.

Future Plans

The improvements to team mixing, queue times, and party adjustments are a great start, but we’ve got much bigger plans in store. We’re developing new systems to allow us to dynamically adjust matchmaking configurations to better accommodate time-of-day and regional population requirements.

Last and largest of all, alongside the improvements for the existing matchmaking system, our team has been developing an entirely new matchmaker. This new system necessitates major backend changes to our service architecture in addition to significant development and tuning work, so we don’t have a release date yet.

Thanks as always to everyone that’s supported Predecessor throughout the Early Access journey. Your feedback continues to be invaluable as we develop this game into everything we know it can be.