Friday, April 15, 2011

Why the Recent SC2 Patch is a Failure

The 1.3.0 patch to the Starcraft 2 client changed the way players view their win / loss statistics... or rather how they don't view them.

The patch notes described this change in a 1-liner:
"Loss counts are no longer displayed in Profile and Ladder pages for players below Master League."

Blizzard, why the hate?

A blue post (Blizzard Employee) on the SC2 Forums reads:

"Win/loss ratios start to matter at the Master's League level because the matchmaking system no longer plays as much of a role, and very good players can rack up many more wins than losses by consistently defeating opponents. Since the matchmaker is no longer as much of a factor, those stats become a more accurate measure of relative success."

"We had a pretty strong rationale for removing losses from the records of players below Master's League. Essentially, prior to that level of play, the losses you've sustained aren't particularly meaningful. Please allow me to explain:

Let's say that you're new to StarCraft II and you play (and lose) a ton of games. You've racked up dozens of losses and a handful of wins, and things look grim. Instead of giving up, you get better, learn effective strategies, and you start playing much more skillfully. Pretty rapidly, the matchmaking system starts pitting you against tougher opponents - but because of the very nature of the matchmaking system, and even though you're making progress and facing tougher opponents, your win/loss rate stays at around 50%. In a scenario like this, it means that your wins will probably never surpass your losses, even though you've become a better player and your wins have become more meaningful than those early losses. Even for someone who starts pretty strong, win/loss numbers should still settle around 50/50 anyway. That doesn't feel good, but worse, those numbers also don't accurately reflect your current level of skill or progress anyway."

Blizzard, I have news for you. Your matchmaking system is fairly broken anyway, and your Leagues and Divisions are pointless. You've successfully replaced one slightly-relevant piece of information with something far less useful.

This change gives players the sense that they're not that terrible by only showing them how many games they've won. Nevermind that you lost 500 games, your 14 wins is wonderful and worth showing. Really, the 14 wins are just as pointless as the 500 losses. If you're going to throw out one statistic, why not both?

The reality is that the wins and losses are both valid statistics, even in ladder matches where they should remain aproximately 50 / 50. The error here is not that the "numbers ... don't accurately reflect your current level of skill or progress" but rather that the interpretation of what these numbers mean is incorrect. People are used to winning far more than they lose. People like to win. They want to see a win ratio of 90 / 10. (90% is an A, after all.) But that's not sustainable for every player, because for someone to win, someone has to lose. Unfortunately, the ladder / division system attempts to boost a player's self-esteem by promoting their mediocrity.

As I mentioned before, the ladder system is completely broken. As it currently exists, there are 7 leagues: bronze, silver, gold, platinum, diamond, master, and grand master. Each league is split into many divisions. Each division has 100 players. Currently, there is no way to tell how many divisions exist in your league.

There is no SC2 Armory, no way to view the other divisions or even know how many other divisions in your league there are. This makes it impossible to size up your competition. The WoW armory was a huge step toward an open-content gaming experience. Much like the advantages that come from open-source, open-content ensures that the community feels the game is being handled fairly, and know what to do to make themselves great. How many books have been written about the game of Chess? Has this diminished it's fun? Has it made the players cheaters? No. Instead, it has inspired some to become better.

Part of the problem with the win / loss system and the ladder / leagues/ divisions is the closed-formula information-hiding employed by the game designers. No one knows exactly what the criteria is to get promoted, or how to improve their MMR. Some may find this mystery intriguing, and, admittedly, at first it is. Until, halfway up the ladder, you find yourself grinding for points, not knowing if you're actually getting better, or if you're simply spinning your wheels.

As of December 2010, Blizzard had sold 4.5 million copies of SC2. On an average night, there are 750,000 people online. Assuming that all accounts are active for the current season, this means that there are 16 players online for every division (100 accounts). That sounds good at first, until you realize that many of them are afk, playing the campaign, custom maps, or vs AI games. Let's be generous and say that 50% of the people online are playing league games. So we're down to 8 players in each division are online. But even if they're playing a league game, they could be playing any match type: 1v1, 2v2, 3v3, 4v4. So really, it's most likely 2 players online in each division playing each format.

Have you ever actually played against someone in your same division? I seriously doubt it. Even if you had, how would you have known?

If you're playing 1v1s, you could play one unique match-up from your division. If you're playing 2v2s, then it's you and your ally online with no one to play against from your division. That sounds pretty lame to me.

So, in order to combat this issue, Blizzard lets you play people from other divisions in your same league, or people from the top of the league below you or the bottom of the league above you. In essence, if there's not enough people to play against that are your skill level, it matches you against people who aren't your skill level... because, let's face it: Even playing a badly-matched game is better than sitting for an hour waiting for a match. We all learned that from the Random Dungeon Finder in WoW.

But take a step back and look at what we just rationalized. Now I'm getting a 1-100 ranking in my division for playing against people that aren't from my division. Isn't that more pointless than my win / loss ratio? Why group players if the groups mean nothing? Furthermore, nobody knows what league or division the people I beat are from. Nobody even cares about that part. All they know is that I'm rank X in platinum league. Talk about misinterpretation of data.

Implementing seasons should help lower the number of inactive players ranked in the division and improving the ability to be matched better to your skill level, but the possibility of playing people outside your division makes the entire division system pointless. In fact, the only thing the division system does is make a cosmetic front for people to feel good about themselves. Even at low levels, people can feel good about making their way up the ranks to #1 in the bronze division, even though that ranking means virtually nothing compared to the people in their own division.

What's so wrong with having only one division for each league? That would make it almost impossible to ever reach the #1 rank. You'd either never get promoted, or would have to get promoted from the top 3,000 players in the league. And who wants to look at their progress for the night and see that they moved up from rank 15672 to 15621? When the numbers are that large, it makes you feel insignificant. But when you're in a division of 100 players, you have a (false) sense that you're better than (a seemingly) large group of players. When in reality, you never even played against any of those people you think you're better than. Implementing the divisions is a facade to promote self-image.

In essence, SC2 is a completely different game in the lower leagues than it is in the higher leagues. Hiding the number of player losses in the UI didn't change that one bit. So, why did you do it, Blizzard?

Improved player experience?
Better game play?
Freed up visual space in the UI?
Offered the players more options to assess their performance?

None of the above. I'd argue that it was purely out of motivation to keep the player base from quitting playing the game. More people playing means more people are buying the game. There is only one reason why Blizzard would do such an awful thing. Money.

Now, I'm not one to openly criticize without offering some suggestions that I think would help alleviate the problem. So, if you're reading this, Blizzard, take some notes.

1. Create a new UI menu where all individual player stats can be viewed. Display Wins and Losses as seperate statistics where they aren't linked together. Show them in various ways so that players can analyze them in their own way. For example, create a option to show wins or losses for the current season compared to last season. Or, show wins versus players that were favored / slightly favored / even. This way, the win/loss count becomes meaningful.

2. Create a SC2 Armory. This shouldn't be hard, you already have the infastructure from WoW to do it. Openly publish what the leagues and division layout is. This type of open-content will please the player base.

3. Increase the size of each division and force matches to be played from people within your division only. Yes, that would cause some slowing of the match-making process, which is why I offer #4.

4. Make the divisions tiered. Within each league, make the divisions openly tiered so that people joining the new league are in the lowest division of the next league. This continues the concept of leagues to a finer granularity, while still separating people into groups of 100 and giving them a sense of progression in their play... but it will actually mean something. Doing so would require the promotion and demotion to happen much more quickly.

5. Open-Content the way you promote the players. Open-content the MMRs, and let the players who are matched up know the MMR of their opponent. Perhaps in the score screen, show a graphical chart of the placement of themselves and their opponent's rankings.

6. The set of static achievements is getting stale. On a regular basis, add new achievements that promote people to get better. For example, add an achievement specific to the current season to get promoted to a new league, or to Win a game against a "Favored" opponent.

In all cases, give the player more information, not less. Make the content more transparent, don't try to hide what's going on. If you need to make another menu to find this information, so be it. Digital real-estate is cheap. Display the information in a new way, don't just make it go away.

1 comment:

Pupil said...

Good call.