Evaluating Eternal: Does Variance make for a less skillful game?

Hi everyone! Today, we are going to take a diversion from draft content and talk about some math that goes into Eternal. If you haven’t had a chance, I would strongly recommend checking out the previous article in this series talking about why we need variance in Eternal. Now, I’m going to build on those foundations and talk about the interaction between skill and variance. Every so often or not, reddit threads pop up with players complaining that the high variance in this game removes any need of skill. I strongly disagree with that assertion, and will attempt to illustrate it with this article. The article will be broken down into the following sections:

  1. What is Variance?
  2. What is Skill?
  3. Variance and Skill as independent properties of a game
  4. Dealing with Variance is a skill in itself
  5. Does it matter that the better player doesn’t always win?

What is Variance?

Before diving into the main gist of my argument, it’s important to first establish some definitions so that we don’t get confused by terminology. In this article, I define any factor that arises due to randomness as variance. For example, Icaria hitting her warcries on a Rakano Artisan instead of another Icaria is a type of variance. Getting stuck on 2 power is also variance.

It’s important to recognize that because there is inherent variance in almost any game, the better player won’t always win every time. There will always be a “noise ceiling” which is what we can imagine to be the highest attainable win rate assuming you are perfect at the game and similarly a “noise floor” which is what you expect a brand new player’s win rate is.

What is Skill?

This is probably a much more controversial term to define but also the most important, because if everyone simply argues using their own definition of skill, it would be impossible to reach a consensus. The definition of skill that I’m going to use is the ability to identify increasingly optimal plays and deck tweaks. Imagine a board state where there are multiple lines of play, an average player may only be able to see 2 lines, and take 1 of them. A more skillful player might see the 3rd and better line, and this ability is what I quantify as skill. Notably, the better line might not always pay off immediately, but in the long run, you will win more by taking the optimal line.

This is a pretty abstract and harder to nail down definition of skill, but conceptually it should be easy to grasp. Ultimately, the idea of skill is this quality that increases your odds of winning and something that you can develop by putting in the time and effort and how skill-testing a game is a reflection of the demand the game places on this quality. However, it is important to highlight that the raw values of the odds of winning is not an accurate criteria to measuring the amount of skill a game tests for (as I will illustrate in the following section).

Variance and skill as independent properties of a game

With our definitions in order, now we can address one of the main points that I want to assert: The skillfulness of a game is nearly independent of the variance. To illustrate this claim, consider a variation of chess, with the only modification that after winning the game of chess, the winner has to flip 2 fair coins, and if both coins come up tails, the winner loses the game instead. Now, assuming that chess is completely deterministic (I know its not, but it’s close enough and this assumption greatly simplifies the math), we can see that the best player with a 100% winrate in normal chess will only have a 75% winrate in this variant. Similarly, the worst player goes from a 0% winrate to a 25% winrate. In fact, we can create a 1-to-1 mapping of a players winrate in normal chess to this variant where:

Variant winrate = (0.75*Chess Winrate)+(0.25*(1-Chess Winrate))

Now, notably, the winrate differential between the best and worst player is only 50%, as compared to 100% in normal chess. Does that make this variant any less skill-testing than chess? By my definitions, the answer is no. The skill-set required to go from 25% to 75% winrate in this variant is exactly the same as the skill-set required to go from 0 to 100% in normal chess. Thus, I would argue that both games are equally skill-testing in concept, despite the better player not always winning in the variant. In fact, we can now come up with a more refined definition of how skill-testing a game is as the amount of skill differential between the player with the “noise ceiling” winrate and the player with the “noise floor” winrate.

However, it is important to realize that there is an implicit assumption in this evaluation that there is a difference between the “noise ceiling” and the “noise floor”. If we were to slightly tweak the variant such that the winner only gets to flip a single coin and loses on a tails, it completely removes the skill-testing element since every player is guaranteed a 50% winrate in the long run.

A similar argument is also applicable to Eternal. Yes, there will be games where you lose because RNGesus decided that you are not drawing more than 2 power. However, that does not make Eternal any less skillful. Only if every player is guaranteed a 50% winrate in the long run (which is not the case, especially if you look at players like Unearthly and Camat0), can you then claim that the variance makes it such that skill doesn’t matter.

Dealing with Variance is a skill in itself

Now, I know we’ve all had those unwinnable games, being stuck on 2 power for 10 turns, or drawing 10 power in a row. Those games suck, but they are also extreme outliers. In most cases, variance simply makes it more favored for one of the two players, and the ability to capitalize on good variance and mitigate the impact of bad variance is a skill in itself.

For example, if you are in a Stonescar aggro mirror and you’ve stumbled on power, resulting in a fair few 3-4 drops being stuck in hand. At this point, you know that you are on the defensive, and you need to maximize value from whatever cards you can play to try and stay alive. Thus, instead of torching on your main phrase (to deny your opponent a warcry), you can, and probably should, hold back the torch to blow out tricks. You can even make it more tempting for your opponent to do so by blocking a Pyroknight instead of a Oni Ronin. Similarly, if you are playing Stonescar aggro against a control deck and notice that your opponent missed his 4th power drop, you should definitely vomit your hand onto the board and abuse the fact that he is unable to harsh rule on turn 5.

There are a lot different lines of plays that you can make that the average player often gloss over and attribute their losses to bad luck. For example, if you were playing Argenport Midrange with only 1 Impending Doom in hand and 2 Protects, you should only play your Doom on 5 (or 6 now, RIP 1-cost protect), so that you can hold up Protect and effectively turn your 1 unit hand into 2 unit. Sure, you got the bad end of variance by only drawing a single threat, but you can mitigate it by playing off-curve to maximize your chances of winning.

Similarly, an interesting situation occurred in one of our Team League matches. This was a crucial game between OND on Argenport Midrange and SPG Revenge on Icaria Blue and the board state was at follows:Capture.JPG

The swing in with the giant Artisan would put us into torch range if we didn’t block, but if we were to chump block, we would lose lethal if he had a removal for our other flier. Now, the tempting thing seems to be to just block (or not block), and blame variance if they have a removal (or a torch). However, because you know each other lists and what has been played so far (2 torches and 2 removal spells in the void), you can make an educated guess on what’s more likely (removal spell in this case). Thus, not blocking is the correct line (despite losing ~2/55 games) because it wins you the other ~10/55 games where it matters. In the remaining ~43/55 games, blocking or not blocking doesn’t matter since they don’t have either removal or torch in hand. As it turns out, they topdecked a Throne Warden, and we would have lost the game then if we had chump blocked. #rewarded

Does it matter that the better player doesn’t always win?

Now, this is detracting slightly from the mathematical nature of this article, and more into a game design concept discussion. Importantly, while I know that it’s impossible that the better player will always win, there is an added discussion as to how often should the better player win? The more often a player wins due to better play, the more they are likely to invest time and effort into improving their skills. But on the flip side, there is also the argument that clear cut differentials would demotivate weaker players and act as a disincentive for them to play (since they just end up getting their butts whooped). So, perhaps the real answer is (as much as a coop-out as it is) that it depends.

However, I want to highlight a distinction between different types of effect that variance can have on the game’s outcome. Imagine that your winrate against opponent’s of differing skill level is as follows (illustrated by the blue line).

Capture.PNGNow, there are 2 ways in which increased variance can affect this graph: Skill-independent variance (orange line) or skill-dependent variance (grey line).

Skill independent variance is a pretty straight forward concept to grasp, the above chess variant is a perfect example of it. Regardless of how good or bad you are, you will always win 25% of the games that you are supposed to lose, and lose 25% of the games that you are supposed to win. In contrast, skill dependent variance is something that can be overcome with a sufficient skill disparity. For example, imagine a variant of chess where both players flip a coin and loses a pawn if they flipped tails. Now, you could start off with a slight disadvantage, but if you are significantly more skilled than your opponent, you will still end up winning the game, and vice versa.

And this is an important subtlety that  I want to highlight. Skill independent variance is bad for the game, whereas skill dependent variance can be good for the game. Skill independent variance only leads to increased saltiness because players are annoyed by games where they have 0 control over the outcome. To take an example from Hearthstone, this is why Yogg Saron was utterly flawed in terms of game design. It is literally a card that you play when you are about to lose to just flip a coin with your opponent, requiring zero skill on either side to win the game. In contrast, I would argue that skill dependent variance is good for the game. This makes it such that the skill-reward curve is less steep and weaker players will be able to see improvements in their winrates with less effort. This acts as an incentive for players to continuously tryhard and improve, because they get a tangible reward (improved winrate) that scales with effort.

However, in reality, most variances are a combination of both aspects. Take the power system as an example of variance. Missing a single power drop is probably mostly skill dependent variance while getting stuck on 2 power for 10 turns is definitely skill independent variance. That said, we can come up with some forms of quantification for the amount of skill dependent and skill independent variance each element of the game introduces and weigh it’s pros and cons. The power system, a topic pretty much talked to death and back again, is a good illustration of where skill dependent impacts of variance (missing 3rd power post mull is approximately 15% on the play) occurs frequently, whereas skill independent impacts of variance (missing 3rd power for 5 turns straight is approximately 1% on the play) occurs much more rarely.

Conclusion

Another long article down! If you made it all the way to the end, Congrats! and thanks for sticking it out with the article! Hopefully, I’ve managed to convince you that just because a game has inherent variance, it doesn’t make it any less skill-testing. If anything, the addition of skill-dependent variance increases the skill level, because being able to capitalize on good variance and negating bad variance is a skill in itself. As always, let me know what you think in the reddit thread!

Variance is but a construct for our finite minds to deal with the infinite possibilities of the universe,
Flash2351