Curating Steam: Moral Complexity versus Automatic Norms

Steam, owned by Valve, is the world’s biggest digital distributor of computer games. For years, it’s had frustratingly inconsistent and unpredictable rules on what games could be sold on its platform. After the most recent kerfuffle, Valve’s Erik Johnson published a post to the Steam blog titled “Who Gets To Be On The Steam Store?”

It’s worth reading in its entirety, but here’s my summary: Deciding which games can be sold on Steam is a hard problem that Steam has struggled with for years. There’s a long list of controversial topics and kinds of content - and for each one, many people in Valve’s huge multinational audience feel strongly that it should be allowed on the store and many people feel strongly that it shouldn’t. Many of these topics are also controversial among Valve’s own employees. So rather than continue to struggle with the increasingly impossible goal of consistent curation, Valve is scaling back to block only games that are illegal or “straight up trolling” (later clarified somewhat to mean “designed to do nothing but generate outrage and cause conflict"). Valve’s efforts will instead go toward creating tools to allow people to control what content they see - customers will be able to block specific kinds of games from their own slice of Steam and creators will be able to avoid harassment if they release something controversial.

We’ll have to wait and see the filtering and anti-harassment tools to know whether this plan will succeed, but the reasoning and intent seem solid and likely to lead to a vast improvement over the current unpredictable mess. So I was shocked to see that the reaction from the game journalism community featured widespread rage and contempt.

There’s justifiable concern over the ambiguity of what constitutes “straight-up trolling”, pessimism about the promised tools, and fear that lack of moderation will allow toxicity to flourish. But there’s also a number of people accusing Valve of abandoning moral responsibility for not codifying and enforcing a single standard. These people write articles with titles like “Steam’s Irresponsible Hands-Off Policy Is Proof That Valve Still Hasn’t Learned Its Lesson” and “Valve’s abdication of responsibility over Steam is the worst possible solution,” rejecting the explanation provided in the blog post and seeing even the disagreement that exists within Valve as a moral failing.

“Let us be clear: Valve has not made this choice because it thinks it is the ethically correct thing to do. It has made this choice because it does not want to think about ethics at all, and because it is afraid of making the difficult decisions that a company in its position must face. . . . Valve, however, by its own admission, can’t even resolve its own internal debates about the issue - which, by the way, shows a worrying lack of leadership and a weak company culture."
—Oli Welsh, Steam’s content policy is both arrogant and cowardly

Initially I found this response confusing. I read Valve’s blog post as essentially saying, “Hey, this is actually a really complicated problem. You might find it obvious that one kind of game is evil and that another is perfectly fine, but other totally reasonable people find the reverse equally obvious. There is no one-size-fits-all solution when people disagree on what the obviously right thing to do is.” And some people seemed to answer this with, “C’mon, Valve, it’s easy - you just do the obviously right thing!”

Now, I don’t know what was going through each of these journalist’s minds when they apparently rejected moral complexity and argued Valve should take and apply a single principled stand (presumably the one that is obviously correct to them). But I know one thing that could easily cause this sort of reaction. It’s what economist Robin Hanson calls “automatic norms.”

Every group has norms - mutually-understood rules that govern behavior. Sometimes they exist as unwritten etiquette, such as wearing a suit to a job interview in white-collar industries. Sometimes they are explicit rules, such as the ones against manipulating votes on Reddit. Sometimes they are enshrined in law, such as bans on littering in multiple countries. They vary widely in importance, but in general violating a group norm will get you shunned or otherwise punished by the group’s members. Some norms are so important that even looking like you might be capable of violating them is enough to get you punished.

Robin Hanson describes an illustrative experiment from a 2000 paper from the Journal of Personality and Social Psychology. In this experiment, subjects were presented with a scenario in which a hospital director had to decide whether to save the life of a five-year-old child named Johnny. There were eight versions of the scenario, based on three variables. First, the other option available to the director besides saving Johnny was either to save a different child or to spend the million dollars the procedure would cost on other hospital needs such as equipment or salary improvements. Second, the director either found the decision easy or difficult. Third and finally, the director either chose to save Johnny or took the other option. Subjects were then asked how they felt about the director and whether he should be punished.

Protecting children is a vital norm in most groups, so it shouldn’t be surprising that in the cases where the director chose between Johnny and other ways to spend the money, subjects were much more likely to punish him if he chose not to save the child. But here’s the interesting part - in those same cases, regardless of how the director chose, he is punished more if the finds the decision difficult.

“However, when [the director chooses whether to spend the money on Johnny or elsewhere], he is punished much more if he treats this as a difficult choice. In fact, he is punished almost as much for saving Johnny after much thought as he is for not saving Johnny after little thought!"
—Robin Hanson, Automatic Norms

It could easily be the case that the director is aware of a way to spend the million dollars that is likely to save the lives of multiple children. But to many people, that doesn’t matter. If you have the opportunity to save the life of a child, you are supposed to do it, no questions asked. You aren’t supposed to pause to think about it. You aren’t supposed to admit the situation is complicated. You aren’t supposed to seek out alternate viewpoints. When a norm is sufficiently important to the group, you are supposed to enact it immediately, reflexively, and automatically. Any hesitation is a partial violation.

“These results suggest that when we face a choice, the categorization of some of the options as norm violating is supposed to come to us fast, and with little thought or doubt. Unless we notice that all of the options violate similarly important norms, we are supposed to be sure of which options to reject, without needing to consult with other people, and without needing to try to frame the choice in multiple ways, to see if the relevant norms are subject to framing effects. We are to presume that framing effects are unimportant, and that everyone agrees on the relevant norms and how they are to be applied."
—Robin Hanson, Automatic Norms

That last part is important. A norm that’s been internalized deeply enough to be automatic doesn’t feel like a group norm - it feels like the obvious right thing. It feels like it applies globally to everybody in every situation. When we see someone fail to follow it, or even imply that it’s worth thinking about instead of following automatically, we don’t take that to mean they see things differently or are acting on different but possibly equally-valid norms. Instead, as Hanson puts it, we “feel justified in accusing others of bad motives when they seem to us to violate norms. It seems to us that either they intended to be guilty, or they were inexcusably sloppy or lacking in control of their passions. We usually don’t need to wonder how they framed the situation, what norms they applied, or how they interpreted those norms.”

For this to be a correct response to norm violations, we all need to have internalized the same norms. And for the small hunter-gatherer groups in our ancestral environment, this may well have been true. In such a society, the automatic application of important norms is a strong signal that you are a trustworthy member of the tribe. Hesitation suggests you haven’t properly internalized the tribe’s norms and are thus less trustworthy. But of course it doesn’t work that way in a globally-interconnected society.

“[…W]e’d all need to learn from a lot of pretty similar examples in order to reasonably have much confidence that we were all applying the same norms the same way. . . . This was plausibly the case for most of our distant ancestors. . . . Today however, there are far more people, and more intermixed, who grow up in widely varying contexts and now face far larger spaces of possible actions and action contexts. . . . So it isn’t very plausible that we’ve all converged on how to reliably interpret most norms in most contexts. Thus today we must quite frequently make different judgements on whether actions violate norms. We may converge in judgement with our closest associates and gossip partners, at least on our most common topics of gossip. But for everyone else, if we consider the details of most of their behavior, we will find fault with a lot of it. As they would if they considered the details of our behavior. We are usually sure that we are innocent, but in fact that’s not how many others would categorize us."
—Robin Hanson, 10 Implications of Automatic Norms

This paints a plausible picture of what’s happening with the responses to Valve’s blog post. Valve says that the question of what should be allowed on Steam is complex and difficult. People with relevant automatic norms - regardless of what those norms actually are and which games they suggest should be banned - see this as a deliberate and unethical violation of those norms. Valve says that their own employees feel strongly about what should be allowed on Valve, but are divided. People with relevant automatic norms interpret this as wishy-washiness and lack of moral fortitude. To them, arguments that views and backgrounds differ are irrelevant. The people who don’t immediately see things their way are negligent or downright evil.

“Valve said it would clarify its stance on what sort of games it would allow on the storefront soon. It followed through on that today with a statement that–after a waffling preamble about how hard the problems are and how its own employees don’t agree on what to do–explained it wouldn’t really be doing anything. . . . All Valve understands is that it doesn’t want to make a choice and draw a line. It doesn’t want the responsibility of being the biggest platform for PC games on the planet; it just wants its 30%."
—Brendan Sinclair, Valve’s new content policy is a gutless attempt to dodge responsibility

Again, I don’t know for sure whether any of these journalists are thinking in this way. But I believe this way of thinking would result in commentary like that I’ve quoted above. And I think it’s a close-minded and frankly dangerous way of looking at things.

We live in a complex and interconnected world. Our leaders - commercial, social, political, and otherwise - make decisions that affect us on scales unimaginable to our tribal ancestors. The ability to recognize and carefully consider moral complexity is vital. We should fear those who find every moral question simple and easy. The ones who think there’s a single and obvious correct set of games that everyone in the world should have access to. The ones who wouldn’t hesitate to save Johnny today instead of asking whether we could save ten like Johnny tomorrow.

If you’re still not convinced, know this: There’s a class of game that I think is obviously evil. These games irresponsibly promote abhorrent worldviews and actions. While it’s impossible to calculate how much psychological and cultural damage they cause, I think there’s a good chance the world would be better off if they were banned. I never play these games and while I try to be tolerant there’s a part of me that can’t help but view anyone who enjoys them as morally suspect. I can tell you all of that without you having any idea what kind of game I’m talking about. Should you trust me to decide for you whether you should be allowed to purchase and play one of these games?

The games I’m talking about, by the way, are military shooters. Even when they don’t glorify military adventurism or present complex geopolitical issues as problems to be solved by shooting enough brown people, real-world war should never be viewed as a fun game to play. It sickens me on a deep level that these games have been so popular for so long. But I don’t say this often and I would never say these games shouldn’t be allowed to exist. I know that to a lot of people these games are obviously harmless, and there are also a lot of people who feel the same revulsion about games that I actively enjoy. I don’t think it would be correct to remove Call of Duty from Steam just because I find it obviously evil. After all, I want to play Senran Kagura and Depression Quest and Gone Home and there’s somebody out there who finds each of those obviously evil instead.

All Valve is saying is that none of us should be able to decide for all of us.