Breaking Moloch: the game theory monster in AI, economics and beyond

Written by Dave Wallace
13th October 2023

Who or what is Moloch? And why is it important? Let’s find out.

Moloch represents the inherent inefficiencies and self-destructive tendencies embedded in various systems

Have you listened to Lex Friedman’s podcast, where he talks to the physicist and AI researcher from MIT, Max Tegmark? If not, I highly recommend it. It is two and a half hours well spent.

In the podcast, Max discusses why he was a signatory to the letter by leading figures in the AI world demanding a pause in the development of AI. It is a big topic, and the length of the podcast allows ample time for Max to explain his concerns about the unfettered race to artificial general intelligence (AGI).

In it, he mentions “Meditations on Moloch”, a blog post written by Scott Alexander deconstructing the poem “Howl” by Allen Ginsberg in the 1950s.

Max elaborates, “It’s this game theory monster that pits people against each other in this race to the bottom where everybody ultimately loses. The evil thing about this monster is that even though everybody sees it and understands it, they still can’t escape the race. A good percentage of all the bad things that we humans do are caused by Moloch.”

Moloch in this sense represents the inherent inefficiencies and self-destructive tendencies embedded in various systems—be it healthcare, politics, or economics.

Examples of Moloch in action include:

Arms races: In a military context, a country may rationally decide to increase its arsenal to deter threats and maintain a balance of power. However, when another country does this, the result is an expensive and dangerous arms race, making everyone less safe. No country can unilaterally disarm without risking vulnerability, thus locking them into the Molochian trap.
Environmental degradation: Businesses often operate to maximise profit. Cutting environmental corners can often be a rational decision for individual companies. However, when every company adopts this approach, the collective result is environmental degradation, pollution, and climate change, affecting the quality of life for all.

Scott Morrison brings this concept to life in his blog with the following scenario.

Consider fish farming in a lake.

Imagine a lake with 1,000 identical fish farms. Each fish farm earns a profit of $1,000/month. All is well.

But the fish produce waste, which fouls the water, reducing productivity in the lake by $1/month.

1,000 farms mean enough waste to lower productivity by $1,000/month, which means none are making a profit.

Someone then invents a filtration system, which removes waste products. It costs each farm $300/month, but on voluntary installation, pollution ends.

The farms are now making $700/month.

One farmer (let’s call him Steve) gets tired of spending money on his filter. Now, one fish farm produces waste but makes a $999 profit. Everyone else is making $699. Others see that the farmer is making a profit and disconnect their filters.

Once 400 have disconnected their filters, Steve earns $600/month, and the virtuous filter users only $300/month. Steve does the maths, has a change of heart, and says we all need to use filters voluntarily; otherwise, everyone’s productivity decreases.

They sign the Filter Pact. Everyone uses the filter except for someone called Mike. Everyone is making $699, except Mike, earning $999. Others see and start disconnecting their filters… And so it goes on.

We see from this that a self-interested person is not incentivised to use a filter.

A self-interested individual has some interest in everyone signing a pact but, in many cases, has a stronger incentive to wait until everyone else has signed the agreement and then opt out, which leads to an undesirable equilibrium in which nobody signs.

This story reveals the absurdity of many of the positions we find ourselves in.

Why do banks still invest in oil? We know that burning fossil fuels is harmful to the planet. The evidence is alarmingly all around us. And yet, the majority of banks continue to invest in carbon. Stopping requires a pact. But the fish farm scenario shows why that is unlikely to happen. So, we all understand the madness of it, but no one can stop it. Moloch rubs its hands in glee!

Why do banks constantly build on old legacy infrastructure that results in a sub-standard digital experience? Their strategies and technology choices are driven by value extraction rather than value creation. All can see that this is a race to the bottom, but they cannot stop themselves because the investment required to refocus on value creation may be too big and so a pact across all banks is needed. Anyway, they are making money from investments (see above).

Meanwhile, big tech companies such as Apple observe this, see the bigger picture, and decide to move into banking. Apple’s latest open banking play – providing access to balances in UK bank accounts through the Apple Wallet – is another nail in the coffin of brand-led banking in the UK.

It turns out Moloch surrounds us. I have opened my eyes to its presence, which has helped me understand the causes of the collective madness of our world.

How do we hack Moloch? Ironically, this is where we may need artificial general intelligence. I say ironically because the current race to AGI is an excellent example of Molochian forces in action.

We can see the dangers in the race to it, but nobody can stop because the potential upside is too huge.

But I am beginning to believe humans cannot escape our Molochian tendencies alone. We may need the machines.

AGI could theoretically model complex systems at a level beyond human capability, identifying novel ways to dismantle or reroute these destructive cycles and help provide a way out.

You can imagine an AGI quickly making sense of the fish farm scenario and providing an answer. But you then have to ask, what would it do to get its way?

For all of us, now is the time to open our eyes and truly appreciate what is happening. We can individually and collectively find ways out of the race to the bottom. Whatever that race is.

As a footnote, here’s an extract from Allen Ginsberg’s poem “Howl”:

“Moloch whose love is endless oil and stone! Moloch whose soul is electricity and banks! Moloch whose poverty is the spectre of genius! Moloch whose fate is a cloud of sexless hydrogen! Moloch whose name is the Mind!”

About the author

Dave Wallace is a user experience and marketing professional who has spent the last 30 years helping financial services companies design, launch and evolve digital customer experiences.