AlphaZero

One program, three games

AlphaZero is a program developed by DeepMind, Google's artificial intelligence lab. It was presented on 5 December 2017 in a preprint by David Silver and colleagues. According to the preprint, the program started from random play, with no examples of games played by humans, and within 24 hours reached a superhuman level in chess, in shogi (Japanese chess) and in Go. The paper also reported wins against a world-champion program in each of the games. The full version, with updated results, was published in the journal Science on 7 December 2018. The novelty was not that a computer won: back in 1997, IBM's chess computer Deep Blue beat world champion Garry Kasparov. The novelty was in the method. According to DeepMind, strong chess engines such as Stockfish relied on thousands of rules and heuristics written by hand by strong players, to cover every possible situation in the game. AlphaZero replaced all of that with a deep neural network and general-purpose algorithms: no hand-made strategy rules, no opening books and no endgame tables. What the researchers did provide, according to the paper's supplement, were the rules of the game and the way the board and the moves are represented.

From the AlphaGo line

AlphaZero is one link in a DeepMind lineage. AlphaGo, which beat Go champion Lee Sedol in March 2016, learned from Go games played by humans, and later from playing against itself. Its successor, AlphaGo Zero, described in the journal Nature in October 2017, no longer used any human game data: it learned only from playing against itself, and after three days of training it beat the version that had defeated Lee Sedol, 100–0. AlphaZero took this approach a step further: the same algorithm was applied to three different games. According to Wikipedia, one of the differences is that Go has no draws, while chess and shogi can end in a draw, so AlphaZero also had to take such a result into account. In 2019, DeepMind presented MuZero, which reached the same level in all three games without being given the rules at all, and also played Atari video games.

How it learns: millions of games against itself

According to DeepMind, to learn a game, an untrained neural network plays millions of games against itself, in a process of trial and error called reinforcement learning. At first it plays completely at random, and over time it learns from wins, losses and draws, and updates itself to prefer good moves. The idea of a machine that learns by playing against itself appeared in film as early as 1983: in the movie WarGames, a military supercomputer is made to play tic-tac-toe against itself, and it concludes from the repeated draws that the only winning move is not to play. In the experiment published in 2018, training lasted about 9 hours for chess, 12 hours for shogi and 13 days for Go, and a separate instance of the program was trained for each game. According to the paper, the training games were generated by 5,000 first-generation TPUs (chips that Google developed specifically for artificial intelligence), and 16 second-generation TPUs trained the networks. During play itself, the trained network guides a search algorithm called Monte Carlo tree search, which checks which moves are more promising. Here lies the striking difference from classic chess engines: according to DeepMind, in the experiment published in 2018, AlphaZero searched on average about 60,000 positions per second in chess, compared with about 60 million for the version of Stockfish that was tested. According to Wikipedia, it makes up for this by having the network focus the search on the most promising continuations. Stockfish, by contrast, uses a game-tree search based on alpha-beta pruning, plus a few hand-designed heuristics.

The results, and the debate over the match conditions

In the early 2017 version, AlphaZero played 100 chess games against Stockfish 8, the 2016 champion of the TCEC computer chess championship, at one minute per move, and finished with 28 wins, 72 draws and zero losses. The result drew criticism. According to Wikipedia, grandmaster Hikaru Nakamura argued that there was a large hardware gap: AlphaZero, he said, ran on Google's supercomputer, and he compared Stockfish's hardware to a laptop. One of Stockfish's developers, Tord Romstad, noted that the engine is not built for a fixed time per move and that the version that played was a year old. In the full paper in Science, DeepMind addressed some of the claims: Stockfish was given 44 processor cores, as in the TCEC championship, and both sides played with three hours per game plus 15 seconds per move. AlphaZero ran on a single machine with four TPUs and 44 cores. In a series of 1,000 games it won 155 and lost 6, and according to Wikipedia 839 games ended in a draw. In shogi it won 91.2% of the games against Elmo, the 2017 champion version, and in Go it won 61% of the games against AlphaGo Zero.

An "alien" style

What fascinated the chess world was less the result and more the style. Demis Hassabis, DeepMind's CEO and a chess player himself, described it, according to Wikipedia, as "alien": "It's like chess from another dimension." According to DeepMind, AlphaZero is willing to sacrifice pieces early in the game for an advantage that pays off only in the long run, and plays so that its own pieces are as active and mobile as possible and its opponent's as little as possible. Grandmaster Matthew Sadler and Natasha Regan, who holds the title of Women's International Master, analyzed thousands of its games for the book Game Changer, published in January 2019, and Sadler compared reading them to discovering the secret notebooks of a great player from the past. Former world champion Garry Kasparov, who lost to Deep Blue in 1997, said he could not disguise his satisfaction that it plays with a very dynamic style, "much like my own." The same issue of Science also carried a perspective article by Murray Campbell, one of the creators of Deep Blue. In 2020, DeepMind researchers, together with former world champion Vladimir Kramnik, used AlphaZero to test versions of chess with small changes to the rules, and to see what games between strong players would look like in them.

Why this is not general AI

AlphaZero is sometimes presented as a step toward general intelligence, but it has clear limits. It must be given the rules of the game in advance: according to DeepMind itself, systems that plan ahead, such as AlphaZero, rely on knowing the dynamics of the environment (the rules of the game or an accurate simulator), which makes them hard to apply to messy real-world problems. It was built for games in which all the information is visible to both sides, and it learns each game from scratch. In 2018, DeepMind wrote that AI systems reach a very high level in a specific skill but often fail when the task changes slightly, and it saw AlphaZero as an important step toward solving that problem, not the solution. A study published in 2026 in the journal Machine Learning showed another limitation: in the game of Nim, in the implementations tested, AlphaZero-style agents reached champion level only on very small boards, and their performance degraded as the board grew. The authors linked this to the difficulty general-purpose neural networks have in learning abstract functions such as parity on their own. The follow-up came in several directions: MuZero learned without being given the rules; a version of AlphaZero called AlphaDev found faster sorting algorithms, which were added to the C++ standard library in the LLVM project; and the chess community built Leela Chess Zero, an open-source engine announced in January 2018 to reproduce the achievement.