Jericho and TextWorld

Two environments, roughly the same researchers, one question

Jericho and TextWorld grew out of roughly the same research group: Microsoft Research scientists, among them Marc-Alexandre Côté and Xingdi Yuan of the Montreal lab, and Matthew Hausknecht, who are authors of both papers. Both environments tackle the same question: how do you test whether an AI agent truly "understands" language, rather than just acting correctly in a visual world like screen-based games? But they solve it in opposite directions. TextWorld, which the researchers published in 2018, generates entirely new adventure games, at whatever difficulty and scope the researcher chooses. Jericho, which Hausknecht and his co-authors published in 2019 — among them Prithviraj Ammanabrolu of the Georgia Institute of Technology — runs real games that people wrote in the past, including Zork itself.

TextWorld: a factory for custom-made games

TextWorld was presented in 2018 at the Computer Games Workshop of the IJCAI conference in Stockholm. It's a Python library that runs text games, tracks the game's state, and decides when to give a reward to the agent playing it. It comes with a fixed list of games the researchers analyzed, but its main advantage is different: you can write a game by hand, and you can also generate games automatically. Automatic generation allows precise control over the difficulty, scope, and language of the game — meaning you can, for example, generate a thousand small games that are all similar but not identical, to test whether an agent that trained on some of them also succeeds on parts it hasn't seen, rather than simply memorizing a single game by heart. TextWorld also frames these games in the formal language of reinforcement learning, and comes with a set of benchmark games the researchers built themselves. An automatically generated game is built as source code in the Inform 7 language and then compiled into an executable file in the Glulx format; object names, room descriptions, and instructions come from a context-free grammar (CFG) with two themes, "house" and "basic." The researchers also guarantee that every generated game is solvable: a winning path from the starting state to the goal is known in advance. The fixed list that comes with TextWorld contains 50 games written by people, chosen following earlier work by Fulda and colleagues from 2017.

Jericho: running the original Zork I

Jericho solves the opposite problem: it doesn't generate games, but runs real text adventure games that people wrote in previous decades, including Zork I itself, a file in the Z-machine format. It's an open-source Python interface styled similarly to OpenAI Gym, so a learning agent can connect to it without dealing with the technical details of each individual game. The paper that introduced it, Interactive Fiction Games: A Colossal Adventure, was presented at the AAAI-20 conference, and was written by researchers from Microsoft Research and the Georgia Institute of Technology. The authors argued that text adventure games are an excellent testbed for language agents, precisely because they combine three challenges at once: a vast action space, language understanding, and commonsense reasoning. The conference itself, AAAI-20, was the 34th conference of the Association for the Advancement of Artificial Intelligence, held in February 2020, a few months after the paper first appeared as a preprint on arXiv. Jericho's own code is open under the GPL-2.0 license and published in the repository github.com/microsoft/jericho.

How Jericho shrinks an action space of billions

An agent choosing a command of up to four words out of a vocabulary of about 700 words faces, in theory, about 240 billion possibilities. Jericho narrows this down through "action templates": a template like "put [object] in [place]" already fixes the structure, and only the blanks need to be filled from the game's vocabulary. For a game with about 200 templates and a roughly 700-word vocabulary, this brings the space down to about 98 million possibilities, three orders of magnitude fewer. Jericho also identifies which actions are actually valid, by executing candidate actions and checking whether the world state changed as a result, and it maintains a "world object tree": a description of what's inside what and what's next to what, which helps identify where the player is and which objects are nearby.

What happened when AI agents actually played: Zork as a test

The researchers ran four different agents on 32 games supported by Jericho, including Zork I: a completely random agent, a heuristic agent called NAIL that had previously won a competition, and two learning agents called TDQN and DRRN. The result was measured as the average percentage of progress in the game, out of the maximum possible score. On average, the random agent reached 1.8% of the possible score, NAIL 4.9%, TDQN 6.1%, and the best of them, DRRN, only 10.7%. Writing on the Microsoft Research blog, the authors said: "Much improvement is needed before these algorithms and others can rival skilled humans at these games." These numbers illustrate a huge gap: games written decades earlier, with a parser that matches words against a closed list, were still, in 2019, a test that language-learning agents mostly failed. In five of the 32 games, no agent at all, from the random one to DRRN, managed to score any points.

The connection to the history of text games

These two environments are the direct continuation of the question at the heart of the earliest text adventures: how do you know whether a computer truly "understands" what you typed? The two-word parser of Colossal Cave Adventure from the mid-1970s, and the more complex parser of the 1977 Zork, gave a clear answer: they didn't understand; they matched words to a closed list. TextWorld and Jericho are the tools today's AI researchers use to test whether learning agents — which don't receive a closed word list but instead learn from free-form language — come any closer to real understanding. The 2019 results show that, at the time, the answer was still no: even the best agent tested came nowhere near a human player's level.