Programmer Creates An AI To (Not Quite) Beat NES Games

Programmer and CMU PhD Tom Murphy created a function to “beat” NES games by watching the score. When the computer did things that raised the score it would learn how to reproduce them again and again, resulting, ultimately, in what amounts to a Super Mario Brothers-playing robot. The program, called a “technique for automating NES games,” can take on nearly every NES game, but it doesn’t always win.

You can read his full paper here but, as you can see from the above video (fast-forward to about six minutes to see Mario in action), the game does most of the things normal humans would do but consistently uses very difficult tricks to, say, attack two Goombas in rapid succession.

Murphy writes:

Bytes in memory (and sometimes 16- and 32-bit words) can contain interesting game facts like the player’s position in the level or score. The central idea of this paper is to use (only) the value of memory locations to deduce when the player is “winning”. The things that a human player perceives, like the video screen and sound effects, are completely ignored. As an additional simplification, we assume that winning always consists of a value going up—either the position in the level getting larger, the score getting larger, the number of lives, the world or level number getting bigger, and so on.

By giving the program a little bit of training – how to jump, what to grab – the program becomes a coin-hungry juggernaut, stomping turtles and taking no mushroom prisoners. Murphy ran a few other games through it, including Tetris, and found that the program would eventually just pause itself rather than continue playing and lose, a tactic shared by annoying, over-competitive cousins around the world since 1985.