AI plays Pokémon Red on a single RTX 3080 Ti
Teaching an AI to Catch a Pokémon: The Quest for Digital Nostalgia
What happens when the desire for digital fun meets the power of artificial intelligence? For developer stmonty, the answer involved a classic Nintendo game and a consumer-grade graphics card. He took on the ambitious task of training a small AI model to master the world of Pokémon Red, turning a beloved childhood memory into a complex machine learning problem.
This wasn’t just a simple game-playing attempt; it was an exercise in teaching a machine to understand context and action—a concept rooted in the research of AI scientist Yann LeCun, who champions the development of world models over traditional large language models. stmonty built his project based on LeWorldModel research, aiming to demonstrate that sophisticated world models can be trained effectively without relying on massive datasets.
The model, built from just 12.5 million parameters, learned to navigate the game not by predicting the entire next screen, but by focusing on compressed summaries of just 192 numbers. By observing the screen output, the AI learned the meaning of every button press, developing an internal map of cause and effect necessary for gameplay.
The initial training phase was intentionally “reward-free.” The model had to learn the rules purely through observation, without an immediate sense of winning or losing. Goals were introduced later, allowing the AI to develop complex strategies and planning skills. This design mirrors emerging research in single-GPU trainable models, showing the potential of smaller, specialized models.
The training process itself was intensive. stmonty fed the model 42,382 grayscale frames collected over more than 1,000 short runs, utilizing a saved game from Professor Oak’s lab. This data included scripted routes mixed with random wandering, a necessary step to ensure the model learned the nuances of the game world rather than just following pre-set paths.
To successfully navigate the game, the model had to move beyond simple reaction. It developed complex plans, attempting sequences of up to 14 button presses in each round. These plans were tested through extensive simulation, where the model would examine 512 different possibilities, narrowing down the search space repeatedly until a successful path emerged.
The challenge was realizing that small errors compound rapidly when predictions are made from the model’s own outputs. stmonty hypothesized that this compounding effect was a major hurdle to success. After fine-tuning, the model achieved a significant breakthrough, successfully selecting a starter Pokémon, demonstrating that even modest models can be tailored to highly specific, complex tasks.
While the initial goal of starting the journey in the lab proved exponentially more difficult—as the difficulty scales with plan length—the success itself is a powerful testament to the viability of small, focused AI models. The project, code for which is open source on GitHub as lePokeRed, remains a fun and accessible blueprint for exploring the exciting intersection of AI and digital nostalgia.
Buy on Amazon
Credit
