Google one step closer to “API” as Gemini completes a Pokemon classic playthrough 

The latest version of Gemini — Google's AI model — has recently completed a playthrough of Pokemon Blue. Current Google CEO, Sundar Pichai, has claimed that the most recent version of Gemini (2.5 Pro) has successfully collected all eight badges in Pokemon Blue. Pichai referred to this as Google coming close to achieving "API" — Artificial Pokemon Intelligence, as he referred to it.

This marks a significant win for Gemini over its competition, Claude, which has been struggling to defeat Pokemon Red. However, there are a few things worth noting. For starters, Claude and Gemini are not similar LLMs (Large Language Models) and function differently. Secondly, Pokemon Red/Blue are the first installments of the franchise, lacking many features of the later games.

---

Gemini achieves "API" — Artificial Pokemon Intelligence

Sundar Pichai on Artificial Pokemon Intelligence (Image via X)

Google CEO Sundar Pichai celebrated Gemini 2.5 Pro's victory with a post on X. He thanked the coder Joel Z for creating and maintaining the live stream. He also thanked the people who watched Gemini complete the game. Joel Z is not affiliated with Google as per his own and Pichai's admission.

While Joel Z was inspired to do the live stream after being inspired by Anthropic's Claude AI's attempt to complete Pokemon Red, he clearly stated that Gemini's successful attempt didn't indicate that Gemini was better than Claude. He even stated that this couldn't be used as a benchmark to determine how LLM would play Pokemon games in the future.

Here's what he had to say:

“Please don’t consider this a benchmark for how well an LLM can play Pokemon. You can’t really make direct comparisons — Gemini and Claude have different tools and receive different information.”

It is also worth noting that both AI models needed help to play the game. Not only do they need screenshots from the game, but external agents also intervened to help Gemini complete the game. When asked about this, Joel Z said:

“My interventions improve Gemini’s overall decision-making and reasoning abilities,”

“I don’t give specific hints — there are no walkthroughs or direct instructions for particular challenges like Mt. Moon. The only thing that comes even close is letting Gemini know that it needs to talk to a Rocket Grunt twice to obtain the Lift Key, which was a bug that was later fixed in Pokemon Yellow.”

Joel Z clarified that this was necessary as LLMs were unable to create mental maps like humans can, i.e., they are unable to recognize that a building is a building and so need help in that regard. He did not consider helping Gemini out in this way "cheating," instead acting as a compensation for the LLM's limitations.