Thiagolira
Show HN: A 178K Neural Net that beats Pokémon Roguelike
Last verified:
What is Thiagolira?
Thiagolira This tool is a reinforcement learning (RL) agent trained to play and beat the fan-made Pokémon game poke(rogue)like (available at https://pokelike.xyz/). The author, Thiago Lira, trained the agent after getting frustrated with losing hundreds of times at the game. The RL agent uses the PPO (Proximal Policy Gradient) algorithm and consists of two neural networks: a Policy Network that learns to take actions given the game state, and a Value Network that assigns expected rewards to possible actions.
Thiagolira pricing
Pricing model: Freemium
Free - The project is completely open-source with all code available for anyone to run for themselves. No paid tiers or plans mentioned.
Thiagolira pros
- Extremely small 178K parameter model size
- Uses PPO algorithm same as LLM post-training
- 9% win rate across 1000 runs
- Open-source code available for anyone to run
- Single perceptron layer architecture is simple
- Dense reward signals enable faster learning
- Full map visibility improves pathing decisions
- Can switch Pokémon order before battles
- Hand-engineered features provide Pokémon knowledge
- 1386-dimensional sparse vector input efficient
- Shared trunk with separate policy/value heads
- Type coverage hints help gym strategy
- Evolution-ceiling score for Magikarp-like mons
- 6 fixed-width option slots unify all decisions
- Value Network discarded after training converges
Thiagolira cons
- Only 9% win rate still means 91% failures
- Most runs die at Misty due to RNG
- Heavy RNG element limits optimal play success
- Requires hand-engineered helper features
- Not generalizable to other Pokémon games
- Value Network needed only during training
- Action head only 6-wide limits complexity
- Initially barely better than random chance
Frequently asked questions about Thiagolira
What game does this RL agent play?
The agent plays poke(rogue)like, a fan-made Pokémon game available at https://pokelike.xyz/. It combines traditional Pokémon gameplay with roguelike mechanics, featuring 9 maps with Gym Leaders and a final Elite Four + Lance gauntlet.
What reinforcement learning algorithm is used?
The agent uses PPO (Proximal Policy Gradient), the same algorithm used to post-train LLMs until better alternatives were invented. PPO trains two networks: a Policy Network and Value Network that are trained in tandem.
How big is the neural network model?
The winning model has only 178K parameters, which is embarrassingly small for today's standards. It consists of basically a single perceptron layer to process inputs with separate output heads for policy and value.
What is the input vector dimension and what does it encode?
The input is a 1386-dimensional sparse Float64 vector encoding: current map & badges, the 6 Pokémon in team (level, HP, types, stats, evolution potential), held items, the 6 options on offer this step, upcoming gym's type, and gym coverage hints.
What win rate does the agent achieve?
The best model beats the game almost 9% of all runs across 1000 total runs. Most runs die at Misty, and the author suspects even with optimal play that would be the case due to the huge RNG element in the game.
Why are hand-engineered features important?
Hand-engineered features give the model Pokémon knowledge for_free_, including type coverage of the team, upcoming gym type, offensive effectiveness, defensive vulnerability, and evolution-ceiling scores. Without these, the small network would need to learn mechanics from scratch.
What is the 'slot trick' for action selection?
Every decision (picking map node, lead Pokémon, or item) is squeezed into the same 6 fixed-width option slots. This means the network has one 'choose slot i' output head regardless of what it's actually choosing, making policy logits always ℝ⁶.
Why is dense reward important?
The reward pays out a little for every badge and every won fight because a reward that only fires when you win the whole game is far too sparse for the agent to ever learn what a good move looks like.
Can the Value Network be used after training?
No, the Value Network is only used during training as the critic that tells the Policy Network whether an action turned out better or worse than expected. Once training converges, you can throw the VN away and just let the PN play.
Where can I get the code to run this myself?
All code is open-source and available in the repo mentioned at the end of the blog post. The author states 'You can check out the repo and run it for yourself here!' allowing anyone to experiment with the RL agent.