A curated, link-checked directory of reinforcement learning environments. Find the right env for your project, compare landmark results, and learn RL from the best material.
Every repo link verified live. ● maintained in the last 12 months, ● classic but unmaintained.
Answer four questions, get a shortlist matched to your goal, experience, and compute.
Landmark results that defined the field — the scores every new method gets compared against.
| Environment | Metric | Score | Agent | Year |
|---|
The canonical path through RL, from first principles to modern deep RL.
Quick answers to the questions every RL practitioner asks about environments.
CartPole-v1 (Gymnasium) — balance a pole on a cart with two discrete actions. Then LunarLander-v3 for shaped rewards and a real solve threshold, and MiniGrid for debugging algorithms in minutes on a laptop.
The Arcade Learning Environment (57 Atari games) for pixel-based control, the MuJoCo locomotion suite and DeepMind Control Suite for continuous control, and Procgen's 16 procedurally generated games for generalization.
The Farama Foundation stewards Gymnasium, PettingZoo, MiniGrid and the ALE. The rising wave is GPU-native: NVIDIA Isaac Lab and MuJoCo Playground. Archived or stale: OpenAI Retro, DeepMind Lab, NetHack Learning Environment, SMAC.
Gymnasium is the standard API for single-agent environments. PettingZoo, from the same community, is its multi-agent counterpart — 100+ multi-agent envs behind one unified API.
Match env to goal and compute: classic control for learning on a laptop, Atari/Procgen for benchmarking, MuJoCo or Isaac Lab for robotics with a GPU, PettingZoo or SMAC for multi-agent research. Or use the env picker above — four questions, one shortlist.
New environments, benchmark updates, and ecosystem notes — monthly.
The directory opens with 51 environments across 8 categories, every repo link verified live and maintenance status checked against real commit history. A few ecosystem notes from the research: