This project investigates how different entropy coefficient decay strategies affect the exploration–exploitation balance in Proximal Policy Optimization (PPO) when training agents in the Super Mario Bros environment.
We compare:
- Progress-based decay – Adapts entropy according to the agent's in-game progress (logistic decay).
- Timestep-based decay – Decreases entropy linearly over total training timesteps.
- Fixed coefficient – Keeps entropy constant throughout training.
Our results show that progress-based decay improves learning speed and final performance compared to the other two strategies.
uv sync --python 3.8
pip install -r requirements.txt- Note that the dependencies in requirements.txt need to be reinstalled every time you run ‘uv sync’.
- Also the version of cuda in pyproject.toml should be adjusted to your own situation