An analysis of the impact of learning techniques on NPC Behavior in action games
DOI:
https://doi.org/10.34117/bjdv12n7-049Keywords:
reinforcement learning, imitation learning, AI in games, game development, NPC behavioAbstract
This paper explores how reinforcement learning and imitation learning can be used to shape non-player character (NPC) behavior in action games, with a focus on their impact on player experience. A game environment was developed using the Unity engine and the ML-Agents framework to analyze how different learning methods influence NPC behavior from a design perspective. Three approaches are evaluated under the same conditions: Proximal Policy Optimization (PPO), PPO combined with Behavior Cloning (PPO+BC), and PPO combined with Generative Adversarial Imitation Learning (PPO+GAIL). The study is conducted in a 3D arena where an agent must navigate obstacles, locate a dynamically positioned target, and perform attack actions. The evaluation considers the adaptability of the NPC to the scene and the victory conditions. The experiments are expected to highlight differences in agent behavior, particularly in terms of consistency and the ability to handle unseen situations. The results may indicate the potential of these techniques as tools to support NPC behavior design in action games.
References
Yannakakis, G. N. and Togelius, J. 2018. Artificial Intelligence and Games. Springer, Cham. https://doi.org/10.1007/978-3-31963519-4.
Gonzalez-Santocildes, A. (2024). Unity-Based Autonomous Driving Environment: A Platform for Reinforcement Learning. In: Proceedings of the International Conference on Interactive Simulation and AI. Springer, Cham. https://doi.org/10.1007/978-3-031-74186 9_23
Juliani, A., Berges, V.-P., Teng, E., Cohen, A., Harper, J., Elion, C., Goy, C., Gao, Y., Henry, H., Mattar, M., and Lange, D. 2020. Unity ML-Agents Toolkit. GitHub repository. Available at: https://github.com/Unity-Technologies/ml-agents
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., and Hassabis, D. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529–533. https://doi.org/10.1038/nature14236.
Ho, J. and Ermon, S. 2016. Generative Adversarial Imitation Learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS ’16). Curran Associates Inc., Red Hook, NY, USA, 4565–4573.
Lai, C.-M., Wang, H.-C., Hsieh, P.-C., Wang, Y.-C. F., Chen, M. H., and Sun, S.-H. 2024. Diffusion-Reward Imitation Learning. In Proceedings of the 37th Adversarial Conference Neural Information Processing Systems (NeurIPS ’24).
Wang, X., et al. 2025. Intelligent Games Meeting with Multi Agent Deep Reinforcement Learning: A Comprehensive Review. Artificial Intelligence Review. https://doi.org/10.1007/s10462-025 11166-1.
Yan, Y., et al. 2025. The Analysis of Deep Reinforcement Learning for Dynamic Graphical Games. Scientific Reports. https://doi.org/10.1038/s41598-025-05192-w
Souchleris, K., et al. 2023. Reinforcement Learning in Game Industry—Review, Prospects and Challenges. Applied Sciences 13, 4 (2023), 2443. https://doi.org/10.3390/app13042443
Silva, F., et al. 2022. A Modeling Environment for Reinforcement Learning in Games. Entertainment Computing (2022). https://doi.org/10.1016/j.entcom.2022.100450
Schulman, J., Wolski, F., Dhariwal, P., Radford, A. and Klimov, O. 2017. Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347.
Sutton, R. S. and Barto, A. G. 2018. Reinforcement Learning: An Introduction. 2nd ed. MIT Press, Cambridge, MA, USA.
Mehta, S. A., Ciftci, Y. U., Ramachandran, B., Bansal, S. and Losey, D. P. 2025. Stable-BC: Controlling Covariate Shift with Stable Behavior Cloning. IEEE Robotics and Automation Letters (RA-L) (2025). arXiv:2408.06246.
