Investigating Behavioural Intelligibility In Reinforcement Learning

Abstract

In this dissertation, an investigation is launched into the fundamentals of reinforcement learning, aiming to answer the question: How can first-principles implementations of RL agents enhance our understanding of their behaviour and improve intelligibility in controlled environments? This investigation is addressed by outlining the overarching mathematical concepts, the theoretical agent framework, the structure of the controlled environment, and the intelligibility methods employed. Subsequently, four selected agents-PPO, Q-Learning, REINFORCE, and DDQN-are implemented using transparent, first-principles approaches. Each implementation is described in detail. The agents are then deployed within the environment, trained, and experimentally evaluated with respect to key features of their decision-making and learning behaviour. These experiments demonstrate that, through these investigative techniques, it is possible to observe and confirm theoretical aspects of the agents in a practical and transparent manner, thereby contributing to answering the research question. In conclusion, by uniting the mathematical concepts with the theoretical understanding of the agents and presenting them in a transparent manner, one can train agents in a controlled environment and investigate them using intelligibility techniques. This approach provides significantly improved insight into the behaviour of these agents and lays the groundwork for future tests to be conducted on these fundamental implementations.