Q-Cogni: An Integrated Causal Reinforcement Learning Framework

Cristiano Da Costa Cunha, Wei Liu, Tim French, Ajmal Mian

Research output: Contribution to journalArticlepeer-review

9 Downloads (Pure)

Abstract

We present Q-Cogni, an algorithmically integrated causal reinforcement learning framework that redesigns Q-Learning to improve the learning process with causal inference. Q-Cogni achieves improved policy quality and learning efficiency with a pre-learned structural causal model of the environment, queried to guide the policy learning process with an understanding of cause-and-effect relationships in a state-action space. By doing so, we not only leverage the sample efficient techniques of reinforcement learning but also enable reasoning about a broader set of policies and bring higher degrees of interpretability to decisions made by the reinforcement learning agent. We apply Q-Cogni on Vehicle Routing Problem (VRP) environments including a real-world dataset of taxis in New York City using the Taxi & Limousine Commission trip record data. We show Q-Cogni's capability to achieve an optimally guaranteed policy (total trip distance) in 76% of the cases when comparing to shortest-path-search methods and outperforming (shorter distances) state-of-the-art reinforcement learning algorithms in 66% of cases. Additionally, since Q-Cogni doesn't require a complete global map, we show that it can start efficiently routing with partial information and improve as more data is collected, such as traffic disruptions and changes in destination, making it ideal for deployment in real-world dynamic settings.

Original languageEnglish
Pages (from-to)6186-6195
Number of pages10
JournalIEEE Transactions on Artificial Intelligence
Volume5
Issue number12
Early online date3 Sept 2024
DOIs
Publication statusPublished - Dec 2024

Fingerprint

Dive into the research topics of 'Q-Cogni: An Integrated Causal Reinforcement Learning Framework'. Together they form a unique fingerprint.

Cite this