Engineering Rational Cooperative AI via Inverse Planning and Probabilistic Programming – Tan Zhi Xuan

​Engineering Rational Cooperative AI via Inverse Planning and Probabilistic Programming

​Tan Zhi Xuan - Incoming Assistant Professor in the National University of Singapore’s Department of Computer Science with a joint appointment at the A*STAR Institute of High Performance Computing (IHPC). PhD in the the MIT Probabilistic Computing Project and Computational Cognitive Science lab.

​How can we build cooperative machines that model and understand human minds — machines that assist us with our goals, coordinate on shared plans, infer the intentions behind our words, and even learn our norms and values? In this talk, I will introduce a scalable Bayesian approach to building such systems via inverse planning and probabilistic programming. By combining online model-based planners and sequential Monte Carlo inference into a single architecture, Sequential Inverse Plan Search (SIPS), we can infer human goals from actions in faster-than-real-time, while scaling to environments with hundreds of possible goals and long planning horizons that have proved intractable for earlier methods. SIPS can additionally make use of large language models (LLMs) as likelihood functions within probabilistic programs, allowing us to build AI assistants and copilots that reliably infer human goals from ambiguous instructions, then provide assistance under uncertainty with much higher success rates than LLMs can on their own. By applying this Bayesian approach in many-agent environments, we are also able to design agents that rapidly learn cooperative social norms from others' behavior, achieving mutually beneficial outcomes with orders of magnitude less data than model-free deep RL. I will conclude by charting out how this research program could deliver a new generation of cooperative AI systems grounded in rational AI engineering, while illuminating the computational foundations of human cooperation and addressing fundamental challenges in building human-aligned AI.

​GS AI seminars

​​​​​​​​The monthly seminar series on Guaranteed Safe AI brings together researchers to advance the field of building AI with high-assurance quantitative safety guarantees.

• ​​​​​Follow the series

• ​​​​​​​​Apply to speak

• ​​​​​​Video recordings

• ​​​​​​Mailing list

• ​​​​​​Series info website

• ​​​​​Feedback

• ​​​​​Donate 🧡