[Nexus] Human-AI Teaming: Multi-Robot Collaboration

​Join us for a session with Prof.Boyuan Chen from Duke University on his latest research on multi-agent collaboration in robotics!

​About Prof.Chen

​http://boyuanchen.com/

​I am an Assistant Professor at Duke University where I lead the General Robotics Lab. I obtained my Ph.D. at Columbia Univesity with Hod Lipson. My research interests span across Robotics, Perception, Machine Learning, Human-AI Teaming, AI for Science, and Dynamical Systems.

​I am interested in developing "generalist robots" that learn, act and improve by perceiving and interacting with the complex and dynamic world. Ultimately, I hope that robots and machines can equip with high-level cognitive skills to assist people and unleash human creativity.

​About the Talk

• ​Intended Outcome: ...

• ​Agenda: ...

​You should come if you are

• ​Research in robotics and agent systems

• ​Developer who loves to tinker and build projects

• ​Interested in the capabilities of state-of-the-art robotic systems

​Now, about this multi-agent collaboration...

​Imagine guiding a team of robots with just ONE person! Our latest work, HUMAC, shows how single-human guidance can unlock powerful multi-agent collaboration in tasks like hide-and-seek with autonomous robots. 🤖💡

The challenge: Collaboration is a fundamental cognitive capabilities that humans have but remain very difficult to teach AI agents. What does collaboration mean? How can we write down a reward function? Multi-agent RL typically relies on billions of interactions and hopes collaborative behaviors will emerge. We don't know when and even whether it will emerge. Multi-agent imitation learning requires a team of human experts in the first place, which is difficult and costly.

Our solution: We developed a framework where one human dynamically controls multiple robots to learn collaborative behaviors. Our method increases success rates by 58% in simulations and real-world experiments!

Key idea: Humans have strong Theory of Mind ability. We can imagine teaching a team of agents to collaborative by "seating in" different agents. We leverage this unique human capability to enable a single human to guide a team of agents to collaborate. We blend Theory of Mind and Imitation Learning. Robots not only learn from human control but also predict both their teammates’ actions and opponents' actions, allowing them to strategize and collaborate on-the-fly! We provide a set of methods to learn effective policy representations.

Impact: Our method shows a single human can effectively guide a team of robots. This is a very important topic for Human-AI teaming and robot swarms.

As always, we fully open sourced our project. Part of the framework is leveraging our open-sourced Human-AI teaming infrastructure, CREW (https://lnkd.in/eQJDUrdd). HUMAC is done with a great team of students and collaborators from Duke University and Columbia University: Zhengran Ji, Lingyu (Michael) Zhang, Paul Sajda, and myself.

Video: https://lnkd.in/e9gurfVD
Website: https://lnkd.in/eWXZRnV8
Paper: https://lnkd.in/eN8jAhMW
Code including physical robot experiments: https://lnkd.in/ewDysB-V

​About Nexus Roundtable

​Exploring AI Landscape through the lens of academia - Nexus Roundtable series brings together brilliant minds from across industry and academia to break echo chambers and faciliate thought-provoking discussions.

​See our newsletter (https://nickgu.substack.com/) and our programs (https://pioneeringminds.ai/programs)