London AI4Code: "Reflexion: Language Agents with Verbal Reinforcement Learning" with Noah Shinn

​Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challenging for these language agents to quickly and efficiently learn from trial-and-error as traditional reinforcement learning methods require extensive training samples and expensive model fine-tuning. Reflexion is a novel framework created to reinforce language agents not by updating weights, but instead through linguistic feedback. Concretely, Reflexion agents verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials. Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%.

​Paper: https://arxiv.org/abs/2303.11366

​Noah Shinn is an undergraduate student at Northeastern University, with research interests in reinforcement learning and computational quantum chemistry as well as in building intelligent, composable, self-aware programs.
Personal website: https://noahshinn.com/