LESSONS FROM A YEAR OF BUILDING WITH LLMS

​In this special live-streamed recording of Vanishing Gradients, Hugo speaks with Eugene Yan (Amazon), Bryan Bischof (Hex), Charles Frye (Modal), Hamel Husain (Parlance Labs), Jason Liu (Instructor), and Shreya Shankar (UC Berkeley).

​Over the past year, these six guests have been building real-world applications on top of LLMs. They have identified crucial and often neglected lessons that are essential for developing and building AI products. 

​They have recently written an O’Reilly report based on these learnings and, in this conversation, they’ll share advice and lessons for anyone who wants to build products informed by LLMs, ranging from tactical to operational and strategic. They’ll cover

• ​Prompting and information retrieval/RAG, 

• ​Tuning and optimizing workflows,

• ​Evaluation and monitoring,

• ​Data and working with models,

• ​Product, teams, and roles,

• ​MVPs and the ideas behind “No GPUs before PMF”,

• ​Key aspects of quick iteration,

• ​How and why to start with prompting, evals, and data collection,

​And much more!