Building Production-Grade AI Systems at MasterClass
Join Aman Gupta (MasterClass) and Hugo Bowne-Anderson for a fireside chat on building and evaluating production-grade LLM systems. Drawing from the MasterClass team’s experience developing interactive, voice-based AI products with branded instructor personas, we’ll explore how post-training, evaluation, and infrastructure choices shape the next generation of applied AI.
Key Topics of Discussion
Building Production-Grade LLM Systems at MasterClass: How ML and AI became one of the highest-leverage functions for enhancing the experience for MasterClass—and why they chose to build in-house ML systems instead of just using off-the-shelf APIs.
From Prototype to Production Reality: Moving beyond basic LLM integrations to build robust, scalable ML systems that serve millions of learners.
Post-Training Excellence: What "post-training" really means in practice and why it's become a critical differentiator to go from AI wrapper to an AI-native product.
Infrastructure Wisdom: How tools like Metaflow can power sophisticated post-training pipelines that adapt as AI technology evolves.
The Hard Problems: Evaluating AI systems for real product impact—connecting technical metrics to business outcomes like user engagement and retention.
Safety & Alignment: Practical approaches to building AI systems that align with product and user goals, including data patterns that help with alignment training.
LLMs writing prompts: How to use AI to improve AI - using automated prompt engineering.
The Future of ML Systems: What's next in post-training, evaluation methodologies, and building differentiated AI products.
This session is ideal for ML engineers, AI product teams, and technical leaders facing the challenge of moving AI systems from experiments to production. Whether you're dealing with model evaluation challenges, infrastructure decisions, or aligning AI with business goals, you'll gain practical insights from real-world implementation.
Mark your calendars for May 27th and join us live!