Building a LLM Testing API - ft. Denys Linkov

​LLM testing is an emerging area of development, with many startups focused on metric tracking and customized metrics. In this talk, we'll discuss the tradeoffs of SDK vs API driven workflows, and how we reduced scope with the types of testing we were working through for a conversational AI use case.

​Denys Linkov (ML Lead @ Voiceflow)

​Denys is the ML lead at Voiceflow focused on building the ML platform and data science offerings. His focus is on realtime NLP systems that help Voiceflow’s 60+ enterprise customers build better conversational assistants. Previously he worked at large bank as a senior cloud architect.

​WORKSHOP INFO

​One day full of LLMs!

​Fri Mar 1st, 9am - 5pm ET(times are in ET)

• ​9:00 - Keynote - Large Language Models in Drug Discovery and Healthcare - ft. Ali Madani (Sr. ML Scientist @ Recursion)

• ​10:00 - Preparing for LLMOps at Enterprise Scale - ft. Andy McMahon (Head of MLOps at NatWest Group)

• ​11:00 - How we chose our Vector DB - ft. Suhas Pai (CTO @ Hudson Labs)

• ​12:00 - Sherpa Update - Configuration as an Agent - ft. Percy Chen (PhD Candidate @ McGill University)

• ​13:00 - Large Language Models as a Building Blocks - ft. Jay Alammar (Director & Engineering Fellow @ Cohere)

• ​14:00 - Navigating the intersection between LLMs and products - ft. Penny Ji (AI Product Manager @ LG Toronto AI Lab)

• ​15:00 - Building a LLM Testing API - ft. Denys Linkov (ML Lead @ Voiceflow)

• ​16:00 - Innovations in Efficient Training, Tuning, and Serving Large Foundational Models - ft. Alireza Darbehani (MLOps Engineer @BenchSci)