LLM Evaluations in Swift: Test, Compare, Swap

​Building apps with LLMs can be challenging because we, as developers, can't predict exactly what each user will see and must guard against factually-incorrect hallucinations that erode trust. To tackle this, we can set up evaluations - quantitative and qualitative tests that measure “correctness” or other desired outcomes of model outputs. By running those evaluation pipelines across multiple LLMs, we can compare metrics like accuracy, latency, and instruction-following to pick the best model for our app. In this workshop, you'll learn concrete techniques in Swift for building reusable, automated evaluation pipelines so you can quickly test and swap out models as your needs evolve.