LLM as a Judge (UBS)

​"LLM-as-a-Judge" refers to the use of Large Language Models to evaluate and judge the quality of generated content or responses in various applications, including the performance of other AI models. In this hands-on, interactive event, we will explore the practical applications of LLMs. We will demonstrate how to use open-source software to download and run models, integrate them into chatbots, and analyze outputs.

​Topics covered will include prompt engineering, addressing model inaccuracies, and implementing Retrieval-Augmented Generation (RAG) for improved accuracy. We will also demonstrate the use of LLM-as-a-Judge in Chain-of-Thought (CoT) and agent-based solutions, while touching on security and guardrails.

​All materials and code are publicly available, showcasing the current capabilities of LLMs.