Developing and Training LLMs From Scratch with Sebastian Raschka

​Sebastian Raschka is a machine learning & AI researcher, programmer, and author. As Staff Research Engineer at Lightning AI, he focuses on the intersection of AI research, software development, and large language models (LLMs).

​How do you build LLMs? How can you use them, both in prototype and production settings? What are the building blocks you need to know about?

​In this live podcast recording, Hugo and Sebastian will tell you everything you need to know about LLMs, but were too afraid to ask: what type of neural network architectures do you need to know about? What are the different steps and ways to train LLMs? What datasets are useful for training? How does model evaluation work? When do you need to pre-train your models? When do you fine-tune vs use RAG vs perform prompt engineering?

​And they’ll do all of the above thinking through not only how to use current state-of-the-art models, but also how to build your own LLMs from scratch! The idea here is not that you’ll need to use an LLM you’ve built from scratch, but that we’ll learn a lot about LLMs and how to use them in the process. They’ll also dive into cutting-edge and popular research techniques, such as low-rank adaptation and direct preference optimization.