Spectrum: Training Domain-Adapted SLMs
Small Language Models (SLMs) are gaining popularity and the trend towards LMs becoming ever-larger and ever-smaller continues.
SLMs are particularly powerful in settings where the LLM can benefit from specialization in a particular domain. In these cases, efficient pre-training and post-training (i.e., supervised fine-tuning and alignment) are necessary! In this event, we’ll focus on using Spectrum for fine-tuning, although it can also be leveraged for pre-training.
Methods for fine-tuning LMs have undergone rapid evolution. Today, Low-Rank Adaptation (LoRA) and Quantized LoRA (QLoRA) are seen and cited as the industry-leading gold standards.
In this event, we present Spectrum, which leverages a signal-to-noise ratio (SNR) calculated for each layer of the LLM to decide which layers to fine-tune and which to hold frozen. This approach is similar in theory to LoRA in that much of the transformer remains frozen, and only part of it is being trained during fine-tuning. We will discuss the performance vs cost tradeoffs, including potential upsides and downsides of each technique.
We will be joined by experts from Arcee.ai, whose team wrote the paper Spectrum: Targeted Training on Signal to Noise Ratio.
Previously, we’ve seen how Arcee has leveraged simple ideas that were well-implemented programmatically to produce great results with Domain Adapted Language Modeling (DALM) and mergekit. This time, we investigate the concepts and code that underlie Spectrum!
You’ll learn:
How Spectrum can be used to train SLMs more efficiently from pre- to post-training
How the implementation of Spectrum differs from LoRA, and the implications
How to fine-tune a domain-adapted SLM with Spectrum!
Who should attend the event?
GenAI enthusiasts interested in LLM innovation by startups at the open-source LLM edge
Aspiring AI Engineers looking to build and fine-tune domain-specific SLMs
AI Engineering leaders who want to understand tools for leveraging SLMs in production
Speakers
Lucas Atkins is a research engineer at Arcee.ai, where he specializes in alignment. As the primary implementer of Spectrum, Lucas played a crucial role in integrating this technology into Arcee's training pipeline. He oversees the company's Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) training pipelines, constantly pushing the boundaries of open-source post-training techniques. Lucas's work focuses on ensuring that Arcee's methodologies remain closely aligned with cutting-edge closed-source solutions, contributing to the advancement of responsible AI development.
Fernando Fernandes Neto, an AI Research Scientist at Arcee.ai. Blending deep technical expertise with business acumen, he transforms complex data into actionable insights and cutting-edge AI solutions. With a PhD in Complex Systems Engineering and a master's in both Industrial Processes and Financial Engineering, he brings a multidisciplinary approach to solving intricate business and technological challenges.
Dr. Greg” Loughnane is the Co-Founder & CEO of AI Makerspace, where he is an instructor for their AI Engineering Bootcamp. Since 2021 he has built and led industry-leading Machine Learning education programs. Previously, he worked as an AI product manager, a university professor teaching AI, an AI consultant and startup advisor, and an ML researcher. He loves trail running and is based in Dayton, Ohio.
Chris “The Wiz” Alexiuk is the Co-Founder & CTO at AI Makerspace, where he is an instructor for their AI Engineering Bootcamp. During the day, he is also a Developer Advocate at NVIDIA. Previously, he was a Founding Machine Learning Engineer, Data Scientist, and ML curriculum developer and instructor. He’s a YouTube content creator YouTube who’s motto is “Build, build, build!” He loves Dungeons & Dragons and is based in Toronto, Canada.
Follow AI Makerspace on LinkedIn and YouTube to stay updated about workshops, new courses, and corporate training opportunities.