DeepSeek Week
On February 20, DeepSeek - the team that took the world by storm with DeepSeek-R1 - announced they would release five repositories for Open-Source Week.
And they did. We want to cover them for you.
As they said on X:
These humble building blocks in our online service have been documented, deployed and battle-tested in production.
When we covered DeepSeek-R1 a few weeks ago, we were quite impressed with how their work on the viral reasoning model was built on the shoulders of the work they did on DeepSeekMath, specifically on Group Relative Policy Optimization (GRPO). GRPO is a variant of Proximal Policy Optimization (PPO) that enhances reasoning while concurrently optimizing the memory usage of PPO.
Perhaps this was from the team being resource constrained, or perhaps it was just innovation from the “tiny team” at DeepSeek contributing to open-source.
As part of the open-source community, we believe that every line shared becomes collective momentum that accelerates the journey.
Daily unlocks are coming soon. No ivory towers - just pure garage-energy and community-driven innovation.
Here’s what the dropped in the subsequent week
FlashMLA: an efficient MLA decoding kernel for Hopper GPUs, optimized for variable-length sequences and now in production.
DeepEP: the first open-source Expert Parallelism (EP) communication library for MoE model training and inference
DeepGEMM: a library designed for clean and efficient FP8 General Matrix Multiplications (GEMMs) with fine-grained scaling
Optimized Parallelism Strategies, including DualPipe, Expert Parallelism Load Balancer (EPLB), and Profile-Data
Fire-Flyer File System (3FS): A thruster for all DeepSeek data access
And, as a bonus, they also released an entire inference system overview for DeepSeek V3 and DeepSeek-R1. The objectives were simple: higher throughput and lower latency.
We have found the DeepSeek team’s work to be technically impressive and ambitious to date, including the “unified paradigm of RFT, DPO, PPO, and GRPO” and their explanation of the “reasons behind the effectiveness of RL” [Ref] and we’re excited to dive into the details of all of their new open-source drops.
Join us live to dig into the details and get your questions answered, from concepts to code!
📚 You’ll learn:
Even more about what’s going on under the hood of DeepSeek-R1
About each of the new DeepSeek #OpenSourceWeek repos
About important dimensions of LLM optimization teams like DeepSeek must consider
🤓 Who should attend the event:
Aspiring AI Engineers who want to go deeper into how production teams build LLMs
AI Engineering leaders who want to understand how state-of-the-art LLMs are built
Speakers:
Dr. Greg” Loughnane is the Co-Founder & CEO of AI Makerspace, where he is an instructor for their AI Engineering Bootcamp. Since 2021, he has built and led industry-leading Machine Learning education programs. Previously, he worked as an AI product manager, a university professor teaching AI, an AI consultant and startup advisor, and an ML researcher. He loves trail running and is based in Dayton, Ohio.
Chris “The Wiz” Alexiuk is the Co-Founder & CTO at AI Makerspace, where he is an instructor for their AI Engineering Bootcamp. During the day, he is also a Developer Advocate at NVIDIA. Previously, he was a Founding Machine Learning Engineer, Data Scientist, and ML curriculum developer and instructor. He’s a YouTube content creator YouTube who’s motto is “Build, build, build!” He loves Dungeons & Dragons and is based in Toronto, Canada.
Follow AI Makerspace on LinkedIn and YouTube to stay updated about workshops, new courses, and corporate training opportunities.