Semantic Chunking for RAG
Chunks are created when natural language (text) from our source documents gets split. Common methods for text splitting include:
By no. of characters, with overlapping sliding window.
By sentence or paragraph.
A hybrid of the two (e.g., recursive character text splitter).
The recursive character text splitter has become the de facto standard for building RAG systems.
Despite the utility of these text splitters, if you have ever built a RAG system, you know that deciding on chunk sizes for your data - even using the most advanced text splitter - is a bit of a black art.
In 2023, the industry asked, "Isn’t it annoying to tune chunk size when building RAG applications?" Early this year, Greg Kamradt asked, “Would it make more sense to take the meaning of our chunks into account?”
And thus, Semantic Chunking entered the stage!
In this event, we’ll learn how the semantic chunking algorithm works! Text is split into sentences that are converted vectors through an embedding model. Similarity is measured between each pair of consecutive sentences. If sentences are too similar, as defined by a threshold, additional chunks are created.
In this way, we can ensure that if any two consecutive sentences are too different from one another, additional chunks can be created. In theory, this will allow us to achieve better results during retrieval within our RAG system.
We do a complete end-to-end demo of the semantic chunking capabilities of a simple RAG system in LangChain, and we will also compare its performance to naive chunking using the primary RAG assessment metrics of context precision, context recall, faithfulness, and answer relevancy.
Finally, we will discuss tradeoffs between performance and cost for semantic chunking and answer the question, “Is it good enough to be the new standard?”
Who should attend the event?
AI Engineers who build and optimize RAG applications.
AI Engineering leaders interested in emerging new standards for RAG applications.
LLM Practitioner who wants to build at the open-source edge with LangChain.
To prepare for this session, watch:
Speakers:
Dr. Greg Loughnane is the Co-Founder & CEO of AI Makerspace, where he is an instructor for their AI Engineering Bootcamp. Since 2021 he has built and led industry-leading Machine Learning education programs. Previously, he worked as an AI product manager, a university professor teaching AI, an AI consultant and startup advisor, and an ML researcher. He loves trail running and is based in Dayton, Ohio.
Chris Alexiuk is the Co-Founder & CTO at AI Makerspace, where he is an instructor for their AI Engineering Bootcamp. Previously, he was a Founding Machine Learning Engineer, Data Scientist, and ML curriculum developer and instructor. He’s a YouTube content creator YouTube who’s motto is “Build, build, build!” He loves Dungeons & Dragons and is based in Toronto, Canada.
Follow AI Makerspace on LinkedIn & YouTube to stay updated with workshops, new courses, and opportunities for corporate training.