Open Source Data Ingestion for RAGs with dlt

​In this hands-on workshop, we’ll build an end-to-end data pipeline using the REST API as a source. You'll learn how to use dlt to ingest fresh data into database, and how to index it in Cognee for powerful semantic querying.

​We'll also explore challenges in handling evolving relationships between resources and experiment with solutions for capturing those changes more intelligently over time.

​​We'll cover the following steps:

• ​Extract and normalize data from REST APIs using `dlt`

• ​Load structured data into a database

• ​Index and query the data semantically using Cognee

• ​Handle time-sensitive and incremental updates across related resources

​By the end of this session, you'll know how to build an intelligent ingestion pipeline that stays in sync with dynamic data sources and powers semantic applications using open-source tools.

​​This event is sponsored by dlthub.

​​​​​DataTalks.Club is the place to talk about data. Join our slack community!