Lakehouse Days: Bengaluru
Iceberg Builders—Bengaluru Edition
Calling all lakehouse nerds! e6data is hosting a hands-on meetup in Bengaluru for folks who live and breathe data infrastructure.
What’s on the menu?
Deep-dive: Iceberg internals – snapshots, manifest lists, and why table metadata finally makes sense.
PyIceberg in the real world – a lean OSS path instead of spinning up yet another clustered engine.
Bringing Iceberg to your OLAP, OLTP, and streaming systems – implementation of Iceberg on Clickhouse OSS, Postgresql.
Unlocking near-real-time querying on lakhouse – a thought experiment.
Speakers:
Diptiman Raichaudhari, Staff Developer Advocate, Confluent
Topic: Design an open lakehouse, brick by brick - with Apache Iceberg and PyIceberg.
Summary:
Why Iceberg? Hive‐style directory hacks crumble at PB scale—Iceberg’s snapshots, manifests, and hidden partitioning don’t.
PyIceberg demo: run notebook-level SQL with filter-pushdown; no heavyweight cluster needed.
Quick wins: compaction, manifest pruning, schema evolution
Shivji Kumar Jha, Staff Engineer and Data Platforms Lead, Nutanix
Topic: Hacking Iceberg on Your Existing DBs
Summary:
Iceberg: the emerging common table format for OLTP, OLAP, and streaming systems.
How the Postgresql implementation is being designed
how the Clickhouse OSS implementation is being designed
Aravindh Sridharan, Engineering, e6data
Topic: Unlocking near real-time querying on lakehouse: A thought experiment
Summary:
Today's query engines can query data from a lakehouse, but there is a significant delay in the data ending up in the lakehouse before it can be queried
We are going to explore how we can make the query engine tap into ingestion so that it can be aware of the data before even it is present in the lakehouse
And the challenges that exist today to do so
More speakers will be announced soon. Mark your calendars for May 10, 2025! We’ll kick things off bright and early at 9:00 AM in Nutanix Technologies India Pvt Ltd, Marathahalli!