Lakehouse Days: Bengaluru
Apache Iceberg Meetup featuring RisingWave
We’re bringing Lakehouse Days back to Bengaluru, in collaboration with RisingWave! Join e6data for an exclusive in-person meetup designed for data engineers, architects, and senior software engineers. We will cover:
🔹 Apache Iceberg™ internals and Optimizations
🔹 Merge-on-read query, serverless compaction and Iceberg table sharing with RisingWave
🔹 Optimizing query performance
🔹 Handling data transfers with Apache Arrow Flight
🔹 Iceberg's integration with GCP
🔹 Real-world case studies from industry pros
Speakers:
Rayees Pasha, CPO, RisingWave Labs
Topic: Streaming-first Approach to Iceberg with RisingWave
The session will provide an overview of the technical challenges of building a new Iceberg Table engine that is purpose-built for streaming workloads. The talk will highlight how RisingWave has built end-to-end key capabilities for Iceberg table management, including Iceberg’s merge-on-read query, Serverless Compaction, and Iceberg table sharing to allow direct queries from other engines. A key feature in this project is the native Iceberg compaction service written in Rust using Apache DataFusion and Apache Iceberg-Rust as foundational components.
Rayees Pasha is the Chief Product Officer at RisingWave Labs, a startup pioneering the development of a Stream Processing Data Platform. Rayees is responsible for Product and GTM strategy. His expertise is in the areas of data management and big data analytics. He has held product management roles, delivering enterprise software in traditional and SaaS environments. Before moving to Product Management, he worked at Hewlett-Packard as a software designer working on different aspects of database management systems.
Ankur Ranjan, Sr Software Engineer, e6data
Topic: Apache Arrow Flight: Reshaping How We Handle Data Transfers
In this talk, we will explore how Apache Arrow Flight overcomes the challenges of traditional protocols like ODBC and JDBC by providing a columnar-native transport that maintains data in its original format throughout the transfer process. Arrow Flight promises to enhance analytical workloads and align perfectly with modern data architectures by eliminating unnecessary conversions and streamlining data transfers. Join us to discover how this innovative approach can substantially improve data processing efficiency.
Ankur Ranjan is a data engineer and blogger passionate about software engineering. He strongly believes that “the cultivation of mind should be the ultimate aim of human existence.” Ankur possesses skills in various technologies, including Apache Spark, Spark Streaming, Kafka, Scala, Python, and cloud services from GCP and AWS, such as BigQuery, DataProc, EMR, and Lambda. He has experience working with Hadoop ecosystems, including MapReduce and Hive.
Sai Vineel Thamishetty, Sr Data Engineer, Walmart
Topic: Apache Iceberg with Google Cloud Platform (GCP)
This talk will explore the exciting developments with Apache Iceberg and its integration with Google Cloud Platform. Iceberg is now allowing users to store tables on Google Cloud Storage, which means we can use GCP’s scalable infrastructure alongside Iceberg’s performance enhancements. Popular data processing engines like Apache Spark and Trino have improved their support for Iceberg, making it easier for us to work with these tables directly in the cloud. There’s also a lot of buzz around improving interoperability with BigQuery, which could facilitate smoother data transfers and queries.
Sai Vineel is a senior data engineer at Walmart. He has expertise in Apache Kafka, NoSQL, GCP, Apache Spark, and many more. He is a Python Expert and an AWS certified developer-advocate.
Mark your calendars for March 22, 2025! We’ll kick things off bright and early at 09:30 AM in Accel Launchpad, Koramangala!