Ins and Outs of Building a Multi-Field Multimodal Clip Search
The Data Phoenix team invites you to our upcoming webinar, which will take place on June 20th at 10 a.m. PT.
Topic: Ins and Outs of Building a Multi-Field Multimodal Clip Search
Speakers: Sauptik Dhar, PhD (Director of AI at Eluvio)
Participation: free (but you’ll be required to register)
At Eluvio we are working towards the vision for a Content Fabric - a fully decentralized platform for video and commerce aimed towards serving the world’s internet video. The Eluvio Content Fabric provides an innovative distributed and decentralized video processing framework with just- in-time and personalized experiences, made possible through our state-of-the-art real-time content routing and just-in-time code execution. To enable actionable insights into the ingested multimedia (video, audio, images) in the content-fabric, our Eluvio ML stack provides automated content tagging including - video content captioning, speech to text, brand/logo detection, celebrity detection etc., and search capabilities on top of it. The AI search generated clips can be embedded and/or downloaded for further consumption. However, providing search as a service necessitates a good understanding of the user’s intent, context and a detailed understanding of the video content. Failure to capture the semantic information / similarity between the user’s query and the video’s context can lead to undesirable results.
In this talk I will cover Eluvio’s search capabilities on the ingested video content. Starting from the basics of text search (previously available in the Search V1.0 of the content fabric), I will discuss the limitations of such text-based approaches. Next, I will discuss how such caveats can be addressed through modern vector embedding based semantic search, and introduce the latest AI Clip Search V2.0 released in the Eluvio Content Fabric and Application Suite (Casablanca release) which won the NAB 2024 Product of the Year Award. I will discuss the advantages of the vector search over the text-based approach and discuss the challenges in enabling a vector search. Finally, I will end the talk discussing the next generation search, recommendation and ad capabilities the Eluvio AI Research team is working on.
Speaker
Sauptik Dhar is the Director of AI at Eluvio. Currently he leads the AI initiatives within Eluvio including, Gen AI, Clip Search, Recommendation, Ad insertion etc. Prior to that he was the AI Group Head at Zeku (Oppo) leading the AI for Wireless (Wifi) initiative. Before Zeku, he served as the AI Lead at LG SVL AI Team. There he was leading global AI initiatives including 1) AI driven Predictive Diagnostics, 2) Research ‘on-device AI’ 3) University collaborations with UofT on AutoML. He also worked as an AI Tech Lead at Bosch on several domains like, Healthcare, Thermotechnology, Manufacturing, Automotive etc.
Dr. Dhar received his PhD in Electrical and Computer Engineering from University of Minnesota, Twin Cities. He received his B.Tech from National Institute of Technology, Silchar in Electronics and Telecommunications. He served as an Associate Editor for the prestigious journal Neural Processing Letters (2016 - 2021), and has also served as a reviewer for several journals like, Neural Networks, Pattern Recognition, Neurocomputing, Plos One, IEEE Systems Man. and Cybernetics and others. He has also served as a PC member of renowned conferences like, KDD, NeuRIPS, AAAI, ICMLA, SDM, IJCNN and acted as judge to several awards including Shorty Awards 2024. He actively publishes in the community.