Metaflow Office Hours

​Discuss, showcase, learn, and share feedback with the Metaflow community!

​​In this edition, Nick Schmeller at Lacework will show how his team is supporting Lacework's latest AI-powered products with Metaflow.

​Open source software is foundational to cloud computing, but keeping up with the ever-growing number and severity of security issues in third party products is overwhelming for many organizations. To address this critical need, Lacework FortiCNAPP is building a new capability called Riskwatch which precisely identifies which vulnerabilities actually live in your environment by coupling dynamic analysis via our runtime agent with a backend static analysis.

​Building our backend static analysis is a significant engineering challenge, requiring not only that we run analyses over an enormous number and variety of open source codebases, but also that we store and version both open source artifacts as well as our analyses without slowing development.To meet these requirements, we use Metaflow.

​Our workflow runs typically execute thousands of tasks using Metaflow-native features like dynamic fan-out and artifact storage as well as extensions we have developed to handle memoization and input validation. In this talk, we will discuss our solutions to problems around developer experience, adopting the Metaflow custom decorator framework, and scaling to large workloads.​

​​We meet fortnightly on Tuesdays on Google Meet.