Experimentation + Causal Inference Debate
"All useful analysis implies something causal."
This quote from Erik Gregory captures why this debate is so important. Data Scientists don't do analysis just for research; we do it to drive action. And only causal analysis can confidently tell us that doing X is better than doing Y.
This leads to a high-stakes dilemma every data leader faces: Do we wait for the ground truth from an A/B test, or do we move faster with an observational study? This choice isn't academic—it's the multi-billion dollar question our guests from Amazon and Meta have had to answer at a scale few can imagine.
Tonight, we’re cutting through the noise. We're not here for platitudes. We're here for a direct debate on the real-world friction, the hidden costs, and the breaking points of both approaches. Get ready for the war stories and frameworks you can quote at work on Monday morning.
Cast
Chris Khawand – The godfather of observational causal inference at Amazon. His model allocates billions of dollars.
John Meakin – One of the most influential data scientists at Meta, known for scaling experimentation analysis and driving mega-million incremental values.
Svet Semov – The only data scientist who has worked on both Amazon’s and Meta’s central experimentation teams.
Moderator: Yuzheng Sun, Statsig