Cover Image for Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification

Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification

Hosted by Peter
 
 
Zoom
Registration
Past Event
Welcome! To join the event, please register below.
You will be asked to verify token ownership with your wallet.
About Event

​In deep learning and computer vision, it is common for data to present certain. As we begin deploying machine learning models in consequential settings like medical diagnostics or self-driving vehicles, knowing a model's accuracy is not enough. We need a way of quantifying an algorithm's uncertainty for a particular test-time instance while rigorously guaranteeing that consequential errors don't happen too frequently (for example, that the car doesn't hit a human). I'll be discussing how to generate rigorous, finite-sample confidence intervals for any prediction task, any model, and any dataset, for free. This will be a chalk talk where I begin with a short tutorial on a method called conformal prediction and tease a more flexible method that works for a larger class of prediction problems including those with high-dimensional, structured outputs (e.g. instance segmentation, multiclass or hierarchical classification, protein folding, and so on).

​Lecture slides: https://drive.google.com/file/d/1Qm3WSNzywRdDLBaJTkurPgwzZIZDGv8f/view?usp=sharing

​
The presentation is based on the speaker's papers:

  1. ​A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification
    http://people.eecs.berkeley.edu/~angelopoulos/blog/posts/gentle-intro/

  2. ​Uncertainty Sets for Image Classifiers using Conformal Prediction
    https://arxiv.org/abs/2009.14193
    https://github.com/aangelopoulos/conformal_classification

  3. ​Distribution-Free, Risk-Controlling Prediction Sets
    https://arxiv.org/abs/2101.02703
    https://github.com/aangelopoulos/rcps

  4. ​Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control
    https://arxiv.org/abs/2110.01052
    https://github.com/aangelopoulos/ltt

​

​Presenter Bio:

​Anastasios Nikolas Angelopoulos, a a third-year Ph.D. student at the University of California, Berkeley, advised by Michael I. Jordan and Jitendra Malik. From 2016 to 2019, he was an electrical engineering student at Stanford University advised by Gordon Wetzstein and Stephen P. Boyd.

​His homepage: http://people.eecs.berkeley.edu/~angelopoulos/

​------------
Find us at:

Hosts
Peter