Lecture 10: Gaussian Processes and Kernel Methods

Assumptions, evidence, and uncertainty

Overview

How much can a few measurements tell us about a function? A Gaussian process uses a kernel to describe how function values vary together, then conditions that distribution on observations. The CO₂ examples show how assumptions affect both predictions and uncertainty, and how uncertainty can guide the next measurement.

  • Distributions over functions and Gaussian conditioning
  • Kernels for smoothness, trends, and cycles
  • Kernel composition and hyperparameter fitting
  • Forecast error, interval coverage, and interval width
  • GP-guided measurement selection compared with random sampling

Learning Objectives

By the end of this lecture, you will:

  • Describe a GP through finite Gaussian vectors and distinguish conditioning from hyperparameter fitting.
  • Design kernels for the relationships the problem requires.
  • Evaluate uncertainty without assuming that a nominal 95% interval is well calibrated.
  • Explain when uncertainty-guided sampling can improve measurement efficiency and what the comparison actually tests.

Materials

TipQuick Access

Lecture notes · PDF version

Gaussian Processes & Kernel Methods notebook

Datasets & Acknowledgments

  • Mauna Loa CO₂: the Scripps CO₂ Program record. The notebook uses 86 monthly observations from 2010 through February 2017 in the historical data/archive.csv included with Lecture 10.
  • CIFAR-10: an optional notebook extension compares pixel neighborhoods with frozen ResNet-18 feature neighborhoods; no image GP is trained in that comparison. Dataset source.
  • Libraries: scikit-learn for GPs and kernels; PyTorch/torchvision for the optional image-feature comparison.

Previous: ← Lecture 9: Ensemble Methods | Next: Lecture 11: K-Means Clustering →