Stanford CS329H: Machine Learning from Human Preferences | Autumn 2024 | Mechanism Design

This lesson builds a solid foundation in basic machine learning by focusing on the simplest and most essential algorithms. It starts by clarifying what a machine learning system is: data as input-output pairs (X and Y), a model f(x;θ) with parameters θ, a loss function to measure how wrong predictions are, and an optimizer to tune the parameters to reduce the loss. The lecture quickly reviews the three main learning settings—supervised learning (with labeled targets), unsupervised learning (with no labels), and reinforcement learning (with rewards)—and then dives deeply into supervised learning, with short but clear coverage of unsupervised learning at the end.

Within supervised learning, two problem types are highlighted: regression and classification. Regression predicts continuous values (like house prices), and the simplest approach is linear regression, which fits a straight line to data when there is one input, or a flat plane/linear function when there are multiple inputs (features). The model is f(x; w, b) = w^T x + b. To learn the parameters w and b, you minimize mean squared error (MSE), which averages the squares of the prediction errors. The optimizer of choice here is gradient descent, an iterative method that updates parameters in steps proportional to how much the loss changes with respect to them (their gradients). An example with two data points, (1,1) and (2,2), demonstrates how gradient descent starts with w=0, b=0 and progressively adjusts them until the loss stops decreasing—this is called convergence.

The lecture also answers a common question: what if the relationship isn’t a straight line? Linear regression won’t fit well if the pattern is curved or complex. In those cases, we can use nonlinear models such as polynomial regression, decision trees, or neural networks. The point is to understand linear regression thoroughly as a baseline, because it’s interpretable, fast to run, and a building block for more advanced methods.

For classification, the lesson introduces logistic regression. The idea is similar to linear regression, but instead of directly predicting a number, the model produces a probability by passing a linear score through the sigmoid function σ(z)=1/(1+e^(−z)). This gives outputs between 0 and 1 and allows easy thresholding (e.g., classify as 1 if probability ≥ 0.5). The training objective switches to cross-entropy (also called negative log-likelihood), which heavily penalizes confident wrong predictions and encourages correct high-confidence ones. The gradient-based updates look similar to linear regression but use the predicted probabilities in the derivatives. The instructor uses a tiny dataset, (1,0) and (2,1), to show how learning proceeds from initial guesses to sensible probabilities that separate the classes.

Finally, the lecture introduces unsupervised learning through k-means clustering, which finds structure in data without labels by grouping points into k clusters. K-means alternates between two steps: assignment (send each point to its nearest centroid) and update (move each centroid to the mean of its assigned points). The objective is to minimize the within-cluster sum of squares (WCSS): the total squared distances from points to their centroids. Choosing k is important: the elbow method looks for a bend in the WCSS curve where adding more clusters yields diminishing returns, and the silhouette score compares within-cluster tightness to separation from other clusters, preferring values closer to 1. A 2D point cloud example with k=2 walks through random initialization, assignment, update, and convergence.

By the end, you will understand how to define problems as regression or classification, select linear or logistic regression as strong baselines, compute losses (MSE and cross-entropy), perform gradient descent updates, and run k-means to discover patterns without labels. You’ll also know how to think about hyperparameters like learning rate and k, plus when to consider moving beyond linear models. This knowledge sets the stage for more advanced algorithms like decision trees and neural networks, which build on the same ideas of models, losses, and optimization.