Stanford CS230 | Autumn 2025 | Lecture 4: Adversarial Robustness and Generative Models

This lecture lays the foundation for understanding what machine learning (ML) is, why it is useful, and how its main branches differ. It begins with a simple motivation: many real-world tasks are too complicated to encode with explicit, hand-written rules. For example, in image recognition, trying to program "look for pointy ears, whiskers, and a tail" quickly breaks down because there are endless exceptions: lighting changes, weird angles, partial views, and look-alike objects. Machine learning avoids rigid rule lists by learning directly from data—thousands of examples labeled as “cat” or “not cat”—so the computer itself discovers reliable patterns. This allows ML models to detect subtle features humans may never think to write down, such as tiny texture cues or certain color combinations.

The lecture offers two helpful definitions of ML. First, ML is the field that gives computers the ability to learn without being explicitly programmed. Second, ML is the process of building models from data. A model is a compact representation of how inputs (x) relate to outputs (y). Once trained, a model can predict outputs for new inputs, which is what makes ML useful in real applications.

The talk then categorizes the three primary types of ML: supervised learning, unsupervised learning, and reinforcement learning. Supervised learning trains on labeled data where each example comes with the correct answer, such as "this image is a cat" or "this email is spam." It aims to learn a function f that maps input x to output y. Two common supervised tasks are regression—predicting a continuous number like house price—and classification—predicting a class like dog vs. cat. Unsupervised learning has no labels and tries to find structure, such as grouping customers by similar purchasing behavior (clustering) or simplifying high-dimensional data into fewer variables while keeping important information (dimensionality reduction). Reinforcement learning is different: an agent interacts with an environment by taking actions and receives rewards or penalties as feedback, like learning to play chess or Go by trying moves and learning which strategies pay off.

A central practical message is the importance of splitting data into a training set and a test set. Training teaches the model the patterns in the data. The test set, which the model never sees during training, measures how well the model generalizes to brand-new examples. If you only evaluate on training data, you might think your model is great when it has actually just memorized answers, a failure called overfitting. The lecture uses a school analogy: memorizing practice answers instead of understanding the material works on the practice test but fails on the real exam. The model should understand the underlying patterns, not just memorize the training set.

To ground this, the lecture presents a simple regression example: predicting house price from square footage using linear regression. The model assumes a straight-line relationship, f(x) = ax + b, and training finds the a and b that best fit the data. Then, to price a new house, you plug in its size to get the prediction. This example captures the flow of supervised learning: define inputs and outputs, choose a model form, train its parameters on labeled data, and test how well it predicts new cases.

Finally, the lecture visits the common problem of overfitting and how to address it. Overfitting means the model learned training examples too perfectly and fails on new ones. Three straightforward remedies are suggested: use a simpler model, gather more data, and apply regularization (a technique that discourages overly complex solutions). The session closes by acknowledging there are many regression methods beyond linear regression—like polynomial regression, support vector regression, and decision tree regression—which help handle non-linear or complex data. Throughout, the main theme is learning from data to make predictions that hold up in the real world, with careful evaluation to ensure models truly generalize.

This lecture is suitable for beginners. You only need a basic comfort with the idea of inputs, outputs, and the concept of a function. After completing it, you will understand the key categories of ML, the idea of training versus testing, the meaning of overfitting, and how a simple model like linear regression works. You will be able to explain why ML is useful, describe the difference between regression and classification, and articulate why evaluating on a test set is essential. The structure is straightforward: start with the motivation for ML, define ML and models, outline the three types of learning, dive deeper into supervised learning with a concrete regression example, and finish with overfitting and practical remedies, plus a preview of other regression options.