Learning with Limited Labeled Data

Catalog description

One of the most remarkable abilities of recent systems created with machine learning is their flexibility. Large pre-trained models (so-called “foundation models”) can be adapted with relative ease to a wide range of tasks. Sometimes this adaptation can happen with no training examples at all. How is this ability achieved? What are its limits? And what should we do when it fails? This seminar course will survey recent research on these topics, including pre-training and transfer learning, instruction tuning, reinforcement learning from human and AI feedback, few-shot and zero-shot learning, weak supervision, and synthetic data generation. Students will lead discussions on recent research papers and develop final research projects.