Inference Algorithms for Language Modeling

Catalog description

As the use of massive and costly-to-train large language models has become increasingly commonplace, much academic and industry interest has focused on the methods used to generate outputs from these models - the inference process. Inference-time algorithms can be applied on top of an already-trained model to improve generation quality, lower latency, or induce additional controllability. Inference-time algorithms can allow users to run models on their laptop, serve millions of outputs at scale, or dramatically increase the quality of generations from a system without additional training. In this class, we survey the wide space of inference-time techniques with a particular focus on the implementation and practical use cases of such methods. Students will understand the different ways to implement and compare inference-time techniques, learn the theory behind different strategies for inference-time scaling of compute, and implement representative examples from several classes of inference-time algorithms. In the final project, students will apply inference-time strategies of their choice to two shared tasks: an open-ended generation task and a reasoning task. Prerequisites: 10-601 or 10-401 or 10-701 or 11-411 or 11-611 or 11-711 or 11-785 or 10-715 or 11-667