Computer-Use Agents: Building Computers to Operate Computers

Catalog description

We are entering a world in which software writes itself with the help of AI coding agents. It is not hard to imagine a near future in which AI can also operate software itself. A computer that can operate a computer - a "computer-use agent" (CUA) - would represent the culmination of decades of work in automation and represent one of the most economically impactful applications of AI. This seminar covers the how, what, and why of CUAs. Students will study foundational papers and benchmarks, learn from speakers at leading academic and industry labs, get a deeper understanding of the frontier of model capabilities, and get hands-on experience training and evaluating CUAs. The seminar culminates in a class-wide competition to build the best CUA on a difficult held-out benchmark.