Performance and Efficiency in Artificial Intelligence Infrastructures

Catalog description

Explores performance, power, energy, latency, and throughput of AI infrastructures with a focus on Deep Neural Networks (DNNs) and Large Language Models (LLMs). Topics include batching, multi-tenancy, quantization, tensor parallelism, speculative decoding, pruning, energy-efficient AI, scheduling, and hardware acceleration. The course includes hands-on assignments, case studies from industry and benchmarking exercises. Prerequisite: a minimum grade of 2.7 in TECE 556; recommended: proficiency in Python; understanding of basic machine learning concepts; and understanding of basic computer architecture and parallel computing.