Reinforcement Learning for Engineers. 3 credits, 3 contact hours

Catalog description

Prerequisites: There are no mandatory prerequisite courses, but students are expected to have a clear understanding of core concepts in probability and linear algebra, as well as programming proficiency in Python. This course provides a mathematically rigorous, engineering-oriented treatment of reinforcement learning (RL) for sequential decision-making in engineering systems. Unlike general machine learning courses focused on supervised learning with i.i.d. data, RL addresses optimization from interaction or logged experience, where actions affect future observations, rewards and constraints - precisely the setting that arises in engineering control, planning and design problems. Foundational topics are taught through an engineering lens, connecting to optimal control, dynamic programming, model-based approach and feedback control theory: Markov decision processes, multi-armed bandits and online learning, Monte Carlo and temporal-difference methods, Q-learning, policy gradient methods (with connections to continuous-time optimization and Linear Quadratic Regulator (LQR), actor-critic methods and continuous control, deep RL and function approximation. The course then advances into frontier topics absent from existing NJIT offerings, each with direct engineering applications: Safe RL (constrained MDPs, risk-sensitive objectives - for safety-critical systems in robotics, transportation and biomedical devices); Offline RL (batch learning from logged data, pessimism under distribution shift - for healthcare, autonomous driving and industrial systems where online interaction is unsafe or expensive); Model-Based RL (world models, robust planning, system identification - for control-theoretic applications); and Inverse RL (reward learning from demonstrations - for human-robot interaction and RLHF). Students complete programming-based, conceptual questions-oriented, and proof-oriented assignments, a written examination and a research-oriented project applying RL to an engineering problem or reproducing and extending a published research result.