Trends in Financial Data Science

Catalog description

This is a full semester course focusing on practical aspects of alternative data, machine learning and data science in quantitative finance. Homework and hands-on projects form an integral part of the course, where students get to explore real-world datasets and software. The course begins with an overview of the field, its technological and mathematical foundations, paying special attention to differences between data science in finance and other industries. We review the software that will be used throughout the course. We examine the basic problems of supervised and unsupervised machine learning, and learn the link between regression and conditioning. Then we deepen our understanding of the main challenge in data science – the curse of dimensionality – as well as the basic trade-off of variance (model parsimony) vs. bias (model flexibility). Demonstrations are given for real world data sets and basic data acquisition techniques such as web scraping and the merging of data sets. As homework each student is assigned to take part in downloading, cleaning, and testing data in a common repository, to be used at later stages in the class. We examine linear and quadratic methods in regression, classification and unsupervised learning. We build a BARRA-style implicit risk-factor model and examine predictive models for county-level real estate, economic and demographic data, and macroeconomic data. We then take a dive into PCA, ICA and clustering methods to develop global macro indicators and estimate stable correlation matrices for equities. In many real-life problems, one needs to do SVD on a matrix with missing values. Common applications include noisy image-recognition and recommendation systems. We discuss the Expectation Maximization algorithm, the L1-regularized Compressed Sensing algorithm, and a naïve gradient search algorithm. The rest of the course focuses on…