Big Data

Brown University - Economics 1660

The spread of information technology has lead to the generation of vast amounts of data on human behavior. This course explores ways to use this data to better understand the societies in which we live. The course weaves together methods from machine learning (OLS, LASSO, trees) and economics (reduced form causal inference, economic theory, structural modeling) to work on real world problems. We will use these problems as a backdrop to weigh the importance of causality, precision, and computational efficiency.

Syllabus

Preparation

Problem Set 0: Scientific Programming in Python,     and Supporting Files

Class 1: Introduction and Clustering

The promise of 'big data', and the theme of this course: what do you need to add to data in order to learn from it?

Problem Set 1: Clustering,     and Supporting Files

References: Hastie, Tibshirani, and Friedman section 14.7

Class 2: Visualization

A bottleneck in understanding and communicating complex ideas is their representation. How can effective visual representations reveal relationships in data?

Problem Set 2: Visualization,     and Supporting Files

References: Edward Tufte (Envisioning Information, Visual Explanations, The Visual Display of Quantitative Information), and Hadley Wickham (ggplot2)

Class 3: Trees

While our visual system can pick out patterns in low dimensional data, it can be less effective with high dimensional data. How can algorithms help us identify patterns in data?

Problem Set 3: Trees,     and Supporting Files

References: Hastie, Tibshirani, and Friedman section 9.2

Class 4: Measurement

Increasingly, everyday behaviors are captured through our interactions with information technologies. What behaviors can we measure?

Problem Set 4: Measurement and Fit,     and Supporting Files (also requires datasets generated by PS0)

Handout: Example Data Sources

Class 5: Fit

How should we weigh detail against confidence and comprehensibility?

Problem Set 5: Generalizing Fit,     and Supporting Files (also requires datasets generated by PS0)

References: Hastie, Tibshirani, and Friedman section 7

Class 6: Linear Models and Regularization

What are the benefits of different representations? How can we control fit in an elegant manner?

Problem Set 6: Regularization and Loss Functions,     and Supporting Files (for both PS6 and PS7; also requires datasets generated by PS0)

References: Hastie, Tibshirani, and Friedman section 3

Class 7: Structure and Causality

How can we decide what actions to take based on relationships we find in data? How can formal thinking help?

Problem Set 7: Structural Models and Identification,     and Supporting Files (for both PS6 and PS7)

References: Pearl, Glymour, and Jewell chapters 3 and 4

Class 8: Structure and Strategic Interaction

What if agents react to our actions--and to each other's reactions? How can formal modeling help predict the results of major changes?

Problem Set 8: Optimal Pricing Competition

Final Project

Final Project Description

Thanks to Simon Freyaldenhoven, Nicholas Hartmann, and Burak Karaca for excellent research assistance.