Steven Lehrer, Queen’s University

An Initial Assessment of the Hype for Machine Learning Strategies for Prediction and Causal Effects
Friday, 6 March 2020 - 2:30 pm to 4:00 pm
Location
Contact information
Contact person: 
Louis-Philippe Morin
Email: 
Louis-Philippe.Morin@uottawa.ca
Registration
Registration required: 
No
Cost to attend: 
Free of charge
Event language: 
Intended audience: 

Numerous off the shelf machine learning algorithms have recently been developed to discover heterogeneous treatment effects. In this paper, we examine their performance using both simulated data and data from an active labour market experiment with the LaLonde (1986) design. Our initial results find poor performance that likely arises since many of the algorithms rely on the assumption of homoskedasticity, which in combination with treatment effect heterogeneity leads to substantial misclassification. We propose and evaluate several simple modifications to the algorithms that can either i) capture more heterogeneity in subgroups, or ii) impose common support. While these modifications lead to small gains, we suggest researchers develop classification algorithms that jointly consider heteroskedasticity and balance in observed covariates. At present, we advise that labour economists should be cautions in using these methods in empirical work, particularly in applications where treatment effect heterogeneity is likely.