Treatment effect heterogeneity

[lcsanford]

I'm interested in exploring treatment effect heterogeneity but am having trouble figuring out how to implement code which gets at this.

An example would be in the 401K IRM section--estimating how the effect changes across age or education of investors.

can someone point me in the right direction for where to start?

Thanks! :)

[PhilippBach]

Hi @lcsanford ,

thanks for your interest in our package!

There is a quickly involving literature on heterogeneous treatment effects. And there are several ways you could estimate heterogeneous treatment effects. I'll start with the easiest one.

1. Interacted regression, based on the PLR class

You could set up everything in a PLR model and interact the treatment variable of interest with the covariates you are interested. In the 401k example, this would correspond to interact the treatment variable on 401k eligibility with the income and education variable and estimate the corresponding coefficients. For valid inference, you could run the multiplier bootstrap and adjust p-values or estimate joint confidence intervals . This is more or less related to the discussion on the estimation of price elasticities in product categories you find here: DoubleML/doubleml-for-py#135 and other examples, like the gender wage gap example and this notebook by Vira Semenova in the 401k example. Unfortunately, these examples are based on manual implementation of the DML estimator, but I think they make clear how to set up the model. Using the interacted PLR model is quite straight forward, but of course imposes some linearity.

2. Conditional average treatment effects (CATEs) / group average treatment effects (GATEs) in the nonparametric IRM model

The second and more code-intense alternative would be to work on the output of an IRM model. The IRM model is nonparametric and hence, allows for full heterogeneity in terms of treatment effects of a binary treatment variable. This makes it possible to estimate conditional average treatment effects and group average treatment effects based on a fitted IRM model (i.e., after calling the fit() method and storing the cross-validated nuisance predictions, i.e, actually calling $fit(store_predictions = TRUE)). The main reference in the double machine learning framework is probably the paper by Semenova and Chernozhukov (2020) which is a pretty general and maybe technical treatment. Vira Semenova has a notebook on the implementation of the CATE estimation online which nicely illustrates the main idea. In the notebook, the "double machine learning part" is done manually, basically you could simply replace the prediction part by using our DoubleML model and export the predictions as described before (which makes it possible to use any learner supported in the mlr3verse) and then use her code afterwards. She even has a manual implementation of the multiplier bootstrap.

For a more general introduction and a very nice literature overview (also outside the double machine learning framework), you could read the paper by Michael Knaus: https://arxiv.org/abs/2003.03191 .

I hope this already helps you a bit. As I said, we are currently working on an implementation of the approach by Semenova and Chernozhukov. It's hard to say, when we will provide a ready-to-use implementation in the package (but I hope it will be soon 😄 ). But I'll add a comment below in case we have it in the development version.

Best,

Philipp

[lcsanford]

Thank you so much! This is super helpful!
Luke