Facebook’s Causal Inference Group

I found this job ad by accident on Twitter and was surprised to see that Facebook has a causal inference group.

Facebook is seeking a Scientist to join the Experimental Design and Causal Inference group as part of the Core Data Science team. The Experiment Design & Causal Inference group’s mission is to improve the way we run experiments and make decisions both within and outside of Facebook. Candidates apply expertise to solve novel problems, ranging from adaptive contextual experiments, to using machine learning to identify heterogeneous treatment effects, to high-dimensional causal inference with observational data, and to generalizability.

People at Facebook seem to have realized that all the fancy machine learning techniques we constantly hear about these days might not be the right choice for answering many important business and policy questions. The problem is that ML is all about prediction, but more often than not you’re actually interested in manipulation. And these two tasks are not equivalent. Judea Pearl calls this the “difference between seeing and doing”.

Let me give you an example from my own work. Assume that we’re interested in the effect of R&D subsidies (let’s call this variable X) on firm growth (Y) . Standard ML techniques are very good at discerning the statistical relationship between these two variables, which we call P(Y|X). This last expression is the conditional probability of Y given X. Typically, you would find a pattern in the data according to which firms that receive an R&D subsidy will grow faster, compared to those that don’t. Estimating such conditional probabilities, in one form or the other, is nothing new. The thrust of modern ML techniques though—and the reason for the hype—comes from the fact that they are pretty damn good at the prediction task. Also, they deal well with non-standard data, such as pictures and speech, and big data.

But the seeing part is actually not what you’re interested here. Rather, policy makers care about what happens to the economy if they give out R&D grants to a typical firm. Will that action spur firm growth? In causal terminology, we’re interested in P(Y|do(X)), the probability of Y if we set X to a specific value. That’s the doing part, and there is a subtle but important difference between the two. Those firms that we observe to secure a grant can be very different from the rest of the population. Might be that they’re better connected, more innovative, or simply have a great idea for a project that is worth funding. So if we would start out paying R&D subsidies to average firms, we might not see an effect at all, because they might lack all these great characteristics.

These things don’t matter if your goal is to teach a computer to differentiate between pictures of dogs and cats. That is a pure prediction (or seeing) task. The machine learning community is traditionally coming from that angle. But economists and social scientists have a competitive advantage when it comes to the second type of task. Because they are trained to think about policy and managerial implications of their research, and this always involves manipulation, in one form or the other.

Let’s have a look at what Facebook is requiring from their potential recruits for their causal inference group.

  • PhD in statistics, political science, economics, biostatistics, or related field, and a strong passion for applied problems in causal inference. Candidates should be familiar with the potential outcomes framework.
  • At least 2 years of hands-on research experience with an empirical problems in the social or biomedical sciences, or in the Internet industry.
  • Scientists should be proficient in R.
  • Programming: Proficient in Hack, PHP, JavaScript, or Python. Knowledge of SQL, Hive, or Impala.

Alright, so they seem to know that social scientists are well-trained for the job, that’s nice! The rest is more or less a list of typical requirements for a data scientist job these days. To keep their competitive edge and up-to-date, economics and business curricula should probably move towards teaching more R and Python, instead of Stata or SPSS, and equip their graduates with general programming and data infrastructure skills.

Further requirements:

At least one of the following:

  • Experience with field experiments, experimental design, missing data, survey sampling, and/or panel data.
  • Experience with observational causal inference (e.g., regression adjustment, matching, propensity score stratification), or quasi-experimental methods (e.g., instrumental variables, regression discontinuity, interrupted time series). Knowledge of causal graphical models is a plus.
  • Experience with bandit optimization, adaptive experimentation, and/or Gaussian processes.

If you obtained a Masters degree in economics and took some classes in modern microeconometrics, you should be familiar with many of  these techniques. What I found most interesting here is that people at Facebook apparently work with causal graphical models—a state-of-the-art tool in causal inference that, unfortunately, gets taught much less often in social science departments. If you want to learn more about it you can check out this earlier post of mine. I’m very curious to hear what kind of concrete business problems people at Facebook are tackling with causal graphs, because so far I’ve only seen them applied in academic research papers (thankful for any pointer!). But diffusion to a wider audience seems to be happening pretty fast. That receives a like from me: 👍🏼

Advertisements

Becoming More Different Over Time

In my class we recently discussed a paper by Higgins and Rodriguez (2006)—published in the Journal of Financial Economics—that contains an important lesson for researchers who want to apply the difference-in-differences (DiD) method in competition analysis and merger control. Continue reading Becoming More Different Over Time

A plea for simple theories

[This is the second part of a fair copy of a recent Twitter thread of mine. I suggest you have a look at part 1 about nonlinear mediation analysis first. Otherwise, it might be hard to follow this post.]

Understanding causal effects is tough, but understanding causal mechanisms is even tougher. When we try to understand mechanisms we move beyond the question whether a certain causal effect exist, and ask instead how an effect comes about. Continue reading A plea for simple theories

Nonlinear Mediation Analysis

This is a fair copy of a recent Twitter thread of mine. I thought it might be interesting to develop my arguments in a bit more detail and preserve them for later use.

Continue reading Nonlinear Mediation Analysis

Why do less and less people start their own business?

There are not many better things (personal things aside) that can happen to a job market candidate than getting mentioned by Tyler Cowen on Marginal Revolution, one of the most widely read economics blogs on the internet. This happened to Nicholas Kozeniauskas from NYU. His paper got judged to be “one of the more important papers of this job market season” by Tyler. And it has indeed many interesting results to offer. Continue reading Why do less and less people start their own business?

Econometrics and the “not invented here” syndrome: suggestive evidence from the causal graph literature

[This post requires some knowledge of directed acyclic graphs (DAG) and causal inference. Providing an introduction to the topic goes beyond the scope of this blog though. But you can have a look at a recent paper of mine in which I describe this method in more detail.]

Graphical models of causation, most notably associated with the name of computer scientist Judea Pearl, received a lot of pushback from the grandees of econometrics. Heckman had his famous debate with Pearl, arguing that economics looks back on its own tradition of causal inference, going back to Haavelmo, and that we don’t need DAGs. Continue reading Econometrics and the “not invented here” syndrome: suggestive evidence from the causal graph literature