Causal by Design
Background
Two research methods are being used with increasing frequency: causal inference and target trial emulation. There are many complex situations where a formal causal calculus such as that developed by Judea Pearl is needed to allow one to infer than an effect is caused by a specific variable such as the use of a treatment or exposure to a specific agent. Practitioners of causal calculus propose that it is a necessary ingredient of virtually every causal inference including that from tightly controlled randomized studies.
Practitioners of target trial emulation claim TTE to be a valuable component of observational treatment components, even though TTE per se does not address the all-important issue of confounding, especially confounding by indication, where one treatment is selected over another due to patient characteristics such as disease severity or drug tolerability. TTE creates good thinking about the inception cohort, time zero, and time-dependent covariates, but does not emulate randomization, blinding of measurements to treatments in effect, or provide any additional help with confounding. The emulation in TTE is a misnomer. I wish that it was called BPOTC (Best Practices in Observational Treatment Comparisons).
TTE is already being used in a haphazard fashion in articles in medical journals, with authors not bothering even to address the potential for confounding and some authors being so naive as to merely state “we assume that all confounders were part of data collection”. Observational treatment comparisons would certainly be simple were that the case!
Study Design
Before delving more into issues introduced above, let’s consider, for our purposes, the existence of four types of studies.
- Randomized controlled experiments in which there is no interruption in the flow in the data generation process and there is no auxiliary time-dependent covariate of interest.
- Randomized experiments in which there is an interruption in the flow, or one is interested in answering a “what if” question related to a time-dependent (post-randomization) covariate.
- Prospective observational studies with very intentional protocolized data collection.
- Non-prospective, non-protocolized observational studies.
In design 1, the response variable is measured on every subject and there are no “what if” questions such as “what would the difference in response between the drug groups have been had all of the animals in the experiment been excellent metabolizers of the new drug?”. An example of design 2 is a randomized trial of an anti-hypertension drug. The usual intent-to-treat (ITT) analysis contrasts subjects assigned to take drug A to those assigned to take drug B, so the inference pertains to what happens when non-adherence to the assigned treatment is averaged out. A non-ITT analysis may ask “but what if everyone adhered to the drug?”. To infer that faithfully taking the drug causes a reduction in blood pressure requires complex thinking.
Design 3 is the only observational study design for which one can hope to make a causal inference about an exposure effect. In this design, one addresses confounding well in advance of starting data collection. This is done by assembling a good many experts in the decision process used to select therapies in the field. The list of possible selection factors is elicited form the experts and data collection is designed to ensure accurate collection of these variables, with minimal missingness. The need for observational studies that are actually designed is not even mentioned in some review articles about TTE.
Design 4 is hopeless except in very simple situations that are amazingly well understood and where one can have confidence in the quality and unbiasedness of the data.
Implications of Study Design on Causal Inference
In design 1 above, there is no explanation for the result other than the experimental manipulation and random variation. Statistical analysis takes care of quantifying uncertainty in estimated treatment effects taking random variation into account. One can even use a randomization test to almost nonparametrically test for treatment effect). Causal calculus is elegant and some researchers may prefer to include a DAG in the paper, but this is not strictly necessary for the primary analysis. One can create a DAG for each experimental design and use the same DAG for all implementations of that design, across many studies.
Design 2 is the perfect spot for causal inference. Asking “what if” questions such as “what if everyone adhered to their assigned drug” are perfect problems for application of causal calculus. The simplest causal inference, which applies to a subject of the problems where patients either always adhere or never adhere to the assigned drug, uses instrumental variables analysis. An instrumental variable must affect the dependent variable only through the treatment, have no direct effect on the outcome itself, and to be independent of unmeasured confounders. The randomized treatment assignment is the perfect instrument.
Observational designs 3 and 4 are in great need to formal causal inference. Design 4 usually remains hopeless no matter how elegant the causal calculus that is used. Even design 3 can easily fail from unmeasured confounders.
Summary
Causal calculus such as the system developed by Pearl is elegant and can provide a great deal of insight for complex research questions. But straight-up randomized experiments are causal by design. Observational studies should be actually prospectively designed so that causal inference has a chance.