Quantitative Issues in Cancer Research Working Group Seminar

Daniel Schwartz
Postdoctoral Research Fellow, Department of Biostatistics, Harvard T.H. Chan School of Public Health

Treatment Effect Estimation in Multisite Trials with Endogenous Design: Old Estimators, New Results

Abstract: In large-scale multisite randomized trials, key design features such as the sample size at each site often arise from an unpredictable social process. As a result, the sample sizes of the treated and control groups within each site, which are generally considered fixed, are more appropriately considered random. When the treatment effect also varies across sites it is often plausible that it will be associated with the site-specific sample sizes, for instance when larger sites are more effective. In this case we say that the trial design is endogenous. In this paper we argue that endogenous designs have a major consequence: to realistically evaluate the performance of an estimator under repeated sampling one must now integrate over the design features. This can challenge intuitions about when different estimators are best used. We present a simple model for multisite trials with random, endogenous designs. We then show, with asymptotic and simulation results, how popular treatment effect estimators can perform very differently under endogenous designs than they do under a classical perspective assuming that the design is fixed. We illustrate these ideas in a case study of a major multisite trial in education, showing how Bayesian analyses can explore the likely performance of the common estimators given the study’s data, under a range of prior beliefs about how endogenous the study design is.