Measuring the effectiveness of public health interventions

Symposium speakers included, from left: Mary Mwanyika Sando of Muhimbili University in Tanzania; Francesca Dominici of HSPH's biostatistics department; and Jonathan Levy of the Boston University School of Public Health

February 11, 2013 — If you’re examining the impact of air pollution control efforts in Denver, how do you statistically account for the fact that air pollution travels east—and that pollution reduction in the western United States could affect air quality in New England?

Likewise, if you’re measuring the effectiveness of a particular HIV-prevention strategy in a village in Botswana, how do you account for the fact that people in one village may have relationships with people in other villages?

These are two of the thorny issues discussed at an afternoon symposium on February 6, 2013 at Harvard School of Public Health (HSPH) titled “Quantitative Methods of Implementation Science & Translational Research.” A dozen researchers from HSPH and other institutions spoke at the symposium, which was sponsored by the departments of epidemiology, biostatistics, and global health and population at HSPH.

Implementation science and translational research aim to use research findings to create real-world interventions—and to figure out how best to measure the success of those efforts, HSPH Dean [[Julio Frenk]] explained in introductory remarks.

Speakers at the symposium focused on three case studies.

Quantitative methods symposium-students-250Measuring air pollution reduction efforts

Several speakers—including [[Francesca Dominici]], HSPH professor of biostatistics, and [[Joel Schwartz]], HSPH professor of epidemiology—discussed techniques researchers use to quantify the public health benefits of air pollution reduction efforts. Researchers try to measure actual health outcomes as well as estimate what outcomes might occur without pollution reduction efforts. To do so, massive amounts of data from thousands of air pollution monitors all over the United States are studied; the challenge for researchers is that this information can be analyzed in a variety of ways.

For example, because air pollution travels, it’s difficult to tell if steps taken to reduce pollution in a particular region occurred because of efforts within that region. Nevertheless, this is the sort of information statisticians attempt to pinpoint because it can have a big impact on public policy.

Maternal-to-child transmission of HIV/AIDS

A second case study discussed at the conference was about the effectiveness and cost-effectiveness of interventions to prevent maternal-to-child transmission of HIV/AIDS in Tanzania.

The researchers in Tanzania are trying to determine whether greater community outreach can help reduce disease transmission, as well as which of two preventive drug therapies—a single antiretroviral or a combination of three—is the most effective. Although it’s a challenge, they must consider a number a “moving targets” in their evaluations, such as the possibility that some mothers in the study may not follow through on their assigned drug therapy; or the fact that the program’s cost-effectiveness could change over time; or even dubious interpretations of handwritten entries on forms filled out by study participants.

“There are a lot of methodological challenges,” said [[Donna Spiegelman]], HSPH professor of epidemiologic methods.

Effectiveness of HIV/AIDS interventions

Efforts to eliminate HIV/AIDS in Botswana, and to assess the effectiveness of those efforts, provided a third case study. [[Victor De Gruttola]], professor and chair of the Department of Biostatistics at HSPH, described the study, which is focused on evaluating the potential spread of disease in 30 villages in Botswana.

In the study, researchers must take into account the fact that people have relationships both within villages and between villages, De Gruttola explained. This “mixing” is much like the “mixing” of air pollution throughout the atmosphere, he said—it makes statistical analysis difficult but must be accounted for to get the most accurate results. The researchers consider factors such as community characteristics and how long typical relationships last as they try to model a “sexual network” that helps them to estimate future disease incidence.

“The challenge becomes how to generate a collection of networks that are plausible given the information we have, and also reflect the uncertainty we have,” De Gruttola said.

[[Michelle Williams]], professor of public health and chair of the Department of Epidemiology, said that each of the case studies presented common themes, like the importance of using valid information and the challenge of sorting out data that isn’t “contained” in one location but “mixes” to create complicated statistical scenarios.

— Karen Feldscher

photos: Mike Mazzanti