Assessing PEPFAR’s Health Spillover Effects Beyond HIV: An Update

A difference-in-difference, quasi-experimental design was used to estimate a “treatment effect” (PEPFAR), based on comparison to a control group (the counterfactual). The difference-in-difference design compares the before and after change in outcomes for the treatment group to the before and after change in outcomes for the comparison group. The outcomes of interest, their definitions and sources are listed in Table 1. Baseline variables are listed in Table 2. The panel data set of 157 low- and middle- income countries used in the prior analysis, covering 1990 to 2018, was updated to include data through 2022 (due to data limitations, the time trend for maternal mortality began in 2000). COVID-related funding was not included. All values were adjusted to constant 2022 dollars.

The PEPFAR group included 90 countries that had received PEPFAR support over the period. The comparison group included 67 low- and middle- income countries that had not received any PEPFAR support or had received minimal PEPFAR support (<$1M over the period or <$.05 per capita) between 2004 and 2022.  Data on PEPFAR spending by country were obtained from the U.S. government’s https://foreignassistance.gov/ database and represent U.S. fiscal year disbursements. Data on mortality rates were obtained from the World Bank’s World Development Indicators (WDI) (https://datatopics.worldbank.org/world-development-indicators/.  Several different model specifications were explored. Each specification controlled for numerous baseline variables, compared to an unadjusted model, variables which may be expected to influence the outcomes of interest and which help make the comparison group more comparable to the PEPFAR group.

Table 3 provides the model specifications tested in the updated analysis. Each model specification produced similar, statistically significant results. All models were also run with and without China and India, the two most populous countries in the world, to assess whether they were influencing the results. In both cases, PEPFAR’s impact was still significant and results were similar. The final reported results are from model specification 3. The pre-intervention period for this model started in 2002. Almost all results were significant at the p<0.001 level; one result was significant at the p<0.01 level and one at the p<0.05 level. Table 4 provides the mean values for baseline outcomes and Tables 5-6 provide model results. The Table 5 difference-in-difference estimates should be interpreted as the unit change (e.g., in number of deaths per 1,000) in the outcome associated with PEPFAR. The Table 6 estimates should be interpreted as the percent change in the outcome, relative to the baseline, associated with PEPFAR.

Despite the strengths of the difference-in-difference design, there are limitations to this approach. While the models adjusted for numerous baseline factors that could be correlated with the outcomes of interest, there may be other, unobservable factors not captured. Similarly, while baseline factors are also intended to adjust for selection bias, and make the PEPFAR and comparison groups more similar, there may be other ways in which comparison countries differed from PEPFAR countries (and factors which influenced which countries received PEPFAR support), which could bias the estimates. Recent, published research, based on the earlier 2004-2018 period, examined several other model specifications as well as double log transformations for each model. The results were similar across all models, adding to the confidence of the analytic approach used here.⁴ Another analysis, under review for publication, conducted additional robustness tests and found consistent results.⁵