Clinical Studies of Vaccine Efficacy

• covid-19
• Vaccination

The study by Bell and colleagues uses linkage between large datasets to analyze severe acute respiratory syndrome coronavirus 2 vaccine efficacy in dialysis and transplant patients.¹ A key finding is a 33% (95% confidence interval, 0%–52%) reduction in infection risk after the second vaccine dose. This estimate is explained in Supplemental Table 1, where the risk of coronavirus disease 2019 (COVID-19) infection is given as 25/235 (11%) in the unvaccinated and 357/5011 (7%) in vaccinated individuals. Real world efficacy of vaccination is an important question, and observational population studies are therefore most welcome, although they face a number of challenges. This study suggests a weaker vaccine effect than has been observed by others,² raising the question of whether this finding might reflect study design, rather than a true difference.

A potential problem in the study design is not considering “time.” Infections are counted per person, rather than per person-day, and vaccination status is treated as fixed, although people transition from unvaccinated to vaccinated throughout the study period. These factors lead to differences in “duration at risk” between vaccinated and unvaccinated groups, as well as a large number of excluded events: of the 814 COVID-19 infections in 2021, 357 were “breakthrough,” so 457 non-breakthrough infections (94 of which were fatal) occurred in individuals who were unvaccinated or had received only a single dose. However, only 25 of these are included in Supplemental Table 1.

The study design also leads to difficulty defining groups without bias. Groups are defined within the “adult kidney replacement therapy population as of 19th September 2021.” Intuitively, to be part of this population, a patient should be alive on that date, so it seems as though the 25 unvaccinated cases are COVID-19 survivors (whereas the vaccinated cases include all 357 breakthrough infections, 33 of which were fatal). Alternatively, the population as of 19th September 2021 may include those who died before that date. In this case the outcome (COVID-19 infection) influences group membership (vaccination status) since those who died of COVID-19 had less time in which to get vaccinated (Figure 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Timeline of study population. Simplified timeline in which each patient is a horizontal line, showing mortality and group membership by vaccination status. COVID-19 cases are shown as colored dots, with colored lines indicating survival postinfection. A per-person analysis based on the final population leads to bias, since only COVID-19 survivors are compared. But inclusion of cases circled in a per-person analysis also leads to bias, since dying from COVID-19 may be the reason for nonvaccination, whereas all the pre-vaccination cases (which are excluded) are immortal up to the time of vaccination.

It would be helpful, therefore, to better understand the definitions and data underpinning the estimate of vaccine efficacy, because it seems unclear what can reliably be concluded from the analysis presented.

• Copyright © 2022 by the American Society of Nephrology