Our analysis employs a risk–benefit framework to evaluate vaccine safety by comparing the expected harm prevented through vaccination against the potential harm caused by vaccine-associated SAEs. We first establish a simple model that calculates the number of severe outcomes expected in vaccinated versus unvaccinated populations under different epidemic scenarios, accounting for vaccine efficacy and baseline disease risk. From this, we derive a safety threshold—the maximum acceptable vaccine risk that still provides net benefit. We then implement this framework probabilistically using Bayesian inference to incorporate uncertainty in both disease and vaccine risk estimates, allowing us to calculate the probability that vaccination provides a net benefit for different risk groups and outcomes.
Quantifying the risks and benefits of vaccines
To understand the risks and benefits of a vaccine, we first estimate the risk of severe outcomes for vaccinated and unvaccinated individuals, given the underlying risk of infection, the risk of severe outcomes following infection, vaccine efficacy and the risk of vaccine-associated SAEs. We assume that the probability of severe outcomes following infection and vaccine-associated SAEs vary by risk group (for example, age). We characterize the underlying risk of infection through four scenarios: a small outbreak (5% attack rate), a large outbreak (30% attack rate), endemic transmission over ten years (2.4% annual infection risk6) and short-term exposure (ten days) of a traveller during a large outbreak.
The number of severe outcomes (S) by vaccination status (v = vaccination, nv = no vaccination) by risk group a is calculated as
$${S}_{a,{\rm{nv}}}={\alpha }_{a}\times {\lambda }_{i}\times N$$
(1)
$${S}_{a,{\rm{v}}}=({\alpha }_{a}\times {\lambda }_{i}\times (1-\nu )+{\delta }_{a})\times N$$
(2)
where i is the epidemiological scenario index, λi is the attack rate for scenario i, αa is the age-specific probability of severe outcome given infection, δa is the age-specific probability of vaccine-linked SAEs, v is the vaccine efficacy against severe outcome, and N is the population size.
The number of severe outcomes averted by vaccination is therefore
$${\rm{Severe}}\,{\rm{outcomes}}\,{{\rm{averted}}}_{a}={S}_{a,\,{\rm{nv}}}-{S}_{a,\,{\rm{v}}}$$
(3)
Vaccine safety threshold
We evaluate a vaccine’s safety by comparing the observed risk of SAEs after vaccination to the expected background risk of severe outcomes given infection without vaccination. Using the same notation as above, to conclude vaccine benefit, we require that the total risk with vaccination does not exceed the risk without vaccination:
$${\alpha }_{a}\times {\lambda }_{i}(1-\nu )+{\delta }_{a}\le {\alpha }_{a}\times {\lambda }_{i}$$
(4)
Rearranging, we obtain the maximum acceptable vaccine risk probability δmax:
$${\delta }_{\max }\le {\alpha }_{a}\times {\lambda }_{i}\times \nu$$
(5)
For vaccination to provide net benefit, we require δa ≤ δmax.
Probabilistic framework that benefits outweigh risks
To account for uncertainty in the infection and vaccine risk estimates, we adopt a Bayesian approach that treats both infection and vaccine risks as random variables. Specifically, we calculate the probability that the vaccine benefit threshold (\({\delta }_{max}={\alpha }_{a}\times {\lambda }_{i}\times \nu\)) exceeds the actual vaccine risk (δa).
For each adverse outcome (vaccine or infection) we use conjugate Bayesian inference. We assume non-informative uniform priors for the risks of severe outcomes given infection and vaccination:
$${\alpha }_{a} \sim \mathrm{Beta}(1,\,1)$$
(6)
$${\delta }_{a} \sim \mathrm{Beta}(1,\,1)$$
(7)
We assume that the number of adverse outcomes x in n total exposures (doses or infection) follows a binomial distribution:
$$P(x|p)=\frac{n}{x}{p}^{\,x}{(1-p)}^{n-x}$$
(8)
where p = αa when x corresponds to adverse outcomes given infection, and p = δa when x corresponds to vaccine-linked SAEs.
The posterior distribution of adverse infection or vaccine-linked outcomes therefore follow a beta distribution:
$$P(\,p|x,\,n) \sim \mathrm{Beta}(x+1,\,n-x+1)$$
(9)
Drawing 100,000 samples from the posterior distributions of both αa (adverse infection outcomes) and δa (vaccine-linked SAEs), we calculate the probability of vaccine benefit as the proportion of samples where δa ≤ δmax. This represents the probability that vaccination provides net benefit given the observed data.
Power to detect at least one death from vaccination
Finally, we extended the safety threshold δmax = αa × λi × ν to determine the minimum sample size required to conclude that the benefits of a vaccine in preventing death outweigh the risks, when no vaccine-associated deaths have been observed. Specifically, we calculate the sample size required to detect at least one vaccine-associated death with 80% power, assuming a one-sided exact binomial test and that the true vaccine risk is equal to the safety threshold δmax:
$${\rm{Power}}(n,\,{\delta }_{max})=1-P(x=0|n,\,{\delta }_{max})$$
(10)
where \(x \sim \mathrm{Binomial}(n,\,{\delta }_{\max })\).
Application to the IXCHIQ vaccine
Next, we apply the framework described above to evaluate the IXCHIQ vaccine, analysing two distinct severe outcomes: medically attended CHIKV cases and death. For each outcome, we estimate separate risk parameters for infection and vaccination by age group (18–64 and 65+ years).
Data
IXCHIQ vaccine benefits are defined as the prevention of medically attended CHIKV cases (that is, severe cases requiring hospitalization or medical care) or deaths following infection. Estimates of the proportion of infections leading to medically attended cases and deaths were based on data from a large chikungunya outbreak in Paraguay (2022–2023), where a post-outbreak seroprevalence study was conducted (298 deaths, 142,412 medically attended cases and 2.3 million infections)4. The limited prior circulation of chikungunya allowed for a more accurate estimation of the IFR, as post-outbreak seroprevalence more closely reflected recent infections rather than background immunity. Medically attended cases were defined using national surveillance data and included symptomatic individuals who sought care and met the clinical case definition (sudden onset of fever and arthralgia or disabling arthritis not explained by another condition)4. In the 18–64 year age group, there were 65 deaths, 80,342 cases and 1,272,530 infections. In the 65+ year age group, there were 178 deaths, 18,747 cases and 144,113 infections.
IXCHIQ vaccine risk outcomes include SAEs (both non-fatal and fatal). To compare vaccine risks and benefits, we assume that medically attended CHIKV cases are equivalent to a non-fatal SAE, but vary this in a sensitivity analysis. We calculated the proportion of vaccine-associated SAEs stratified by age group (18–64 and 65+ years) from data on the number of vaccine doses administered and the number of vaccine-associated SAEs12 (Valneva communication). In the 18–64 year age group, there were seven non-fatal SAEs and zero fatal SAEs out of 32,949 vaccinated individuals. In the 65+ year age group, there were 22 non-fatal SAEs and one fatal SAE out of 18,445 vaccinated individuals.
Statistical analysis
We estimated the risks and benefits of the IXCHIQ vaccine by calculating the net number of severe outcomes averted using equation (3). To incorporate uncertainty in the observed data, we sampled 100,000 times from binomial distributions representing the number of observed events (medically attended cases and deaths given infection, and fatal and non-fatal SAEs) given the total exposures (infections or vaccine doses). From these samples, we estimated the distribution of outcomes averted for each age group and epidemic scenario.
We also evaluated the risks and benefits of IXCHIQ using DALYs, which represent the total number of healthy years lost due to illness, disability and premature death. DALYs are defined as the sum of years of life lost (YLLs) and years lived with disability (YLDs). YLLs were calculated using the average life expectancy of individuals in La Réunion and the probability of death associated with infection or vaccination described above. To calculate YLDs, we assumed that 50% of infections are symptomatic6. Among symptomatic infections, 13% (ages 18–64 years) and 26% (ages 65+ years) require medical attention4, while the remaining 87% and 74%, respectively, are mild cases. We assumed that 50% of medically attended chikungunya cases develop chronic arthralgia4. Mild and medically attended acute cases were assumed to last seven days, with disability weights of 0.006 and 0.133, respectively24. Chronic chikungunya was assumed to last six months with a disability weight of 0.233 (refs. 25,26). As above, we assumed that SAEs are equivalent to medically attended chikungunya, with a duration of seven days and a disability weight of 0.133.
Additionally, we estimated the probability that vaccination provides net benefit for each outcome and age group separately. Using the posterior sampling approach (equations (6)–(9)), we drew 100,000 samples from the Beta distributions of infection risks (αa for deaths and cases) and vaccine risks (δa for SAEs), then calculated the proportion of samples where δa ≤ δmax for each combination of outcome (deaths or medically attended cases) and age group (18–64 or 65+ years).
We initially assumed a fixed vaccine efficacy of 95% for all age groups3. We included sensitivity analyses where we assumed vaccine efficacies ranging from 0 to 100%, attack rates of 0–50%, more or no vaccine-associated deaths, and that SAEs are instead comparable to chronic chikungunya cases17.
All analysis was conducted in R version 4.4.3 (28 February 2025).
Reporting Summary
Further information on research design is available in the Nature Portfolio Reporting Summary linked to this Article.
