Data collection

We analyzed Omicron BA.1 neutralization geometric mean titers (GMT) from 85 different sources which at the time of data collection (2021/12/08–2022/08/14) were mainly in not peer-reviewed preprint form on bioRxiv, medRxiv or otherwise in the public domain (Supplementary Fig. S1). By now, many of these preprints have been published in peer-reviewed journals. However, we collected the data in real time and updated a publicly available Google Slide deck², summarizing each study, and a publicly accessible Google Sheets document incrementally with incoming data³, also available in the manuscript’s GitHub repository (https://github.com/acorg/netzl_et_al2025/blob/main/data/google_sheet_tables/Netzl%20et%20al.%20-%20Collected%20Omicron%20antigenic%20data.csv). We base our analysis here on such collected data, the first publicly available preprints, in order to emulate the real-world scenario of a novel emerging variant and the urgency that comes with reporting its immune escape. To test that our analysis holds for published data, we randomly selected eight studies, corresponding to roughly 10% of the preprint data we used at the time, and compared the preprint-extracted data with the data given in the final publication (Supplementary Table S1). All values were consistent between preprint and publication except for a single study which added samples after BA.1 or BA.2 (breakthrough) infection that marginally changed reported GMTs. Importantly, we found no differences in the 2x and 3x Vax groups, which were the focus groups from a public health perspective at the time of BA.1 emergence. Given the consistency of preprinted and published data in this subset, and the substantial time-consuming nature of data extraction, we did not repeat the process for the remaining 90% of studies.

We collected data indiscriminate of academic institution or geographical location, but found a strong bias towards studies from the US and Germany in our final dataset, with more than half of all collected studies having a corresponding author located at a German or US institution (Table 1, Supplementary Fig. S2). Further, the majority of subjects across studies were female (Supplementary Fig. S3).

The collected data contained neutralization data of various Omicron sublineages (B.1.1.529: BA.1 and BA.1+R346K (BA.1.1); BA.2, BA.2.12.1 and BA.2.75; BA.3; BA.4/5) as well as ancestral and other SARS-CoV-2 variants by different vaccine sera and sera of individuals infected with the ancestral virus (614D/G, from here onwards referred to as wild type WT), Alpha (B.1.1.7), Beta (B.1.351), Gamma (P.1) or Delta (B.1.617.2) variant. As time progressed, other Omicron lineages emerged, and we kept collecting the neutralization data as it was generated (Supplementary Fig. S1). In this study, however, we did not include neutralization data from Omicron sublineages later than BA.2. For the purpose of the current study, to assess the usefulness of collating public data from variable sources for an early assessment of a variant’s immune escape, we focused our analyses on the BA.1 lineage. BA.1 was the first Omicron variant to circulate, and therefore the sublineage for which we extracted the most data, allowing us to draw more meaningful conclusions on the impact of variability of public data.

The majority of the collected data was generated using the Omicron BA.1 lineage. Some research groups indicated that the virus they used had the R346K mutation (BA.1.1, BA.1+R346K). To identify whether this substitution impacted neutralization, we compared BA.1 and BA.1.1 neutralization titers in the same sera and found no statistically significant difference of GMTs for the two lineages (Supplementary Fig. S4, Supplementary Table S2).

As the data was generated in different laboratories with little coordination between laboratories, a variety of neutralization assays and cell types was used, an overview is given in Table 1. We categorized the serum panels used by the different laboratories by their infection or vaccination history into different serum groups as described in the methods section.

Collective data can inform public health decisions early after variant emergence

Early in the pandemic, neutralization titers were established as correlates of protection against severe disease^11,12. Therefore, measures such as breadth of neutralization across variants and fold reduction of titers were used to inform public health decisions, including vaccine strain updates after the emergence of the BA.1 variant^13,14. Data from government approved laboratories and clinical trials using standardized assays contribute hugely to informing these decisions but take time to generate. At the same time, individual studies generate these data quickly, but in a less controlled manner. To assess BA.1’s immune escape from WT, we extracted the reported mean fold changes or calculated fold changes from reported geometric mean titers (GMTs) of individual studies in different serum cohorts. We did this incrementally, adding new data in real time and summarizing it in public documents², and we show data within the first two weeks of reporting in Fig. 1a.

We continued collating data and were interested how soon after variant emergence enough data was available to obtain stable immune escape estimates relevant for public health guidance. To do so, we examined how the mean neutralization titer fold drop from WT to BA.1 changes over time with the addition of more data (Fig. 1b). For WT conv (614D/G convalescent) and 2x Vax (twice vaccinated), there was already enough data reported within the first 15 days of data reporting such that the cumulative mean was within the 95% CI (confidence interval) of the total mean and remained stable over the remaining collection period. In the 3x Vax group, the fold change from WT was overestimated by one two-fold between two and six weeks after data reporting and converged to the total mean thereafter.

We calculated the confidence intervals of the cumulative means as a measure of how reliable the mean estimates are up to a given time point. We found that 2 weeks of data collection for the WT conv, 2x Vax and 3x Vax group was sufficient to bring the cumulative mean 95% CI of all three groups to a single log2 unit precision. After a month of data collection it was further improved up to half a log2 unit for the Vax groups. The cumulative mean 95% CI for the Vax + BA.1 group was generally wider throughout data collection, especially early on, owing to less data availability for this type of sera. Nevertheless, the 95% CI for the Vax + BA.1 was less than two log2 units one month after the first Vax + BA.1 data was collected, and less than one and a half log2 units after a month and a half.

BA.1 neutralization titer fold change from WT (614D/G) titers over time. (a) Fold changes of GMT neutralization titers as reported by individual studies are shown for the first two weeks of data reporting (n=10 studies), ordered by magnitude and stratified by serum group. Row labels show the study name (Table 1), time since exposure and serum type. The big red circle indicates the mean value for each group. Below the serum group label, the BA.1 geometric mean titer (GMT) is given followed by the mean fold change from WT titers. Arrows in the plot and “>” in the label indicate uncertainties in the point estimate due to titers below the limit of detection (LOD) of the assay. A short arrow (“>”/”>”/”b) Log2 fold changes from WT GMTs to BA.1 GMTs are shown for individual studies by time since the first reported data (1 month = 30 days). The blue line shows the cumulative mean with 95% CI indicated by the shaded area, the red line with shaded area corresponds to the mean fold change with 95% CI at the end of the data collection period. The data is stratified by exposure history.

In addition, we found clear evidence of the benefit of a third vaccine dose over two doses already after two weeks, which only became more certain over time. Further, we found that an exposure to BA.1 after vaccination greatly reduces BA.1’s immune escape. This observation was only possible once enough BA.1 breakthrough infections occurred to be assayed. Despite this lag in reporting, employing public data indicated the benefit of a vaccine strain update to BA.1 already 4 months after its emergence. These observations are further explored in section “Antigenic cartography of BA.1’s immune escape” via use of antibody landscapes.

To get a confidence estimate of the variance in the titer fold change calculation after, say, ten studies we randomly selected (with replacement) ten of all samples of 2x Vax data. We repeated this random selection (bootstrapping) 11 times, corresponding to 10% of the number of 2x Vax samples, and calculated the mean WT to BA.1 fold change each time. We next calculated the 95% CI of this distribution of mean fold changes and repeated this bootstrap process for all number of samples from 1 to 108, and additionally for the 3x Vax data (Supplementary Fig. S5). We found that with just 10 randomly selected samples, the overall fold change was within the 95% CI of the mean fold changes for both the 2x and 3x Vax cohorts. With 20 and 15 random samples, respectively, the range of lower to upper 95% CI bound was lower than 1.5x, meaning that both would be within one dilution step of a neutralization assay, and within the population’s mean 95% CI.

Omicron BA.1’s immune escape from wild type in different exposure histories

In the same style as Fig. 1a, we present the BA.1 fold drops from WT at the final stage of data collection in Fig. 2 for more types of exposure histories and more laboratories for exposure groups that already exist in Fig. 1a. The numerical data in all serum groups and for other variants is summarized in Tables 2,3,4.

Omicron BA.1 neutralization titer fold changes relative to WT (614D/G). Same as Fig. 1a but without row labels.

The double and triple vaccinated serum groups constituted the majority of the data that have been reported and consequently analyzed here and were the most relevant from a public health perspective at the time of data collection. The 2x Vax serum group contained the highest number of individual measurements and exhibited the widest spread and largest uncertainty in fold drops of BA.1 neutralization compared to WT. The 2x Vax group showed the most variability in fold changes from WT, which we attribute to a wide range of reported WT titers, serum collection times from two weeks to nine months post second dose, and limit of detection (LOD) censoring of low to non-detectable Omicron titers (Supplementary Fig. S6–S7). Time since exposure impacts titer magnitude and measured fold changes if the waning across variants over time is non-homogeneous, as we discuss later in section “Increased cross-neutralization over time since exposure”. Neutralization titers measured around the peak of immune responses, between two and 4 weeks post exposure, were highly variable in the collected data (Supplementary Fig. S7).

We found an average fold drop of 18x in this serum group (Fig. 2, Table 3) when treating measurements below an assay’s LOD in the common manner as LOD/2. However, the majority of fold changes were likely greater than the point estimates due to many BA.1 titers being below the LOD. Consequently, the average fold drop is likely substantially greater than 18x. We found that in many studies with an unexpectedly low fold drop from WT to BA.1 titers, titers against WT were very low^15,16,17,18. Low titers against the reference antigen limit the amount of further reduction until an assay’s detection limit is reached, resulting in LOD censoring of titers and seemingly low fold drops (Supplementary Fig. S6–S7).

LOD censoring did not occur in the 3x Vax group where all WT and almost all BA.1 titers were detectable (Supplementary Fig. S6). Hence, the estimated average fold drop of 6.3x for this group is more reliable compared to the 2x Vax group and demonstrates the benefit of a third WT vaccination, which was much debated at the time of BA.1’s emergence. We investigated if the substantially lower fold drop in 3x Vax compared to 2x Vax is because higher titers in 3x Vax were underreported, either by laboratories not titrating to the endpoint, or because of a high-titer non-linearity in the assay by looking at titers and reported fold drops and found that the fold drop from WT to BA.1 was independent of titer magnitude against WT (Supplementary Fig. S6). A third vaccine dose also reduced fold changes more than non-Omicron breakthrough infections, with fold drops from WT of 15x and 9.4x in the Vax + Inf and Inf + Vax groups. Additionally, a third dose lifted titers against BA.1 above a level identified to be protective against symptomatic infection during WT circulation (Supplementary Fig. S6).

The effect of low WT titers on fold drops from WT to BA.1 is also seen in convalescent serum cohorts, despite the limited amount of data for most of these cohorts. While high titers against WT after infection with a WT-like variant resulted in mean fold drops from WT of 19.5x and 27.4x in the WT and Alpha conv (convalescent), respectively, mean fold drops in Beta and Delta conv were much lower, at 6.8x and 6.3x due to lower WT titers (Supplementary Fig. S7, Table 2–3). On the other hand, BA.1 fold drops relative to the infecting Beta and Delta variants were much higher compared to BA.1 fold drops from WT for the same groups: 24.7x and 36.7x (vs 6.8x and 6.3x) (Table 3). One possible mechanism is again an LOD effect reducing fold drops from WT in Beta and Delta convalescent serum groups, since these groups will have higher homologous titers compared to WT titers. Another possible mechanism could be neutralizing antibodies in Beta and Delta sera focusing on regions that are distinct from both WT and BA.1 and therefore, with respect to these sera, WT and BA.1 appear antigenically less distinct leading to lower titer differences between these antigens.

As expected, a (breakthrough) infection with either BA.1 or BA.2 substantially increased titers against BA.1, at times even above WT titer levels, and resulted in average fold drops smaller than 3x.

In general, we saw high variability of fold drop data within all serum groups in the whole dataset, likely owing to several factors such as age of participants, serum collection times, and different assays and cell types used to assess serum neutralization ability (Table 1). For the 3x Vax and WT conv groups, we found a statistically significant difference between fold changes measured using authentic virus (LV) and pseudovirus (PV), but while LV fold changes were significantly higher than PV in 3x Vax it was the other way around in WT conv (Supplementary Table S3, Supplementary Fig. S7). For reported GMTs however, we consistently found higher PV than LV GMTs, often at a significant level (Supplementary Tables S5–S6, Supplementary Fig. S7). Different serum collection times can affect fold drops either through an LOD mechanism as explained above or through higher cross-reactivity as demonstrated in Wilks et al.¹⁹. Still, combining data from various sources gives quick and reliable results as shown in Fig. 2, and a public database where laboratories enter their results would greatly contribute to quick decision making, and further, assay refinement across laboratories.

Antigenic cartography of BA.1’s immune escape

As an additional way to evaluate the reliability of variable source data for immunological surveillance, we constructed antigenic maps from the collected data and compared them to single source antigenic maps. In an antigenic map, virus variants are positioned based on their antigenic properties inferred by fold drops in serum neutralization titers²⁰. Variants that elicit similar titers in the same sera are positioned at small distances from each other, and vice versa. Antigenic maps are a key instrument for vaccine strain selection for influenza vaccines and have also been used to investigate SARS-CoV-2 vaccine strain updates²¹.

Using data only from convalescent serum cohorts, we again found that data within one month of reporting produced a very stable result (Fig.3 a-b) (Supplementary Fig. S8–S13). Only Gamma’s position changed slightly when creating a map from all data (Fig. 3b). Already after one month of data reporting, the map captured BA.1’s complete escape from all pre-Omicron sera and variants. The early data map and the full data map were highly consistent with maps from single laboratories, both a map constructed using pseudovirus neutralization data¹⁹ (Fig. 3c) and one using authentic virus²² (Fig. 3d). Maps constructed using only authentic virus or pseudovirus neutralization data resulted in very similar variant positions for variants with sufficient titrations in the different serum groups (Supplementary Fig. S8). We found that the authentic virus map is marginally condensed compared to the pseudovirus map, visible by arrows pointing outwards in Supplementary Fig. S8. This is in line with the on average lower fold drops reported in authentic virus assays in single exposure cohorts (Supplementary Tables S3–S4).

Whereas antigenic maps show individual sera as points based on their reactivity, antibody landscapes show the distribution of neutralization titers for individual sera against multiple strains as a surface in a third dimension above an antigenic map²³. To illustrate the effect of a third vaccination or breakthrough infection, we constructed antibody landscapes for the 2x Vax, 3x Vax and Vax + BA.1 serum groups, again for different end points (Fig. 3e-g). As before, we found that combining data from different sources gives stable representations of population immunity early after variant emergence. Moreover, we found a much heightened and broadened neutralizing response after, firstly, a third Wu-1 dose compared to only two doses, and secondly, BA.1 breakthrough compared to a third Wu-1 exposure. A third dose was necessary to lift BA.1 titers to a somewhat protective level compared to two doses (Supplementary Fig. S6).

Antigenic cartography. (a) An antigenic map from convalescent data up to 1 month post first available data was constructed. Variants are shown as labeled, colored circles and sera are shown as open squares with the color matching the infecting variant. The x- and y-axes correspond to relative antigenic distances, each grid line reflects an additional 2-fold dilution in the neutralization assay. (b) An antigenic map constructed from all convalescent data with arrows pointing to the variants’ positions in a. (c) Comparison of the map by Wilks et al.¹⁹. to the full data map in b and (d) the map by Roessler et al.²². to the full data map in b. Arrows point to the variants’ positions in the respective map. The numbers on the bottom left corner show the stress of the map. (e–f) GMT antibody landscapes for the 2x Vax (grey), 3x Vax (dark grey) and Vax + BA.1 (red) serum groups are shown, subset to early data up to 1 month post first report (e), medium data up to 90 days post first report (f) and all data (g). GMTs against variants are indicated by impulses.

Increased cross-neutralization over time since exposure

Antibody responses are highly dynamic processes, hence the timing of sampling impacts neutralization titer magnitude and fold changes. If sampled too early since exposure, immune responses did not have enough time to fully mature, resulting in generally low neutralization titers and seemingly broad cross-neutralization. After the peak immune response, titers gradually wane. Wilks et al.¹⁹, however, found that variant cross-neutralization increased over time after the second vaccination, and that cross-reactive antibodies were recalled after the third dose rather than induced by the third dose, indicating ongoing affinity maturation after the second dose. Following their approach, we constructed antibody landscapes for serum cohorts by time since exposure when the information was available. We binned individual sera by 2 weeks, 1, 3, 6, 9 and 12 months post exposure and fitted the landscape slope for these binned cohorts, with a smaller slope indicative of more cross-neutralization across antigenic space (Fig. 4).

Antibody landscape slopes since exposure. Antibody landscapes were fitted to sera binned by their time since exposure for the 2x and 3x Vax cohorts. The time since exposure is given on the x-axis, the y-axis shows the fitted landscape slope. A low slope is indicative of broad cross-neutralization. The numbers in the top row indicate the number of sera that contributed to each landscape. The color of the point shows the fraction of variant titer measurements that contributed to the landscape fit, with a darker color indicating a more complete data set and better geometrical information for individual sera. The dot size gives the amount of data (n × fraction of titrated variants) per point scaled by the maximum data amount across serum cohorts. A detailed description is given in the methods section. The slopes reported by Wilks et al.¹⁹. are shown in grey and not scaled by data or fraction of titrated variants.

We found an overall trend of lower slopes as time since exposure increased, indicating higher cross-neutralization. In general, the data with information since exposure was scarce and binning resulted in additional inaccuracies, explaining incoherences of individual data points. Still, the slope value 9 months after the 2^nd dose is remarkably similar to the slope values half a month and one month after the 3^rd dose, supporting the hypothesis by Wilks et al.¹⁹ that a third dose recalls broadened immunity rather than induces substantial broadening.