Epidemics 47 (2024) 100757

Contents lists available at ScienceDirect

Epidemics
journal homepage: www.elsevier.com/locate/epidemics

A multiscale modeling framework for Scenario Modeling: Characterizing the

heterogeneity of the COVID-19 epidemic in the US
Matteo Chinazzi a,b ,1 , Jessica T. Davis b ,1 , Ana Pastore y Piontti b , Kunpeng Mu b , Nicolò Gozzi c ,
Marco Ajelli d , Nicola Perra e,b , Alessandro Vespignani b,c ,∗
a
The Roux Institute, Northeastern University, Portland, ME, USA
b Laboratory for the Modeling of Biological and Socio-technical Systems, Network Science Institute, Northeastern University, Boston, MA, USA
c Institute for Scientific Interchange Foundation, Turin, Italy
d Laboratory for Computational Epidemiology and Public Health, Department of Epidemiology and Biostatistics, Indiana University School of Public

Health, Bloomington, IN, USA

e
School of Mathematical Sciences, Queen Mary University, London, UK

ARTICLE INFO ABSTRACT

Dataset link: https://coronavirus.jhu.edu/, http The Scenario Modeling Hub (SMH) initiative provides projections of potential epidemic scenarios in the United
s://www.oag.com/, https://www.iata.org/, ht States (US) by using a multi-model approach. Our contribution to the SMH is generated by a multiscale
tps://www.google.com/covid19/mobility/, htt model that combines the global epidemic metapopulation modeling approach (GLEAM) with a local epidemic
ps://github.com/OxCGRT/covid-policy-tracker
and mobility model of the US (LEAM-US), first introduced here. The LEAM-US model consists of 3142
Keywords: subpopulations each representing a single county across the 50 US states and the District of Columbia, enabling
Metapopulation dynamics us to project state and national trajectories of COVID-19 cases, hospitalizations, and deaths under different
Multi-strain epidemic modeling epidemic scenarios. The model is age-structured, and multi-strain. It integrates data on vaccine administration,
COVID-19 pandemic human mobility, and non-pharmaceutical interventions. The model contributed to all 17 rounds of the SMH,
and allows for the mechanistic characterization of the spatio-temporal heterogeneities observed during the
COVID-19 pandemic. Here we describe the mathematical and computational structure of our model, and
present the results concerning the emergence of the SARS-CoV-2 Alpha variant (lineage designation B.1.1.7)
as a case study. Our findings show considerable spatial and temporal heterogeneity in the introduction and
diffusion of the Alpha variant, both at the level of individual states and combined statistical areas, as it
competes against the ancestral lineage. We discuss the key factors driving the time required for the Alpha
variant to rise to dominance within a population, and quantify the impact that the emergence of the Alpha
variant had on the effective reproduction number at the state level. Overall, we show that our multiscale
modeling approach is able to capture the complexity and heterogeneity of the COVID-19 pandemic response
in the US.

1. Introduction poor coordination among modeling teams, divergent results caused by

different underlying assumptions and scenarios, and a lack of clarity
Mathematical and computational models have been essential in regarding the implemented methods. To address these issues, the Sce-
understanding the transmission mechanisms of SARS-CoV-2, provid- nario Modeling Hub (Scenario Modeling Hub, 2023) has coordinated
ing situational awareness throughout the COVID-19 pandemic, and multiple modeling teams in the generation and analysis of multi-model
allowing the exploration of hypothetical intervention scenarios for projections of well-defined epidemic scenarios (Borchering et al., 2021;
public health planning and response (Holmdahl and Buckee, 2020; Biggerstaff et al., 2022; Truelove et al., 2022; Howerton et al., 2023;
Jewell et al., 2020; Poletto et al., 2020; Brooks-Pollock et al., 2021; Borchering et al., 2023). This hub approach allows for a synoptic
Biggerstaff et al., 2022; Reich et al., 2022). Despite the many suc-
analysis of results, ensembling different estimates, rigorous validation
cessful applications of predictive modeling, there are often challenges
of findings, and clearer communication of results.
in communicating the results to policymakers and the public due to

∗ Corresponding author at: Laboratory for the Modeling of Biological and Socio-technical Systems, Network Science Institute, Northeastern University, Boston,
MA, USA.
E-mail address: [email protected] (A. Vespignani).
1
Authors contributed equally.

https://doi.org/10.1016/j.epidem.2024.100757
Received 15 August 2023; Received in revised form 22 January 2024; Accepted 26 February 2024
Available online 5 March 2024
1755-4365/© 2024 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-
nc-nd/4.0/).
M. Chinazzi et al. Epidemics 47 (2024) 100757

As of July 2023, we have contributed 17 rounds of projections transportation (OAG, 2020). Ground mobility and commuting flows
coordinated by the SMH, which consist of 70 different modeling sce- are modeled from data collected from statistics offices of 30 countries
narios defined at various points in time starting in 2021. Our mod- on 5 continents and account for travel restrictions and government
eling approach combines two stochastic, age-structured, multi-strain, policies (Hale et al., 2021). The model also considers the reduction
metapopulation models operating on different scales (Balcan et al., of internal country-wide mobility and changes in contact patterns
2010). This approach allows us to model both the local dynamics in each country and state (Google LLC, 2021a). In both models we
that integrate the vaccination rollout plans and the strength of non- consider individuals divided into 10 age groups: 0–9, 10–19, 20–24,
pharmaceutical interventions (NPIs) as well as the global dynamics 25–29, 30–39, 40–49, 50–59, 60–69, 70–79, and 80+ years old. We use
responsible for the introduction of new variants. In particular, the effective contact matrices to model age-dependent and country/state
Global Epidemic and Mobility model (GLEAM) which has been used to specific mixing across four settings: households, schools, workplaces,
study the international spread of pathogens such as Zika (Zhang et al., and contacts occurring in the general community. The contact matrix
2017), Ebola (Gomes et al., 2014; Pastore y Piontti et al., 2016), and for a given location is a weighted linear combination of the derived
the initial wave of COVID-19 (Chinazzi et al., 2020; Davis et al., 2021), matrices for the four social settings and encodes information on the
can simulate introduction events of new variants in the United States average number of effective contacts (contacts that can lead to the
(US) from other countries. The output of this model is used to define transmission of a disease) between individuals of particular age groups.
the initial conditions of the Local Epidemic and Mobility model (LEAM- Details on the contact data and the construction of the matrices can be
US) that in turn simulates the disease dynamics in the US at the county found in Mistry et al. (2021) and Prem et al. (2017).
level. In the LEAM-US model, contact matrices, age-specific traveling
Here we describe our general modeling approach and report the probabilities, and air traffic flows are properly mapped to the county-
detailed results obtained following the guidelines of the scenario design level resolution. Counties’ populations and age distributions are based
of Round 5 of the SMH (COVID-19 Scenario Modeling Hub, 2021). on the Census’ annual resident population estimates during 2019, and
The four scenarios of this round address the impact of vaccination commuting flows between counties are obtained from the 2011–2015
coverage and relaxation of NPIs during the wave initiated by the Alpha 5-Year ACS Commuting Flows survey and properly adjusted to account
variant (Phylogenetic Assignment of Named Global Outbreak, PANGO, for differences in population totals since the creation of the dataset (US
lineage designation B.1.1.7). The Alpha variant was first identified in Census Bureau, 2024). Google’s COVID-19 Community Mobility Re-
December 2020 in the United Kingdom (UK) (Walensky et al., 2021; ports data collected at the county-level resolution are used to model
World Health Organization, 2021). It was traced back to two samples mobility and the effects of NPIs on individual behaviors (Google LLC,
collected in September 2020 (Science Magazine, 2020; Rambaut et al., 2021a).
2020). The multiscale structure of our model and its capacity to link
international importations with domestic contact patterns and mobility, 2.1. SARS-CoV-2 transmission model
provides a detailed characterization of the heterogeneous spread of the
Alpha variant in the US. We estimate that its introduction and trajectory In both GLEAM and LEAM-US, within each subpopulation, we adopt
towards dominance exhibited significant spatiotemporal variation. In a classic 𝑆𝐿𝐼𝑅-like disease infection dynamics. Depending on the
particular, our model finds that by March 2021 the Alpha variant SMH round, the model has been extended to account for the presence
accounted for 50% or more of the total infections only in roughly one of multiple lineages and vaccination protocols. After establishing the
third of states. In contrast, other states did not reach this proportion mobility data layers and defining the dynamics of the disease, the
until the end of April or early May. Notably, this heterogeneity is even population count within each compartment, denoted as 𝑚, for each age
more pronounced at the combined statistical areas (CSAs) level within group 𝑖, and for each subpopulation 𝑗, is governed by a discrete and
states. Additionally, we show how the emergence of the Alpha variant stochastic dynamical equation. This equation is formulated as follows:
affected variations in the effective reproduction number at the state
𝑋𝑗[𝑚,𝑖] (𝑡 + 𝛥𝑡) − 𝑋𝑗[𝑚,𝑖] (𝑡) = 𝛥𝑋𝑗[𝑚,𝑖] + 𝛺𝑗 ([𝑚, 𝑖]) (1)
level.
Overall, our multiscale model offers a comprehensive and detailed where the term 𝛥𝑋𝑗[𝑚,𝑖]
denotes the change attributable to transitions
approach to projecting the COVID-19 pandemic in the US, incorpo- within compartments, which are driven by the dynamics of disease
rating factors such as population demographics, travel patterns, NPIs, transmission. Additionally, the operator 𝛺𝑗 ([𝑚, 𝑖]) captures the varia-
vaccination status, and new SARS-CoV-2 variants. These projections tions arising from individual mobility. This particular operator accounts
can inform public health policy and decision-making by capturing the for long-range mobility, specifically via airlines, and establishes the
heterogeneity and complexity of the COVID-19 pandemic response in minimal integration time scale as one day. Finally, the impact of
the US. commuting flows on mobility is incorporated by defining effective
forces of infection. This is achieved through a time scale separation
2. Methods approximation detailed in Balcan et al. (2010), Balcan and Vespignani
(2011). The function 𝛥𝑋𝑗[𝑚,𝑖] is defined as the aggregate of all transitions
The multiscale modeling approach combines two distinct epidemic into and out of the disease compartment 𝑚 for individuals within age
models that work at different geographical resolutions: the Global group 𝑖, denoted as [𝑚, 𝑖]. The operator 𝑗([𝑚, 𝑖], [𝑛, 𝑖]) quantifies the
Epidemic and Mobility model (GLEAM) and the Local Epidemic and transitions from [𝑚, 𝑖] to [𝑛, 𝑖] over the time interval 𝛥𝑡. Each element of
Mobility model in the US (LEAM-US) (Fig. 1). Both models are stochas- this operator is derived as a random variable, following a multinomial
tic, spatial, age-structured, metapopulation models (Balcan et al., 2009, distribution. Therefore, the change in the compartment [𝑚, 𝑖] over the
2010; Pastore y Piontti et al., 2018; Chinazzi et al., 2020). LEAM-US, interval 𝛥𝑡, represented as 𝛥𝑋𝑗 [𝑚,𝑖] , is calculated by summing all the
considers 3142 counties (or their statistical equivalent) as individ- random variables {𝑗 ([𝑚, 𝑖], [𝑛, 𝑖])} as follows
ual subpopulations in each of the 50 US states and the District of ∑{ }
Columbia. GLEAM considers 3200 subpopulations across nearly 190 𝛥𝑋𝑗[𝑚,𝑖] = −𝑗 ([𝑚, 𝑖], [𝑛, 𝑖]) + 𝑗 ([𝑛, 𝑖], [𝑚, 𝑖]) . (2)
countries, defined as catchment areas of major transportation hubs. [𝑛,𝑖]

GLEAM and LEAM-US integrate a human mobility layer, represented To illustrate the above equation with a specific example, let us in-
as a network, using both short-range (i.e., commuting) and long-range spect the dynamics of the latent compartment. Consider individuals
(i.e., air traveling) mobility data from different sources. International within age group 𝑖 of subpopulation 𝑗. These individuals have two
and domestic airline mobility data, in the origin–destination format, are potential transitions: they can either move into the latent compart-
from the Official Aviation Guide database and are used to model airline ment, denoted as 𝐿𝑖𝑗 , from the susceptible compartment, represented

2
M. Chinazzi et al. Epidemics 47 (2024) 100757

Fig. 1. Visual depiction of the multiscale modeling approach that combines the GLEAM and LEAM-US models.

as 𝑆𝑗𝑖 , or they can exit the latent compartment to enter the infectious gives insights into contact reductions in broader community settings.
compartment, indicated by 𝐼𝑗𝑖 . The components of the operator that We achieve this by proportionally rescaling the corresponding layers
define the 𝐿𝑖𝑗 dynamic are thus determined by the following binomial in the contact matrices. This rescaling factor, 𝜔𝑠 (𝑡) = 𝜔𝑠 (1 + 𝑟𝑙 (𝑡)∕100)2 ,
distributions𝑃 𝑟𝐵𝑖𝑛 (𝐿𝑖𝑗 (𝑡), 𝑝𝐿𝑖 →𝐼 𝑖 ) and 𝑃 𝑟𝐵𝑖𝑛 (𝑆𝑗𝑖 (𝑡), 𝑝𝑆 𝑖 →𝐿𝑖 ), where 𝑝𝐿𝑖 →𝐼 𝑖 is applied, where 𝑟𝑙 (𝑡) represents the daily percentage change in visitors
𝑗 𝑗 𝑗 𝑗 𝑗 𝑗
and 𝑝𝑆 𝑖 →𝐿𝑖 are the transition probabilities from the latent to the infec- to specific locations 𝑠 relative to pre-pandemic levels. The squared
𝑗 𝑗
tious state and from the susceptible to the latent state, respectively. We term in this factor reflects the understanding that the potential number
model the transition process as memoryless, discrete, and stochastic. of contacts at a location is proportional to the square of the visitor
The transition probability 𝑝𝑆 𝑖 →𝐿𝑖 , representing the force of infection, is count. We selected specific fields from Google’s Community Mobility
𝑗 𝑗 Report data due to their alignment with the definitions of various place
influenced by several factors: commuting flows, interaction patterns as
categories. The ‘retail and recreation percent change from baseline’
defined in age-structured contact matrices, and the implementation of
field effectively represents mobility trends for locations such as movie
local Non-Pharmaceutical Interventions (NPIs). For a complete descrip-
theaters, restaurants, cafes, and shopping centers. This particular data
tion of the analytical framework of the model, we direct readers to the
is most representative of the interactions occurring within the general
detailed presentation provided in Balcan et al. (2010).
community layer of our contact matrices. Meanwhile, the ‘workplaces
In the removed compartment, individuals can no longer infect oth-
percent change from baseline’ field is instrumental in measuring the
ers, meaning they have either recovered, been hospitalized, or isolated.
mobility trends of individuals commuting to and from their workplaces,
Hospitalizations and deaths are computed from the removed compart-
providing valuable insights for our modeling purposes (Google LLC,
ment by considering a geometrically distributed time delay between
2021b).
the time of removal to hospitalization and death (details on the delay
implementation are provided in the Supplementary Information). Infec-
2.3. Vaccine allocation and administration
tion hospitalization ratios (IHR) and infection fatality ratios (IFR) are
age-structured and taken from the literature to account for different
Our model explicitly incorporates the time series data of daily
variants and vaccination statuses (Shapiro et al., 2021; Verity et al.,
administered COVID-19 vaccine doses. In the United States, the allo-
2020; Salje et al., 2020). It is worth remarking that the model’s pa-
cation of the daily vaccine stockpile for each county is based on the
rameters vary across SMH rounds as new variants and knowledge on observed vaccination rates at the state level. We then distribute these
vaccine efficacy emerged and as the prescribed scenarios changed. doses within each state, proportionally to the population size of each
county. Furthermore, the strategy for vaccine rollout is designed to
2.2. Non-pharmaceutical interventions and human mobility align with the recommendations of the Advisory Committee on Immu-
nization Practices (ACIP). This approach involves prioritizing different
In our model, we dynamically incorporate international travel re- age groups in a phased manner, depending on the specific stage of the
strictions based on data from the Oxford COVID-19 Government Re- vaccination campaign (Dooling et al., 2020). In particular, in phase
sponse Tracker (Hale et al., 2021). To accurately reflect changes in 1a doses were distributed between the 10 age groups according to
travel patterns since the pandemic’s onset, both international and the number of healthcare workers and long-term care facility residents
domestic travel flows are adjusted using real-time origin–destination in the population; in phase 1b they were distributed with priority
data provided by OAG (OAG, 2020), capturing the observed reductions to front-line essential workers and adults aged 75+; in phase 1c to
in air traffic. Additionally, we adjust short-range mobility by utilizing other essential workers, adults with high-risk conditions, and the 65–
workplace visitation data as a proxy. This approach, that consider the 74 age group; and lastly, in phase 2, doses were distributed to the
mobility changes from pre-pandemic levels, is informed by Google’s general population aged 18+. The vaccine uptake in the in-sample
COVID-19 Community Mobility Reports (Google LLC, 2021a), and pro- calibration window follows the data provided by the CDC and Our
vides a dynamic characterization of mobility patterns during the pan- World in Data platform (CDC, 2021; Our World in Data, 2021). In the
demic. Contact patterns and mixing rates among different age groups out-of-sample projection period, the vaccine uptake of each SMH round
in our model are adjusted to reflect the impact of policy interventions follows the directions of the specific scenarios, available at COVID-19
on individual behaviors. Specifically, we modulate the school contacts Scenario Modeling Hub (2021). Our model incorporates various vaccine
matrix layer to simulate the effects of school closures, whether due effects, including vaccine efficacy in reducing the risk of infection
to governmental policies or scheduled holiday breaks. For workplace (VE𝑆 ), hospitalization (VE𝐻 ), and deaths (VE𝐷 ). The specific values for
and general community settings, we utilize data from Google’s Mobil- these vaccine efficacies vary across different scenario rounds and are
ity Reports. The workplaces percent change from baseline informed by the ongoing analysis of efficacy against different variants.
metric informs us of the reduction in contacts within workplaces, while Additionally, the model accounts for the waning of vaccine-induced
the retail and recreation percent change from baseline protection starting from round 8.

3
M. Chinazzi et al. Epidemics 47 (2024) 100757

Fig. 2. (A) We implement a 𝑆𝐿𝐼𝑅-like model extended to account for the presence of two strains and vaccination. The superscript 𝛼 refers to compartments with individuals
infected with the Alpha variant of concern. Subscripts 𝑣𝑎𝑥1 and 𝑣𝑎𝑥2 are used to identify compartments with individuals who received one or two doses of the vaccine, respectively.
Vertical dashed lines represent transitions between compartments due to vaccinations. (B) Ratios of WIS scores between the GLEAM/LEAM-US model and the COVIDhub baseline
reference model. (C) Ratios of WIS scores between the GLEAM/LEAM-US model and the COVIDhub ensemble reference model.

2.4. Model calibration susceptible; two latent and infectious compartments (capturing individ-
uals infected with both the ancestral lineages and the Alpha variant);
The model is initialized by considering the introductions of infec- and the removed compartments. Additionally, each of the previous
tions during the early stage of the COVID-19 pandemic by coupling the compartments appears in the model in three different ways (as shown
LEAM-US model to the importations from the GLEAM model calibrated in Fig. 2A) to distinguish between unvaccinated individuals, individuals
as reported in Davis et al. (2021). In each state, we assume a flat who received the first vaccine dose, and vaccinated individuals who
prior for the effective reproductive number 𝑅𝑒𝑓 𝑓 at the start of the in- received two doses. Susceptible (𝑆) individuals become latent through
sample calibration time window. In order to account for variations in interactions with infectious individuals carrying either the ancestral
the IFR and IHR across states, we also consider a ± 30% difference with lineage or the variant. In the first case, individuals will transition
respect to the baseline parameterization, assuming a uniform prior. The into the ancestral lineage latent compartment (𝐿); in the second they
specifications set by the SMH in each round inform the time window will transition into the variant latent compartment (𝐿𝛼 ). We assume
used to calibrate the model. We calibrate our model using an Approxi- that the two lineages have different transmission rates (𝛽 and 𝛽 𝛼 ) but
mate Bayesian Computation (ABC) rejection approach (Sunnåker et al., the same latent and infectious periods (𝜀−1 and 𝜇 −1 ). Furthermore,
2013). This process involves comparing the model’s weekly estimated we capture the increase in transmissibility of the Alpha variant by
deaths and/or hospitalizations with the actual figures reported by the assuming that 𝛽 𝛼 = 𝛽(1 + 𝜓) (Galloway et al., 2021). The increase of
Johns Hopkins Coronavirus Resource Center (Dong et al., 2020) and transmissibility was introduced following previous studies indicating
the U.S. Department of Health and Human Services (US Department that the Alpha variant was 30%–70% more transmissible with respect
of Health & Human Services, 2021). To assess the accuracy of our to ancestral SARS-CoV-2 lineages (NERVTAG, 2020; PHE, 2021; Davies
et al., 2021). Latent individuals move to the infectious stage, 𝐼 for
model, we calculate the distance, denoted as 𝑠(𝐸 ′ , 𝐸), between the
the ancestral lineage and 𝐼 𝛼 for the Alpha variant, at a rate 𝜀 that
surveillance data (evidence 𝐸) and the model estimates (𝐸 ′ ) for each
is inversely proportional to the latent period. Infectious individuals
stochastic realization. Distances are measured using either the weighted
transition to the removed compartment (𝑅) at a rate 𝜇 that is inversely
mean absolute percentage error or the residuals. We then establish a
proportional to the infectious period. In our model, individuals transi-
tolerance level, based on a selected quantile of the empirical distance
tion between different compartments through stochastic binomial chain
distribution, to serve as our threshold. Any realizations that result in
processes. These transitions are guided by parameter values sourced
distances exceeding this threshold are rejected (Beaumont et al., 2002).
from existing literature, which outline the natural progression of the
Specifically, we keep the top 2.00% of realizations with the smallest
disease. During the period of our projections, the vaccination campaign
distance. For each specific SMH-scenario definition, we performed
was focused on administering the initial complete regimen of two doses.
between 15,000 to 50,000 stochastic independent realizations. We have
Accordingly, our model accounts for varying levels of vaccine efficacy
also performed extensive sensitivity analyses testing the calibration
against infection, hospitalization, and death, distinguishing between
approaches at the global and local level as reported in Davis et al.
the effects after the administration of the first and second doses. In
(2021).
collaboration with the SMH, the vaccine efficacy (VE) values for one
dose and two doses were established at 70% and 90% for susceptibility
2.5. Round 5 specific model design: Integrating the alpha variant to infection (VE𝑆 ), and 75% and 95% for both hospitalization (VE𝐻 )
and deaths (VE𝐷 ). It is important to note that during the scenario design
To incorporate the emergence of the Alpha variant mechanistically phase, detailed information on vaccine efficacy was limited, except for
we employ a two-strain model. This model allows us to mechanistically the efficacy against symptomatic disease, which was informed by phase
capture the cocirculation of the ancestral SARS-CoV-2 lineages and 3 trials (Polack et al., 2020; Pilishvili et al., 2021). The protection
the Alpha variant. The model considers the following compartments: conferred by the vaccination for the Alpha variant was assumed to be

4
M. Chinazzi et al. Epidemics 47 (2024) 100757

similar to those of the ancestral lineages, after considering the increased 3.1. Out of sample projections
transmissibility.
The assumptions of the future levels of NPIs and vaccination uptake Our model is calibrated using the complete epidemic history within
were incorporated based on the scenarios presented by the SMH. A the US, spanning from March 2020 to May 2021, with the calibration
full description of all scenarios can be found at this link (COVID-19 process based on weekly reported deaths (in-sample model estimates
Scenario Modeling Hub, 2021). In round 5 we explore two scenarios and goodness of fit details are provided in the SI). The model is
that assume different levels, moderate and low, of NPIs. More precisely, calibrated separately for each of the four round 5 scenarios (COVID-19
starting on May 1, 2021, we consider a reduction in the effect of Scenario Modeling Hub, 2021). We generate out-of-sample projections
NPIs on mobility and contacts relative to the effectiveness of control for the expected number of deaths and hospitalizations, along with
during the last two weeks in April, 2021. The two scenarios assume a associated uncertainties expressed as quantile ranges. These quantile
gradual reduction of social distancing measures by October 31, 2021 ranges are determined by considering the out-of-sample dynamics of
with respect to the April 2021 levels: an effective 50% reduction in individual stochastic trajectories, selected using an ABC rejection al-
the moderate NPI scenario, and an effective 80% reduction in the low gorithm during the in-sample calibration period. Specifically, for each
NPI scenario. For example, if NPIs caused mobility to decrease to 50% scenario, our models provide target projections consisting of 23 quan-
of its pre-pandemic value at the end of April, an 80% reduction in the tiles (ranging from 0.01 to 0.99 with increments of 0.025), covering
effectiveness of the interventions would imply a final mobility value by each week of the projection period. These quantiles represent ex-
end of October, 2021, that would be equal to 90% of its pre-pandemic pected incident hospitalizations and deaths. To facilitate the visual
value (i.e., 0.9 = 1 − (1 − 0.5) × (1 − 0.8)). representation and assessment of the probabilistic estimates, the quan-
Similarly, in the out-of-sample region, we consider two different tile projections are transformed into central prediction intervals (PIs).
scenarios for vaccine uptake. The high vaccination scenario assumes These prediction intervals encapsulate the model’s level of confidence
that vaccination coverage saturates at 83% of the eligible population, that future observations will fall within a specified range of values.
while the low vaccination scenario assumes a 68% coverage. These Evaluating scenario projections requires a fundamentally different
different scenarios were used to address the impact of vaccine hesi- approach compared to forecast models. While accuracy in predicting
tancy. Vaccination data was taken from Ref. Our World in Data (2021), actual outcomes is the main goal in forecasts, scenario projections have
CDC (2021) until May 1, 2021. Afterward, according to the SMH different purposes. They are developed to map the range of possible
scenario specifications, 50 million first doses were available per month, epidemic dynamics, rather than to offer precise predictions. Therefore,
following the 2-dose protocol (100 million total doses per month). assessing the quality of scenario projections it is not just about how
In round 5 the introduction of the Alpha variant in the US is mech- closely they match the ground truth, but also about the robustness of
anistically modeled by simulating the international spread using the the underlying assumptions of each scenario and the effectiveness in
GLEAM model. The GLEAM model is initialized with the introduction of enveloping the spectrum of potential trajectories. The consideration
a cluster of Alpha variant infections during the week of September 13– of both accuracy and the quality of scenario-based assumptions is
19, 2020, specifically in London and Kent, UK. These initial infections the key for scenario modeling performance evaluation. Despite these
are modeled as being drawn from a Poisson distribution with a mean caveats, to assess the performance of scenario projections, we utilize
of 40. This approach is based on the fact that the UK was sequencing the weighted interval score (WIS) as a performance indicator (Gneiting
approximately 5% of positive COVID-19 cases at that time (WHO, and Raftery, 2007; Bracher et al., 2021). The WIS considers the size
2020). We have incorporated into our model the assumption that the and positioning of prediction intervals relative to actual outcomes.
Alpha variant is 50% more transmissible than the ancestral strain, Lower WIS values indicate better forecasting performance (see SI for
denoted by 𝜓 = 0.5. From this setup, the model generates around a discussion of WIS methodology). For comparison, we consider two
300,000 stochastic realizations, each tracing the movement of indi- reference models generated by the COVID-19 Forecast Hub: the naive
viduals exposed to the Alpha variant traveling to the United States. baseline forecast, which predicts weekly values similar to the median
By aggregating this data, we are able to statistically characterize the of the previous week with observed fluctuations, and the ensemble
timeline of the stochastic introductions of the Alpha variant into the forecast, aggregating predictions from all modeling teams from the
US. This timeline is particularly important as it provides a day-by-day Forecasting Hub (Cramer et al., 2022). Both reference models focus
count of individuals traveling from various international locations to on four-weeks ahead predictions. We calculate the WIS for our weekly
US entry points that is used at run time by the LEAM-US model as it model projected incident deaths during the first six weeks of the
simulates the dynamic of the Alpha wave. projection period (from May 8 to June 19, 2021) for each state in the
US and the District of Columbia. We compare these WIS scores with the
3. Results WIS scores of the baseline and ensemble forecasting models from the
COVID-19 Forecast Hub. Weeks beyond this period are excluded due
Our multiscale model has been used to generate scenario projec- to the emergence of the Delta variant, which was not considered in
tions for all rounds of the Scenario Modeling Hub (SMH). Each round the scenario design. To compare the performance of the scenarios with
required modifications to the model to accommodate specific analy- the reference models, we compute a WIS ratio. This ratio is obtained
ses and variations in the epidemic landscape, such as the emergence by dividing the WIS of a given scenario and location by the WIS of
of new variants and changes in mitigation and vaccination policies. the corresponding reference model. A ratio smaller than one implies
This required adapting the model during the different scenario rounds a better performance of the projections with respect to the reference
to incorporate the mechanistic description of multiple co-circulating scenario (lower WIS). An inferior performance is indicated by a WIS
variants (up to 4 strains), waning vaccine efficacy (after round 7), ratio larger than one. The distribution of the WIS ratios of the scenario
and variations in key disease progression times. The Supplementary projections is presented in Fig. 2B and C for each analyzed region and
Information (SI) provides a narrative description of the model’s changes scenario, comparing them against the COVIDhub baseline and 4-weeks
over the 10 rounds, along with a summary table of key parameters used, ahead ensemble models. The WIS ratios indicate that the scenario
and an assessment of model performance. In the following, we focus on projections outperforms the naive baseline in all scenarios and performs
the results concerning the emergence of the Alpha variant in early 2021 comparably to the four-week ahead ensemble model. The median ratios
(round 5 projections). We will discuss the scenario assumptions and are well below one for the baseline model and close to one for the 4-
show how our multiscale modeling approach enables us to analyze the week ahead ensemble, suggesting similar performance for nearly half
introduction and spread of the alpha variant across the US, emphasizing the states performing better and the other half performing worse. No
the role of geographical heterogeneity. significant differences in performance are observed across scenarios,

5
M. Chinazzi et al. Epidemics 47 (2024) 100757

Fig. 3. (A) Out of sample model projections of weekly reported deaths for the US and selected states until June 28, 2022. The solid lines represent the median values, the darker
shaded regions the IQR and the lighter shaded regions the 95% reference range. (B) Out of sample model projections of weekly hospital admissions for the US. The solid lines
represent the median values, the darker shaded regions the IQR and the lighter shaded regions the 95% reference range. (C) 𝑅𝑡 estimates for the US and selected states. The solid
lines represent the median values and the lighter shaded regions the 95% reference range.

likely due to the relatively short assessment window of six weeks. average number of secondary infections generated by a single infected
Additional rounds of the SMH are evaluated for the most plausible individual at time 𝑡. The 𝑅𝑡 value is a useful metric because it is af-
scenarios in the SI. A comprehensive discussion of the performance fected by factors such as population immunity and behavioral changes
evaluation of scenario projections is provided in Howerton et al. (2023) (e.g., NPIs). In Fig. 3C we report the effective reproduction number
and in this issue (Bay et al., 2023). 𝑅𝑡 of each lineage, including the overall 𝑅𝑡 for the US and selected
In analyzing the performance of our models, it is however crucial states (see SI for all states). The 𝑅𝑡 was estimated using a Bayesian
to recognize that both the baseline and the four-weeks ahead forecast approach on the time series of the daily new infectious individuals
models are not naive in their design. These models undergo weekly for each lineage taken from the median estimates of the calibrated
revisions incorporating updates in surveillance data and changes in model (Zhang et al., 2020). We observed large heterogeneity’s across
contact and mobility levels. This iterative updating process sets them states in the behavior of the overall effective reproduction number.
apart from scenario projections. Unlike the forecast models, scenario As the more transmissible variant spreads, its prevalence, 𝑃 , defined
projections are based on a set of initial assumptions and do not adjust as the proportion of infections generated by that variant, increases
on the basis of new information gathered in the out-of-sample regime. which could result in an increase in the overall effective reproduction
number. However, other factors such as population immunity, vac-
3.2. The dynamics of the alpha variant cination prevalence, and NPIs could limit the disease burden of the
more transmissible lineage. Across the US we find a heterogeneous
To analyze the evolution of the Alpha variant across the US, we burden of the Alpha wave. It is also important to stress that a more
focus on the scenario assumptions of the high vaccination scenario transmissible variant is bound to become dominant even if the overall
and ensemble the moderate and low NPIs together, assuming a future number of cases is decreasing and the overall effective reproductive
decline in NPIs effectiveness ranging from 50% to 80%. These two number is smaller than one. This is evident for a number of states where
scenarios can be regarded a posteriori as the most plausible scenarios, the increase of the Alpha variant was not associated with a sustained
meaning they closely align with the actual occurrence. In Fig. 3A and increase in epidemic activity.
B we show the results of the out-of-sample projections for 7 weeks While a full mechanistic understanding of the dynamics of mul-
of the weekly number of deaths and hospitalizations for the US and tiple strains is beyond the scope of this study, it is possible to use
selected states (see SI for all states). In the figure, the out-of-sample a simple two-strain deterministic model with full cross-protection to
data are considered up to June 28, 2021, after that date the epidemic obtain the expression for the early growth of the prevalence of the more
trajectory shows the emergence of the Delta variant (lineage B1.617.2), transmissible strain as
which was not considered in the scenario design. Our projections align
𝑃 (𝑡) ≃ 𝑒𝜇𝜓𝑅𝑡̂(𝑡−𝑡0 ) , (3)
with the trajectories of the deaths and hospitalizations that capture the
decline of Alpha wave. where 𝑅𝑡̂ is the effective reproductive number of the dominant and
With a two-strain model we can distinguish between the infections less transmissible strain during the initial introduction and spread of
that are generated from the ancestral lineage and Alpha variant sep- the new variant (during the time window 𝑡̂) and is assumed to be
arately. Using the daily time series of new infections per lineage, we constant, 𝑡0 is the time of introduction, and 𝜇 is the generation time
can disentangle the contribution of each lineage to the effective repro- assumed to be the same for both strains (a full derivation of this result
duction number, 𝑅𝑡 . The effective reproduction number represents the and its assumptions are reported in the SI). This expression shows

6
M. Chinazzi et al. Epidemics 47 (2024) 100757

Fig. 4. (A) Weekly fraction of infections due to the Alpha variant for each state as a function of time for each state in the contiguous US. The black circles indicate the median
day the variant becomes dominant. The gray lines indicate the IQR and the white lines the 90% reference range. The triangles show when the variant became dominant for some
states according to the Helix data source (Helix, 2021). (B) Fraction of cases due to the Alpha variant over time for: California, Florida, and Michigan. The green line (median) and
the shaded areas (90%RR) are the results projected by our model. The orange circles are the reported Helix data and the orange line corresponds to the 5-day moving average.
(For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

that the emergence of a more transmissible strain’s dominance can be define the time of dominance as the date when the prevalence of the
highly variable across geographic regions, contingent upon the timing variant exceeds 50%; i.e. more than half of the new infections are due
of its introduction and the local effective reproductive numbers of the to the Alpha variant.Fig. 4A shows the weekly fraction of infections due
ancestral strain, which in turn depend on factors such as NPIs, residual to the Alpha variant over time. The results highlight the heterogeneous
immunity, and vaccination rates that vary among different states. paths towards dominance. The median estimates of the dominance
times span three months across the states. Our results are in agreement
3.3. The introduction and establishment of the alpha variant with previously published projections that found that the variant would
become dominant by the end of March 2021 (Davies et al., 2021;
In our study, we used a compartmental structure specifically de- Galloway et al., 2021; Washington et al., 2021).
signed as a two-strain model that intentionally excludes direct genomic To further validate our results, we use data from the The Helix
data integration. This decision was made to prioritize the validation COVID-19 Surveillance Dashboard (Helix, 2021) that is based on S-gene
of the simulated epidemiological dynamics, enabling the model to target failure. The data reported by this dashboard include the state of
effectively characterize the general behaviors of multiple viral strains residence, the date of collection of the sample, the number of positive
without relying on detailed genomic information. During the calibra- tests results, the number of positive tests results with S gene target
tion process, specific data on the growth and prevalence of the Alpha failure, the number of sequenced test results with S gene target failure,
variant were not incorporated. Remarkably, despite the absence of and the number of positive test results that were sequenced and known
direct genomic data, our model is capturing the prevalence trends of to be of the Alpha variant (for biases and limitations see Helix (2021)).
the Alpha variant over time. Indeed, the multiscale modeling approach By using these metrics, we can build a timeline of the prevalence of the
used here leverages the international travel patterns that drive the Alpha variant for each state reported in the dataset and compare it to
initial dispersion and introduction of the Alpha variant. Our results our estimates. In Fig. 4B, we compare the daily fraction of infections
show that the amount of international travel generated by the global due to the variant from our model (median and 90% reference range)
transportation network is strongly associated to the initial seeding with the data from Helix for three states: California, Florida, and
time of the Alpha lineage (see SI). However, the internal mobility Michigan. The surveillance data from Helix generally fall within the
and contact patterns at the county level, which are integrated into confidence interval of our model. However, for some states, we observe
the mechanistic structure of the multiscale model, highlight that the a plateauing after reaching dominance which deviates from our results.
local factors play a critical role in the spread of the Alpha variant as This is due to other strains like the Gamma (or P.1) and Delta (or
it competes with the ancestral lineage. This result parallels findings B.1.617.2) variants of concern increasing in prevalence, which are not
concerning the heterogeneities in the initial introduction of SARS-CoV- included in our modeling scheme. In the SI, we show a comparison for
2 to the US during the beginning of the COVID-19 pandemic (Davis all states reported by Helix with a statistically significant number of
et al., 2021). The heterogeneities found here go beyond the simple samples.
expression reported in Eq. (3). Therefore, to study in detail the path We leverage the resolution of our model to study combined statisti-
to dominance of the Alpha variant across the US, we calculate time- cal areas (CSA, 2020). Our results at a higher geographical resolution
varying prevalence of the Alpha lineage according to our model. We confirm that the heterogeneity in reaching dominance is not only

7
M. Chinazzi et al. Epidemics 47 (2024) 100757

Fig. 5. (A) The fraction of Alpha variant infections during 4 different weeks across the US for all CSAs. (B) The time to dominance for selected states and their CSAs. The dark
green triangles indicate the median date the variant becomes dominant in a given state and the dark (light) gray bars indicate the IQR (90% reference range). The light green
circles indicate the median date the variant becomes dominant in a given CSA that is a part of that state. (For interpretation of the references to color in this figure legend, the
reader is referred to the web version of this article.)

present at the state level but also when we look within a state. In The results concerning the introduction of the Alpha variant in
Fig. 5A we show the dynamics of the prevalence of the Alpha variant the US indicate that the importation events were both temporally and
across 4 selected weeks in early 2021. In early March (epiweek 2021– spatially heterogeneous and determined by the source location’s con-
09), most CSAs have either no detections or a less than 25% prevalence nectivity in the global transportation network. The initial importation
of the Alpha variant according to the model except for a few high-traffic events and the prevalence of the more transmissible Alpha variant
regions such as New York-Newark, NY-NJ-CT-PA, Chicago-Naperville, progressed differently across various locations due to the changes in
IL-IN-WI, and Miami-Port St. Lucie–Fort Lauderdale, FL Springs, GA-AL. mobility patterns, the distribution of population, and the strength
Zooming in, in Fig. 5B, for states containing multiple CSAs, we find of NPIs. The initial importation of variants into different regions of
high intra-state heterogeneity with respect to the time of dominance of the US are linked to the global airline traffic determining the entry
the Alpha variant. The results show that the heterogeneity is not only points and early spread patterns of the virus. Furthermore, international
observed at the state level but also at CSA level. Interestingly, across all transportation hubs generally resides in areas with high population
eight states, the week marking the dominance of the Alpha variant in densities with densely interconnected local mobility (commuting) net-
several of their CSAs is outside (and mostly occurring after the median) works. These networks in their turn contribute to the disease spread
the 90% reference range computed at the state level. However, some in nearby regions. Finally, the strength and adherence to NPIs also
CSAs anticipate the state median. This is the case for Miami and New varied considerably, further contributing to the heterogeneous dynamic
York. These two cities in particular are the location of two important of variants. Specifically our model indicates that these factors led to
international port-of-entries in the US that are associated with a large
considerable differences in the time when the Alpha variant became
incoming flux of travelers as they have the first and third largest traffic
the dominant strain, ranging across states from March to May, 2021.
volume in the US, respectively.
Leveraging the resolution of the model we also studied results at the
level of CSAs. In doing so, we uncover high heterogeneities even within
4. Discussion
states. CSAs featuring high mobility fluxes and populations experienced
an early growth of infections caused by the new variant with respect to
As of March, 2023, the multiscale model presented here has been
less populated and more secluded areas (when considering within state
used to submit 17 rounds of projections to the SMH. Our approach has
results). This is evident in the contrast between international travel
undergone many changes to adapt to the scenario specifications and
hubs, where the Alpha variant dominance was noted as early as March,
variations in the epidemiological landscape. The model’s calibration
time window has also varied based on the SMH coordinating team’s and more isolated regions, which saw a later dominance in mid-August.
direction. Despite these changes, the basic geographical structure and Like all modeling approaches, our multiscale model has limitations
resolution of the model have been maintained. Further details on how and requires specific assumptions. Although two geographical levels
the epidemic transmission model and other parameters have changed of analysis are considered, there could be heterogeneity in the tim-
can be found in the SI. Additionally, we report the performance of our ing of variant establishment at even smaller scales. Moreover, when
model across 10 rounds by measuring the WIS for the projection period projecting scenarios, it is often challenging to obtain accurate informa-
and calculating the ratio between the scores of two reference COVID- tion about the growth advantage of emerging variants, which can be
19 ensemble forecasting models. It is important to note that the initial attributed to increased transmissibility and/or immune escape (Volz,
conditions of the model were developed as scenarios and not with the 2023). Assumptions about how to handle this growth advantage at the
goal of forecasting. The scenario projections are also analyzed over mechanistic level can generate different results on long-term projec-
a longer time window, unlike the COVID-19 Forecasting Hub models tions. Additionally, changes in characteristic times such as the genera-
which only forecast a maximum of 4 weeks ahead. For a full assessment tion time, which are not always available at the moment of estimating
of all rounds and models submitted to the SMH we refer the reader the impact of an emerging variant, can also contribute to uncertainty.
to Howerton et al. (2023). Furthermore, scenario modeling requires assumptions about vaccine

8
M. Chinazzi et al. Epidemics 47 (2024) 100757

uptake, as well as changes in pathogen transmissibility due to popula- References

tion behavioral changes. Therefore, scenario projections should not be
considered as a forecast of the epidemic’s future trajectory but rather Balcan, D., Colizza, V., Gonçalves, B., Hu, H., Ramasco, J.J., Vespignani, A., 2009.
Multiscale mobility networks and the spatial spreading of infectious diseases.
an attempt to bound possible future trajectories based on different
Proc. Natl. Acad. Sci. 106 (51), 21484–21489. http://dx.doi.org/10.1073/pnas.
assumptions. 0906910106.
Although the results presented here focused on a particular variant, Balcan, D., Gonçalves, B., Hu, H., Ramasco, J.J., Colizza, V., Vespignani, A., 2010.
the methodology can be extended to study how other, more transmis- Modeling the spatial spread of infectious diseases: The global epidemic and mobility
computational model. J. Comput. Sci. 1 (3), 132–145. http://dx.doi.org/10.1016/
sible strains can spread quickly, take over the share of new infections,
j.jocs.2010.07.002.
and drastically alter the epidemic trajectory even during a successful Balcan, D., Vespignani, A., 2011. Phase transitions in contagion processes mediated
vaccine rollout. While modeling approaches cannot replace ground by recurrent mobility patterns. Nat. Phys. 7, 581–586. http://dx.doi.org/10.1038/
truth data, mechanistic modeling frameworks can complement genomic nphys1944.
Bay, C., St-Onge, G., Davis, J.T., Chinazzi, M., Howerton, E., Lessler, J., Runge, M.C.,
surveillance efforts to track the unfolding of variants of concern and
Shea, K., Truelove, S., Viboud, C., Vespignani, A., 2023. Ensemble2: Scenarios
model their introduction, establishment, and path to dominance at a ensembling for communication and performance analysis. Epidemics 46, 100748.
fine-grained geographical scale. http://dx.doi.org/10.1016/j.epidem.2024.100748.
Beaumont, M.A., Zhang, W., Balding, D.J., 2002. Approximate Bayesian computation
Code availability in population genetics. Genetics 162 (4), 2025–2035, https://www.genetics.org/
content/162/4/2025.
Biggerstaff, M., Slayton, R.B., Johansson, M.A., Butler, J.C., 2022. Improving pandemic
The GLEAM model is publicly available at http://www.gleamviz. response: Employing mathematical modeling to confront Coronavirus disease 2019.
org/. Clin. Infect. Dis. 74 (5), 913–917. http://dx.doi.org/10.1093/cid/ciab673.
Borchering, R.K., Mullany, L.C., Howerton, E., Chinazzi, M., Smith, C.P., Qin, M.,
Reich, N.G., Contamin, L., Levander, J., Kerr, J., Espino, J., Hochheiser, H.,
CRediT authorship contribution statement
Lovett, K., Kinsey, M., Tallaksen, K., Wilson, S., Shin, L., Lemaitre, J.C., Hulse, J.D.,
Kaminsky, J., Lee, E.C., Hill, A.L., Davis, J.T., Mu, K., Xiong, X., Piontti, A.P.y.,
Matteo Chinazzi: Data curation, Investigation, Writing – original Vespignani, A., Srivastava, A., Porebski, P., Venkatramanan, S., Adiga, A., Lewis, B.,
draft. Jessica T. Davis: Data curation, Investigation, Writing – original Klahn, B., Outten, J., Hurt, B., Chen, J., Mortveit, H., Wilson, A., Marathe, M.,
Hoops, S., Bhattacharya, P., Machi, D., Chen, S., Paul, R., Janies, D., Thill, J.-C.,
draft. Ana Pastore y Piontti: Data curation, Investigation, Writing –
Galanti, M., Yamana, T., Pei, S., Shaman, J., España, G., Cavany, S., Moore, S.,
original draft. Kunpeng Mu: Data curation, Investigation, Writing – Perkins, A., Healy, J.M., Slayton, R.B., Johansson, M.A., Biggerstaff, M., Shea, K.,
original draft. Nicolò Gozzi: Investigation, Writing – original draft. Truelove, S.A., Runge, M.C., Viboud, C., Lessler, J., 2023. Impact of SARS-CoV-2
Marco Ajelli: Investigation, Writing – original draft. Nicola Perra: vaccination of children ages 5–11 years on COVID-19 disease burden and resilience
Investigation, Writing – original draft. Alessandro Vespignani: Data to new variants in the united states, november 2021–march 2022: A multi-model
study. Lancet Regional Health – Am. 17, 100398. http://dx.doi.org/10.1016/j.lana.
curation, Investigation, Writing – original draft.
2022.100398.
Borchering, R.K., Viboud, C., Howerton, E., et al., 2021. Modeling of future COVID-19
Declaration of competing interest cases, hospitalizations, and deaths, by vaccination rates and nonpharmaceutical
intervention scenarios — United states, april–september 2021. MMWR. Mor-
bidity Mortality Weekly Rep. 70, 719–724. http://dx.doi.org/10.15585/mmwr.
The authors declare that they have no known competing finan-
mm7019e3.
cial interests or personal relationships that could have appeared to Bracher, J., Ray, E.L., Gneiting, T., Reich, N.G., 2021. Evaluating epidemic forecasts
influence the work reported in this paper. in an interval format. PLoS Comput. Biol. 17 (2), e1008618. http://dx.doi.org/10.
1371/journal.pcbi.1008618.
Brooks-Pollock, E., Danon, L., Jombart, T., Pellis, L., 2021. Modelling that shaped the
Data availability
early COVID-19 pandemic response in the UK. Philos. Trans. R. Soc. B 376 (1829),
20210001. http://dx.doi.org/10.1098/rstb.2021.0001.
Epidemic surveillance data were collected from the Johns Hopkins CDC, 2021 COVID-19 Vaccinations in the United States. https://covid.cdc.gov/covid-
Coronavirus Resource Center https://coronavirus.jhu.edu/. Proprietary data-tracker/vaccinations_vacc-total-admin-rate-total".
Chinazzi, M., Davis, J.T., Ajelli, M., Gioannini, C., Litvinova, M., Merler, S., y Pio-
airline data are commercially available from OAG (https://www.oag.
ntti, A.P., Mu, K., Rossi, L., Sun, K., et al., 2020. The effect of travel restrictions
com/) and IATA (https://www.iata.org/) databases. Other model in- on the spread of the 2019 novel Coronavirus (COVID-19) outbreak. Science 368
tervention data includes Google’s COVID-19 Community Mobility Re- (6489), 395–400. http://dx.doi.org/10.1126/science.aba9757.
ports available at https://www.google.com/covid19/mobility/ and the COVID-19 Scenario Modeling Hub, 2021. Round 5 scenario specifications.
Oxford COVID-19 Response Tracker available at https://github.com/ https://github.com/midas-network/covid19-scenario-modeling-hub/blob/master/
previous-rounds/README_Round5.md.
OxCGRT/covid-policy-tracker. Cramer, E.Y., Huang, Y., Wang, Y., Ray, E.L., Cornell, M., Bracher, J., Brennen, A.,
Rivadeneira, A.J.C., Gerding, A., House, K., Jayawardena, D., Kanji, A.H., Khandel-
Acknowledgments wal, A., Le, K., Mody, V., Mody, V., Niemi, J., Stark, A., Shah, A., Wattanchit, N.,
Zorn, M.W., Reich, N.G., 2022. The united states COVID-19 forecast hub dataset.
Sci. Data 9 (1), 462. http://dx.doi.org/10.1038/s41597-022-01517-w.
MC, MA, JTD, KM, APP and AV acknowledge support from COVID
CSA, 2020. Combined Statistical Areas. https://www.census.gov/geographies/reference-
Supplement CDC-HHS-6U01IP001137-01, and the Cooperative Agree- maps/2020/geo/csa.html. (Accessed 27 May 2021).
ment no. NU38OT000297 from the Council of State and Territorial Davies, N.G., Abbott, S., Barnard, R.C., Jarvis, C.I., Kucharski, A.J., Munday, J.D.,
Epidemiologists (CSTE) and the CDC-75D301Cl4810 contract. MC ac- Pearson, C.A., Russell, T.W., Tully, D.C., Washburne, A.D., et al., 2021. Estimated
transmissibility and impact of SARS-CoV-2 lineage B. 1.1. 7 in England. Science
knowledges support from CDC-JHU-2005702123. MC and AV acknowl-
372 (6538), http://dx.doi.org/10.1126/science.abg3055, eabg3055.
edge support from Google Cloud and Google Cloud Research Credits Davis, J.T., Chinazzi, M., Perra, N., Mu, K., Pastore y Piontti, A., Ajelli, M., Dean, N.E.,
program to fund this project. The findings and conclusions in this study Gioannini, C., Litvinova, M., Merler, S., Rossi, L., Sun, K., Xiong, X., Longini, I.M.,
are those of the authors and do not necessarily represent the official Halloran, M., Viboud, C., Vespignani, A., 2021. Cryptic transmission of SARS-CoV-2
position of the funding agencies, the National Institutes of Health, and the first COVID-19 wave. Nature 600 (7887), 127–132. http://dx.doi.org/10.
1038/s41586-021-04130-w.
United States, or the U.S. Department of Health and Human Services, Dong, E., Du, H., Gardner, L., 2020. An interactive web-based dashboard to track
United States. COVID-19 in real time. Lancet Infect. Dis. 20 (5), 533–534. http://dx.doi.org/10.
1016/S1473-3099(20)30120-1.
Appendix A. Supplementary data Dooling, K., McClung, N., Chamberland, M., et al., 2020. The advisory committee on
immunization practices’ interim recommendation for allocating initial supplies of
COVID-19 vaccine — United states, 2020. MMWR. Morbidity Mortality Weekly
Supplementary material related to this article can be found online Rep. 69, 1857–1859. http://dx.doi.org/10.15585/mmwr.mm6949e1, URL https:
at https://doi.org/10.1016/j.epidem.2024.100757. //www.cdc.gov/mmwr/volumes/69/wr/mm6949e1.htm.

9
M. Chinazzi et al. Epidemics 47 (2024) 100757

Galloway, S.E., Paul, P., MacCannell, D.R., Johansson, M.A., Brooks, J.T., MacNeil, A., Modeling for Emerging and Re-Emerging Infectious Diseases. Springer International
Slayton, R.B., Tong, S., Silk, B.J., Armstrong, G.L., et al., 2021. Emergence of Publishing, Cham, pp. 39–56. http://dx.doi.org/10.1007/978-3-319-40413-4_4.
SARS-CoV-2 b.1.1.7 lineage—United States, december 29, 2020–january 12, 2021. Polack, F.P., Thomas, S.J., Kitchin, N., Absalon, J., Gurtman, A., Lockhart, S.,
Morbidity Mortality Weekly Rep. 70 (3), 95. http://dx.doi.org/10.15585/mmwr. Perez, J.L., Pérez Marc, G., Moreira, E.D., Zerbini, C., Bailey, R., Swanson, K.A.,
mm7003e2. Roychoudhury, S., Koury, K., Li, P., Kalina, W.V., Cooper, D., Frenck, R.W.,
Gneiting, T., Raftery, A.E., 2007. Strictly proper scoring rules, prediction, Hammitt, L.L., Türeci, Ö., Nell, H., Schaefer, A., Ünal, S., Tresnan, D.B., Mather, S.,
and estimation. J. Amer. Statist. Assoc. 102 (477), 359–378. http://dx. Dormitzer, P.R., Şahin, U., Jansen, K.U., Gruber, W.C., 2020. Safety and efficacy
doi.org/10.1198/016214506000001437, Publisher: Taylor & Francis _eprint: of the BNT162b2 mRNA Covid-19 vaccine. N. Engl. J. Med. 383 (27), 2603–2615.
10.1198/016214506000001437. http://dx.doi.org/10.1056/NEJMoa2034577.
Gomes, M.F.C., Pastore y Piontti, A., Rossi, L., Chao, D., Longini, I., Halloran, M.E., Poletto, C., Scarpino, S.V., Volz, E.M., 2020. Applications of predictive modelling
Vespignani, A., 2014. Assessing the international spreading risk associated with early in the COVID-19 epidemic. Lancet. Digit. Health 2 (10), e498–e499. http://
the 2014 west African Ebola outbreak. PLoS Currents 6, http://dx.doi.org/10. dx.doi.org/10.1016/S2589-7500(20)30196-5, URL https://www.ncbi.nlm.nih.gov/
1371/currents.outbreaks.cd818f63d40e24aef769dda7df9e0da5, URL https://www. pmc/articles/PMC7417175/.
ncbi.nlm.nih.gov/pmc/articles/PMC4169359/. Prem, K., Cook, A.R., Jit, M., 2017. Projecting social contact matrices in 152 countries
Google LLC, 2021a. Google COVID-19 community mobility reports. https://www. using contact surveys and demographic data. PLoS Comput. Biol. 13 (9), e1005697.
google.com/covid19/mobility/. http://dx.doi.org/10.1371/journal.pcbi.1005697.
Google LLC, 2021b. Google COVID-19 community mobility reports: Mobil- Rambaut, A., Loman, N., Pybus, O., Barclay, W., Barrett, J., Carabelli, A., Connor, T.,
ity report CSV documentation. https://www.google.com/covid19/mobility/data_ Peacock, T., Robertson, D.L., Volz, E., COVID-19 Genomics Consortium
documentation.html?hl=en. UK (CoG-UK), 2020. Preliminary genomic characterisation of an emergent
Hale, T., Angrist, N., Goldszmidt, R., Kira, B., Petherick, A., Phillips, T., Webster, S., SARS-CoV-2 lineage in the UK defined by a novel set of spike mutations.
Cameron-Blake, E., Hallas, L., Majumdar, S., Tatlow, H., 2021. A global panel https://virological.org/t/preliminary-genomic-characterisation-of-an-emergent-
database of pandemic policies (Oxford COVID-19 government response tracker). sars-cov-2-lineage-in-the-uk-defined-by-a-novel-set-of-spike-mutations/563.
Nat. Hum. Behav. 5 (4), 529–538. http://dx.doi.org/10.1038/s41562-021-01079-8. Reich, N.G., Lessler, J., Funk, S., Viboud, C., Vespignani, A., Tibshirani, R.J., Shea, K.,
Helix, 2021. Data sourced from the Helix® COVID-19 surveillance dashboard. https: Schienle, M., Runge, M.C., Rosenfeld, R., Ray, E.L., Niehus, R., Johnson, H.C.,
//www.helix.com/pages/helix-covid-19-surveillance-dashboard. (Accessed 18 May Johansson, M.A., Hochheiser, H., Gardner, L., Bracher, J., Borchering, R.K.,
2021). Biggerstaff, M., 2022. Collaborative hubs: Making the most of predictive epidemic
Holmdahl, I., Buckee, C., 2020. Wrong but useful — What Covid-19 epidemiologic modeling. Am J Public Health 112 (6), 839–842. http://dx.doi.org/10.2105/AJPH.
models can and cannot tell us. N. Engl. J. Med. 383 (4), 303–305. http://dx.doi. 2022.306831.
org/10.1056/NEJMp2016822. Salje, H., Kiem, C.T., Lefrancq, N., Courtejoie, N., Bosetti, P., Paireau, J., Andron-
Howerton, E., Contamin, L., Mullany, L.C., Qin, M., Reich, N.G., Bents, S., Borcher- ico, A., Hozé, N., Richet, J., Dubost, C.-L., Strat, Y.L., Lessler, J., Levy-Bruhl, D.,
ing, R.K., Jung, S.-m., Loo, S.L., Smith, C.P., Levander, J., Kerr, J., Espino, J., Fontanet, A., Opatowski, L., Boelle, P.-Y., Cauchemez, S., 2020. Estimating the
Panhuis, W.G.v., Hochheiser, H., Galanti, M., Yamana, T., Pei, S., Shaman, J., burden of SARS-CoV-2 in France. Science 369 (6500), 208–211. http://dx.doi.org/
Rainwater-Lovett, K., Kinsey, M., Tallaksen, K., Wilson, S., Shin, L., Lemaitre, J.C., 10.1126/science.abc3517.
Kaminsky, J., Hulse, J.D., Lee, E.C., McKee, C., Hill, A., Karlen, D., Chinazzi, M., Scenario Modeling Hub, 2023. COVID-19 scenario modeling hub. https:
Davis, J.T., Mu, K., Xiong, X., Piontti, A.P.y., Vespignani, A., Rosenstrom, E.T., //covid19scenariomodelinghub.org/.
Ivy, J.S., Mayorga, M.E., Swann, J.L., España, G., Cavany, S., Moore, S., Perkins, A., Science Magazine, 2020. Mutant Coronavirus in the United Kingdom sets off alarms,
Hladish, T., Pillai, A., Toh, K.B., Longini, I., Chen, S., Paul, R., Janies, D., Thill, J.- but its importance remains unclear. https://www.sciencemag.org/news/2020/12/
C., Bouchnita, A., Bi, K., Lachmann, M., Fox, S., Meyers, L.A., Consortium, U.C.-.M., mutant-coronavirus-united-kingdom-sets-alarms-its-importance-remains-unclear.
Srivastava, A., Porebski, P., Venkatramanan, S., Adiga, A., Lewis, B., Klahn, B., (Online; Accessed 21 December 2020).
Outten, J., Hurt, B., Chen, J., Mortveit, H., Wilson, A., Marathe, M., Hoops, S., Bhat- Shapiro, J., Dean, N.E., Madewell, Z.J., Yang, Y., Halloran, M.E., Longini, I., 2021.
tacharya, P., Machi, D., Cadwell, B.L., Healy, J.M., Slayton, R.B., Johansson, M.A., Efficacy estimates for various COVID-19 vaccines: What we know from the
Biggerstaff, M., Truelove, S., Runge, M.C., Shea, K., Viboud, C., Lessler, J., 2023. literature and reports. http://dx.doi.org/10.1101/2021.05.20.21257461, medRxiv.
Evaluation of the us covid-19 scenario modeling hub for informing pandemic URL https://www.medrxiv.org/content/10.1101/2021.05.20.21257461v2.
response under uncertainty. Nature communications 14 (1), 7260. http://dx.doi. Sunnåker, M., Busetto, A.G., Numminen, E., Corander, J., Foll, M., Dessimoz, C.,
org/10.1038/s41467-023-42680-x. 2013. Approximate Bayesian computation. PLoS Comput. Biol. 9 (1), 1–10. http:
Jewell, N.P., Lewnard, J.A., Jewell, B.L., 2020. Predictive mathematical models of the //dx.doi.org/10.1371/journal.pcbi.1002803.
COVID-19 pandemic: Underlying principles and value of projections. JAMA 323 Truelove, S., Smith, C.P., Qin, M., Mullany, L.C., Borchering, R.K., Lessler, J., Shea, K.,
(19), 1893–1894. http://dx.doi.org/10.1001/jama.2020.6585. Howerton, E., Contamin, L., Levander, J., Kerr, J., Hochheiser, H., Kinsey, M.,
Mistry, D., Litvinova, M., Chinazzi, M., Fumanelli, L., Gomes, M.F., Haque, S.A., Tallaksen, K., Wilson, S., Shin, L., Rainwater-Lovett, K., Lemairtre, J.C., Dent, J.,
Liu, Q.-H., Mu, K., Xiong, X., Halloran, M.E., et al., 2021. Inferring high-resolution Kaminsky, J., Lee, E.C., Perez-Saez, J., Hill, A., Karlen, D., Chinazzi, M., Davis, J.T.,
human mixing patterns for disease modeling. Nature Commun. 12 (1), 323. http: Mu, K., Xiong, X., Pastore y Piontti, A., Vespignani, A., Srivastava, A., Porebski, P.,
//dx.doi.org/10.1038/s41467-020-20544-y. Venkatramanan, S., Adiga, A., Lewis, B., Klahn, B., Outten, J., Orr, M., Harrison, G.,
NERVTAG, 2020. NERVTAG meeting on SARS-CoV-2 variant under investigation Hurt, B., Chen, J., Vullikanti, A., Marathe, M., Hoops, S., Bhattacharya, P.,
VUI-202012/01. (Online; Accessed 21 December 2020). Machi, D., Chen, S., Paul, R., Janies, D., Thill, J.-C., Galanti, M., Yamana, T.K.,
OAG, 2020. Official aviation guide. https://www.oag.com/. Pei, S., Shaman, J.L., Healy, J.M., Slayton, R.B., Biggerstaff, M., Johansson, M.A.,
Our World in Data, 2021, State-by-state data on COVID-19 vaccinations in the United Runge, M.C., Viboud, C., 2022. Projected resurgence of COVID-19 in the United
States. https://ourworldindata.org/us-states-vaccinations. States in july—December 2021 resulting from the increased transmissibility of
PHE, 2021. Investigation of novel SARS-CoV-2 variant. Variant of concern 202012/01. the delta variant and faltering vaccination. eLife 11, e73584. http://dx.doi.org/
Technical briefing 3. https://assets.publishing.service.gov.uk/government/uploads/ 10.7554/eLife.73584.
system/uploads/attachment_data/file/950823/Variant_of_Concern_VOC_202012_01_ US Census Bureau, 2024, 2011–2015 5-Year ACS Commuting Flows. https://www.
Technical_Briefing_3_-_England.pdf. (Online; Accessed 13 January 2021). census.gov/data/tables/2015/demo/metro-micro/commuting-flows-2015.html.
Pilishvili, T., Fleming-Dutra, K.E., Farrar, J.L., Gierke, R., Mohr, N.M., Talan, D.A., US Department of Health & Human Services, 2021, COVID-19 Reported Patient Impact
Krishnadasan, A., Harland, K.K., Smithline, H.A., Hou, P.C., Lee, L.C., Lim, S.C., and Hospital Capacity by State Timeseries. https://healthdata.gov/Hospital/COVID-
Moran, G.J., Krebs, E., Steele, M., Beiser, D.G., Faine, B., Haran, J.P., Nandi, U., 19-Reported-Patient-Impact-and-Hospital-Capa/g62h-syeh.
Schrading, W.A., Chinnock, B., Henning, D.J., LoVecchio, F., Nadle, J., Barter, D., Verity, R., Okell, L.C., Dorigatti, I., Winskill, P., Whittaker, C., Imai, N., Cuomo-
Brackney, M., Britton, A., Marceaux-Galli, K., Lim, S., Phipps, E.C., Dumyati, G., Dannenburg, G., Thompson, H., Walker, P.G.T., Fu, H., Dighe, A., Griffin, J.T.,
Pierce, R., Markus, T.M., Anderson, D.J., Debes, A.K., Lin, M., Mayer, J., Bab- Baguelin, M., Bhatia, S., Boonyasiri, A., Cori, A., Cucunubá, Z., FitzJohn, R.,
cock, H.M., Safdar, N., Fischer, M., Singleton, R., Chea, N., Magill, S.S., Verani, J., Gaythorpe, K., Green, W., Hamlet, A., Hinsley, W., Laydon, D., Nedjati-Gilani, G.,
Schrag, S., Team, V.E.A.H.P.S., 2021. Interim estimates of vaccine effectiveness of Riley, S., van Elsland, S., Volz, E., Wang, H., Wang, Y., Xi, X., Donnelly, C.A.,
Pfizer-BioNTech and moderna COVID-19 vaccines among health care personnel — Ghani, A.C., Ferguson, N.M., 2020. Estimates of the severity of Coronavirus disease
33 U.S. sites, january–march 2021. MMWR Weekly 70, 753–758. http://dx.doi.org/ 2019: A model-based analysis. Lancet Infect. Dis. 20 (6), 669–677. http://dx.doi.
10.15585/mmwr.mm7020e2externalicon, On May 14, 2021, this report was posted org/10.1016/S1473-3099(20)30243-7.
online as an MMWR Early Release. This report has been corrected. Volz, E., 2023. Fitness, growth and transmissibility of SARS-CoV-2 genetic variants.
Pastore y Piontti, A., Perra, N., Rossi, L., Samay, N., Vespignani, A., 2018. Charting Nature Rev. Genet. 24 (10), 724–734. http://dx.doi.org/10.1038/s41576-023-
the Next Pandemic: Modeling Infectious Disease Spreading in the Data Science Age. 00610-z.
Springer, http://dx.doi.org/10.1007/978-3-319-93290-3. Walensky, R.P., Walke, H.T., Fauci, A.S., 2021. SARS-CoV-2 variants of concern in the
Pastore y Piontti, A., Zhang, Q., Gomes, M.F.C., Rossi, L., Poletto, C., Colizza, V., United States—Challenges and opportunities. JAMA 325 (11), 1037–1038.
Chao, D.L., Longini, I.M., Halloran, M.E., Vespignani, A., 2016. Real-time assess- Washington, N.L., Gangavarapu, K., Zeller, M., Bolze, A., Cirulli, E.T., Barrett, K.M.S.,
ment of the international spreading risk associated with the 2014 west African Larsen, B.B., Anderson, C., White, S., Cassens, T., Jacobs, S., Levan, G., Nguyen, J.,
Ebola outbreak. In: Chowell, G., Hyman, J.M. (Eds.), Mathematical and Statistical Ramirez, J.M., Rivera-Garcia, C., Sandoval, E., Wang, X., Wong, D., Spencer, E.,

10
M. Chinazzi et al. Epidemics 47 (2024) 100757

Robles-Sikisaka, R., Kurzban, E., Hughes, L.D., Deng, X., Wang, C., Servellita, V., Zhang, J., Litvinova, M., Wang, W., Wang, Y., Deng, X., Chen, X., Li, M., Zheng, W.,
Valentine, H., Hoff, P.D., Seaver, P., Sathe, S., Gietzen, K., Sickler, B., Antico, J., Yi, L., Chen, X., Wu, Q., Liang, Y., Wang, X., Yang, J., Sun, K., Longini, I.M.,
Hoon, K., Liu, J., Harding, A., Bakhtar, O., Basler, T., Austin, B., MacCannell, D., Halloran, M.E., Wu, P., Cowling, B.J., Merler, S., Viboud, C., Vespignani, A.,
Isaksson, M., Febbo, P.G., Becker, D., Laurent, M., McDonald, E., Yeo, G.W., Ajelli, M., Yu, H., 2020. Evolving epidemiology and transmission dynamics of
Knight, R., Laurent, L.C., Feo, E.d., Worobey, M., Chiu, C.Y., Suchard, M.A., Coronavirus disease 2019 outside Hubei province, China: A descriptive and mod-
Lu, J.T., Lee, W., Andersen, K.G., 2021. Emergence and rapid transmission of elling study. Lancet Infect. Dis. 20 (7), 793–802. http://dx.doi.org/10.1016/S1473-
SARS-CoV-2 B.1.1.7 in the united states. Cell 184 (10), 2587–2594.e7. http:// 3099(20)30230-9.
dx.doi.org/10.1016/j.cell.2021.03.052, Publisher: Elsevier. URL https://www.cell. Zhang, Q., Sun, K., Chinazzi, M., Pastore Y Piontti, A., Dean, N.E., Rojas, D.P.,
com/cell/abstract/S0092-8674(21)00383-4. Merler, S., Mistry, D., Poletti, P., Rossi, L., Bray, M., Halloran, M.E., Longini, I.M.,
WHO, 2020. SARS-CoV-2 variant – United Kingdom of great Britain and North- Vespignani, A., 2017. Spread of Zika virus in the Americas. Proc. Natl. Acad. Sci.
ern Ireland. https://www.who.int/csr/don/21-december-2020-sars-cov2-variant- 114 (22), http://dx.doi.org/10.1073/pnas.1620161114.
united-kingdom/en/. (Online; Accessed 21 December 2020).
World Health Organization, 2021. Tracking SARS-CoV-2 variants. https://www.who.
int/en/activities/tracking-SARS-CoV-2-variants/. (Online; Accessed 24 June 2021).

11