A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loops

Abstract

High-quality data is essential for training large generative models, yet the vast reservoir of real data available online has become nearly depleted. Consequently, models increasingly generate their own data for further training, forming Self-consuming Training Loops (STLs). However, the empirical results have been strikingly inconsistent: some models degrade or even collapse, while others successfully avoid these failures, leaving a significant gap in theoretical understanding to explain this discrepancy. This paper introduces the intriguing notion of recursive stability and presents the first theoretical generalization analysis, revealing how both model architecture and the proportion between real and synthetic data influence the success of STLs. We further extend this analysis to transformers in in-context learning, showing that even a constant-sized proportion of real data ensures convergence, while also providing insights into optimal synthetic data sizing.

1 Introduction

The quest of high-quality data is paramount in the training of generative artificial intelligence (AI), such as large language models (LLMs). However, the vast reservoir of publicly available data on the internet has nearly been exhausted (Villalobos et al., 2022), pushing the research community to seek innovative yet plausible solutions. One promising approach is to train the next generation of LLMs using synthetic data generated by earlier generations of the models themselves (Briesch et al., 2023). Additionally, reliance on synthetic data has become almost unavoidable, as many existing datasets are already polluted with synthetic content (Schuhmann et al., 2022), which proves difficult to detect reliably (Sadasivan et al., 2023). This has led to the development of Self-consuming Training Loops (STLs), as illustrated in Figure 1, where generative models are recursively trained on a mix of real and synthetic data generated by the models themselves. In theory, these STLs of data creation and refinement could propel models to new levels of capability, reducing reliance on external datasets.

However, despite their potential, the empirical results of STLs have been highly inconsistent across studies (Shumailov et al., 2024; Alemohammad et al., 2024a; Xing et al., 2025; Dohmatob et al., 2024b). Some studies (Shumailov et al., 2024) have encountered significant setbacks—certain models have shown signs of stagnation, failing to improve or adapt, while others have even regressed, leading to sharp declines in performance. Conversely, other works (Gerstgrasser et al., 2024; Gillman et al., 2024; Alemohammad et al., 2024b; Ferbach et al., 2024) have successfully avoided model collapse by incorporating sufficient real data, augmenting with synthetic data, or introducing guidance during the generation process. However, these observed phenomena lack thorough theoretical explanations.

When and how do STLs generalize effectively, thereby preventing model collapse from a theoretical perspective? Even among “refined” LLMs drawing from similar pools of model-generated data, the results vary significantly (Briesch et al., 2023; Fu et al., 2024a). These inconsistencies highlight the urgency of establishing theoretical guarantees for STLs by exploring the underlying mechanisms that determine when synthetic data generation either facilitates or impedes model development. Initial theoretical explorations have started to address these gaps. For instance, Shumailov et al. (2024) and Alemohammad et al. (2024a) demonstrated model collapse when models were trained exclusively on synthetic data, using simplified Gaussian models to illustrate this issue. In a more detailed theoretical study, Bertrand et al. (2024) derived upper bounds on parameter deviations for likelihood-based models in STLs, establishing convergence under strict statistical and optimization assumptions. Meanwhile, Fu et al. (2024b) relaxed these assumptions and provided bounds on the divergence between synthetic and real data distributions for a simplified diffusion model.

However, existing theoretical research lacks a unified framework and has yet to thoroughly investigate the generalization error of STLs. Additionally, current studies often overlook the role of model architectures in preventing model collapse. Moreover, the behavior of transformers within STLs remains largely unexamined, leaving significant theoretical gaps in the literature. Notably, there is limited exploration of the theoretical trade-offs introduced by synthetic data augmentation. This paper aims to address these gaps with the following contributions:

1. Theoretical Generalization Framework: This paper fills a gap in prior research by being the first to establish generalization error bounds. The key innovation, recursive stability, is introduced to address the core theoretical challenges, specifically the complex recursive structures and the non-i.i.d. nature of the data. Moreover, we demonstrate that the generalization error converges under the following conditions: (1) the generative model satisfies recursive stability, and (2) the proportion of real data is maintained at a non-negligible constant level, thus preventing model collapse.

2. Application to Transformers in In-Context Learning: This paper is the first to extend the theoretical framework to transformer models in in-context learning. We prove that transformers in this setting satisfy recursive stability with a constant-level proportion of real data, controlling output differences in STLs under small perturbations to the initial dataset. Consequently, we show that the generalization error is bounded by $\mathcal{O}(n^{-1}\log^{2}(n)+n^{-1/2}\log(n)+n^{-1/4})$.

3. Trade-off Analysis of Synthetic Data Augmentation: We investigate the trade-off in synthetic data augmentation. By employing decomposition techniques, we demonstrate that while synthetic data improves the generalization performance of each generation on mixed datasets, it concurrently exacerbates distribution divergence across successive generations. Our theoretical findings further show that the optimal size of synthetic augmentation increases as the size of real dataset decreases.

Figure 1: Self-consuming Training Loops: The initial model $\mathcal{G}_{0}$ is trained on the real dataset $S_{0}$. For generation $1\leq j\leq i$, the model $\mathcal{G}_{j}$ is trained on the mixed dataset $\widetilde{S}_{j}$.

2 Related Work

This section reviews STLs research and algorithmic stability studies.

Self-consuming Training Loops. Recent research has increasingly focused on generative models trained within STLs (Shumailov et al., 2024), with much of the analysis conducted from an empirical perspective (Martínez et al., 2023). For example, Shumailov et al. (2024); Briesch et al. (2023) observe a decline in diversity in language models when a portion of the model’s outputs is recursively used as inputs. Additionally, Wyllie et al. (2024) highlights that recursive training on synthetic data amplifies biases, resulting in significant fairness concerns. To mitigate model collapse, some studies suggest incorporating real data into the training process (Alemohammad et al., 2024a), expanding the size of synthetic datasets (Dohmatob et al., 2024c; Gerstgrasser et al., 2024; Dohmatob et al., 2024a; Feng et al., 2024b), or providing guidance during the generation process (Gillman et al., 2024; Alemohammad et al., 2024b; Feng et al., 2024a).

While empirical research has extensively explored STLs of generative models, theoretical studies on this process remain relatively sparse (Kanabar & Gastpar, 2025; Seddik et al., 2024; Marchi et al., 2024; Gerstgrasser et al., 2024; Zhu et al., 2024; Tao et al., 2024). Notably, Shumailov et al. (2024) and Alemohammad et al. (2024a) offer initial theoretical insights by analyzing a simplified Gaussian model. In a more comprehensive analysis, Bertrand et al. (2024) derive upper bounds on parameter deviations between those obtained within a STL and the optimal values, relying on assumptions about statistical and optimization error bounds. In contrast, Fu et al. (2024b) propose bounds on the divergence between synthetic and real-world data distributions, without such assumptions. However, current research lacks a unified theoretical framework that accounts for the influence of different model architectures and does not provide generalization error bounds for STLs, thus failing to rigorously establish the conditions that guarantee the prevention of model collapse. Furthermore, the behavior of transformers within STLs remains unexplored, leaving substantial theoretical gaps.

Algorithmic stability. Algorithmic stability ensures generalization bounds independent of model capacity. A key measure, uniform stability, was introduced by Bousquet & Elisseeff (2002) and has been instrumental in analyzing the generalization behavior of regularization methods. This measure was later extended to SGD (Hardt et al., 2016), including non-convex and non-smooth settings (Charles & Papailiopoulos, 2018; Bassily et al., 2020; Lei, 2023). Recent work shows that uniform stability can also provide near-optimal bounds with high probability (Feldman & Vondrak, 2019; Bousquet et al., 2020; Klochkov & Zhivotovskiy, 2021; Li & Liu, 2022; Wang et al., 2024).

Building on these foundations, recent research has focused on stability in more complex, non-i.i.d. settings. A common approach models data from stationary and mixing sequences (Doukhan, 1994; Yu, 1994), where weakening dependencies allow stability bounds through mixing coefficients (Mohri & Rostamizadeh, 2010; He et al., 2016; Fu et al., 2023). However, estimating these coefficients remains challenging. Additionally, some studies (Zheng et al., 2023) address non-i.i.d. data by leveraging conditional independence properties. Nonetheless, current methodologies struggle with the complexities of STLs, as the non-i.i.d. nature of mixed datasets, where each generation’s data is influenced by previous generations, presents unresolved challenges for stability frameworks.

Remark 1.

Building on previous challenges, our work advances this area by developing a more comprehensive theoretical framework for analyzing generative models within STLs. Specifically, we present the first generalization error bound by addressing the additional complexity arising from the non-i.i.d. nature of mixed datasets. To address this, we propose the key innovation of recursive stability, which quantifies error propagation across generations of synthetic data. Moreover, we are the first to extend this theoretical framework to transformers, explicitly utilizing error decomposition to illustrate the trade-off introduced by augmenting datasets with synthetic data.

3 Preliminaries

In this section, we begin by formally describing the training process of generative models in STLs, then introduce algorithmic stability with a focus on uniform stability, and finally define recursive stability to address the challenges specific to STLs.

3.1 Generative Models within Self-consuming Training Loops

Generative models have made significant strides in producing highly realistic data, such as images and text, which are frequently shared online and often indistinguishable from real content. Meanwhile, the supply of real data has nearly been exhausted. Consequently, deep generative models increasingly rely on synthetic data, either unintentionally (Schuhmann et al., 2022) or intentionally (Huang et al., 2022). This reliance creates a recursive cycle where successive generations are trained on mixed datasets of real and synthetic data, a process known as an STL, as shown in Figure 1.

More concretely, we explore a stochastic process that evolves through sequential generations. In an STL, we start with an initial dataset $S_{0}$, consisting of real data points $\bm{z}\in\mathcal{Z}$, sampled from the original real distribution $\mathcal{D}_{0}$. The initial generative model $\mathcal{G}_{0}$ is trained on this real dataset $S_{0}$, producing the first generation synthetic dataset $S_{1}$, whose distribution is denoted as $\mathcal{D}_{1}$. Next, the real dataset $S_{0}$ is combined with the synthetic dataset $S_{1}$ in a certain proportion to form a new mixed dataset $\widetilde{S}_{1}$, with distribution $\widetilde{\mathcal{D}}_{1}$. The next generation generative model $\mathcal{G}_{1}$ is then trained on this mixed dataset $\widetilde{S}_{1}$. Moving forward, for each subsequent generation $1\leq j\leq i$, the mixed dataset $\widetilde{S}_{j}$ is composed of real data and synthetic data from previous generations. The generative model $\mathcal{G}_{j}$ is trained on $\widetilde{S}_{j}$, producing the synthetic dataset $S_{j+1}$. This STL proceeds iteratively until the maximum generation, denoted as $i$, is reached.

3.2 Algorithmic Stability

Algorithmic stability measures the impact of modifying or removing a small number of examples from the training set on the resulting model, a key concept in statistical learning theory (Bousquet & Elisseeff, 2002). Its primary advantage lies in providing generalization bounds independent of model capacity. Among various stability notions (Shalev-Shwartz et al., 2010), we focus on uniform stability, the most widely studied form. Let $S$ and $S^{\prime}$ be two datasets differing by one point. Then, we formally define uniform stability as follows:

Definition 1.

(Uniform Stability (Bousquet & Elisseeff, 2002)). Algorithm $\mathcal{A}$ is uniformly $\beta_{n}$-stable with respect to the loss function $\ell$ if the following holds

$$\forall S,\ S^{\prime}\in\mathcal{Z}^{n},\ \forall\bm{z}\in\mathcal{Z},\ \sup_{\bm{z}}\left|\ell(\mathcal{A}(S),\bm{z})-\ell\left(\mathcal{A}\left(S^{\prime}\right),\bm{z}\right)\right|\leq\beta_{n}.$$

Traditional notions of stability have predominantly been studied in the context of learning algorithms, such as SGD (Lei & Ying, 2020). More recently, there has been significant progress in extending the concept of stability to generative models (Farnia & Ozdaglar, 2021; Zheng et al., 2023; Li et al., 2023). Building on these advancements, we propose recursive stability to specifically address generative models within STLs. This new stability measure is designed to quantify the differences in a generative model’s outputs after multiple generations of recursive training when small perturbations are applied to the initial real dataset. The formal definition of recursive stability is presented below.

Definition 2.

(Recursive Stability) Let $S_{0}$ represent the original real dataset, and $S^{\prime}_{0}$ denote a dataset differing from $S_{0}$ by a single example. A generative model $\mathcal{G}$ is said to be recursively $\gamma_{n}^{i,\alpha}$-stable with respect to the distance measure $d$ after the $i$-th generation of STLs, where the ratio of real to synthetic data is set to $\alpha$, if the following condition holds:

$$\forall S_{0},S^{\prime}_{0}\in\mathbb{Z}^{n},\quad d\left(\mathcal{G}^{(i)}(S_{0}),\mathcal{G}^{(i)}(S^{\prime}_{0})\right)\leq\gamma_{n}^{i,\alpha}.$$

where $\mathcal{G}^{(i)}$ denotes the output of the generative model at the $i$-th generation in the STLs. The distance measure $d$ quantifies the deviation between the outputs generated from inputs $S_{0}$ and $S_{0}^{\prime}$ across STLs. Specifically, $d$ can be defined using Total Variation (TV) distance, Kullback-Leibler (KL) divergence, or various norms (e.g., $\ell_{2}$ norm), allowing flexibility in assessing the differences in generated outputs.

4 General Theoretical Results

In this section, we present a general framework for analyzing generalization error. Moving beyond traditional analyses of parameter changes (Bertrand et al., 2024) and distributional discrepancies (Fu et al., 2024b), we focus on evaluating the utility of synthetic data after recursive training (Hittmeir et al., 2019; Xu et al., 2023). Specifically, we examine the behavior of a uniformly stable learning algorithm $\mathcal{A}$ trained on the mixed dataset $\widetilde{S}_{i}$ in the $i$-th generation. Our goal is to study the generalization error of the hypothesis $\mathcal{A}(\widetilde{S}_{i})$. Formally, we aim to bound $|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|$, where $R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))=\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}[\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z})]$ represents the population risk of $\mathcal{A}(\widetilde{S}_{i})$ under the real distribution $\mathcal{D}_{0}$, and $\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))=\frac{1}{n}\sum_{\bm{z}_{i}\in\widetilde{S}_{i}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})$ denotes the empirical risk on the mixed dataset. To derive this bound, we first decompose the generalization error as follows.

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|\leq\underbrace{\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|}_{\text{Cumulative distribution shift across generations}}+\underbrace{\left|R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|}_{\text{Generalization error on mixed distributions}}.$$

The first term captures the accumulation of error and distribution divergence over multiple generations within the STLs. This heavily depends on the capacity of the generative model to preserve distributional fidelity across generations, requiring recursive techniques to manage error propagation. The second term reflects the generalization performance of the learning algorithm on the non-i.i.d. mixed dataset, where synthetic data points are influenced by the initial real dataset. Drawing on Zheng et al. (2023), we observe that while $S_{0}$ satisfies the i.i.d. assumption, the synthetic datasets $S_{i}$ follow a conditional i.i.d. assumption given $S_{0}$. Leveraging this, along with moment bounds and concentration inequalities, we address the challenge of bounding the second term and managing dependencies within the STLs. We now present the following result.

Theorem 1 (General Generalization Bound).

Assume that $\mathcal{A}$ is a $\beta_{n}$-uniformly stable learning algorithm and the loss function $\ell$ is bounded by $M$. Let $n$ represent the sample size of the mixed dataset $\widetilde{S}_{j}$, defined as $\widetilde{S}_{j}=\alpha S_{0}+(1-\alpha)S_{j}$ for $1\leq j\leq i$, where $0<\alpha\leq 1$ denotes the proportion of real data. Assume further that the generative model $\mathcal{G}$ is recursively $\gamma_{n}^{i}$-stable, and the TV distance for each generation $TV(\widetilde{\mathcal{D}}_{j},\mathcal{D}_{j+1})$ is of the same order, denoted by $d_{\mathrm{TV}}(n)$. Then, for any $\delta\in(0,1)$, with probability at least $1-\delta$, the following holds:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|\lesssim\gamma_{n}^{i}\alpha M\log(n\alpha)\log(1/\delta)+n^{-1/2}M\sqrt{\log 1/\delta}$$

$$\quad+\beta_{n}\left(\log n\log(1/\delta)+\alpha\sqrt{(1-\alpha)n\log(1/\delta)}\right)+d_{\mathrm{TV}}(n)M\left(1-(1-\alpha)^{i}\right)\alpha^{-1},$$ (1)

where $\gamma_{n}^{i}=\sup_{j}TV(\mathcal{D}_{i}^{n(1-\alpha)}(S_{0}^{\prime}),\mathcal{D}_{i}^{n(1-\alpha)}(S_{0}))$, with $S_{0}$ and $S_{0}^{\prime}$ representing two real datasets of size $n$, differing by only a single data point.

Remark 2.

Recursive Stability in STLs. In Theorem 1, the recursive stability parameter is quantified using the TV distance to measure the divergence between the distributions of the $n(1-\alpha)$ synthetic data points generated by the model $\mathcal{G}_{i}$ at the $i$-th generation. Notably, the concept of recursive stability, introduced in Definition 2, is adaptable to various metrics, making it applicable across different types of generative models. In Theorem 2, the recursive stability parameter for transformers is instead defined using the $\ell_{2}$ norm between tokens, allowing this concept to be generalized to a broader range of model architectures.

Moreover, Theorem 1 demonstrates that generative models with higher recursive stability exhibit better performance after undergoing the STL. Specifically, the results indicate that the convergence rate of recursive stability parameter is at least faster than $\mathcal{O}(1/\log n)$, which is a relatively mild condition. Furthermore, Theorem 2 shows that, under mild assumptions, the recursive stability parameter for transformers in in-context learning settings achieves a convergence rate of $\gamma_{n}^{i}=\mathcal{O}(1/n)$ when measured by the $\ell_{2}$ norm between tokens.

Remark 3.

Effect of Real Data Proportion on Error Control. Previous experimental results (Shumailov et al., 2024; Alemohammad et al., 2024a) have demonstrated that incorporating real data can mitigate model collapse and help control errors. This remark focuses on exploring the effect of the real data proportion $\alpha$ on the generalization error within the STLs. As shown in Theorem 1, the real data proportion $\alpha$ plays a significant role in the cumulative distribution shift across generations, specifically in the term $2M\left(1-(1-\alpha)^{i}\right)\alpha^{-1}d_{\mathrm{TV}}(n)$.

When $\alpha\to 0$, we observe that $\frac{(1-(1-\alpha)^{i})}{\alpha}\to i$, leading to a linear accumulation of errors due to the Distribution Shift, making it increasingly challenging to control the overall error. This observation aligns with the theoretical results reported in Shumailov et al. (2024); Dohmatob et al. (2024a); Fu et al. (2024b). However, it is important to note that the conditions on $\alpha$ for controlling this term are not strict. In fact, as long as $\alpha$ remains at a non-negligible constant level, the expression $\left(1-(1-\alpha)^{i}\right)\alpha^{-1}$ remains bounded, effectively controlling the error. This aligns with theoretical intuition: when $\alpha$ is too small, the mixed dataset contains insufficient real data, resulting in a more severe distribution shift.

Moreover, the proportion of real data $\alpha$ also impacts the generalization error on mixed distributions, primarily through its effect on the recursive stability parameter $\gamma_{n}^{i}$. As $\alpha$ increases, the generative model becomes more recursively stable. We will further explore the influence of $\alpha$ on the recursive stability parameter $\gamma_{n}^{i}$ for specific generative models, such as transformers, in Theorem 3, particularly in Remark 8.

Remark 4.

Convergence Rate of Uniform Stability Parameter. With respect to the uniform stability parameter $\beta_{n}$, we observe from the third term on the right-hand side of inequality 1 that the convergence rate of $\beta_{n}$ must be at least $\mathcal{O}(1/\sqrt{n})$ to adequately control the error. This is a relatively mild requirement.

For example, in the case of widely used algorithms such as SGD, it has been shown that the uniform stability parameter $\beta_{n}$ converges at a rate of $\mathcal{O}(\log(n)/n)$ under the assumptions of Lipschitz continuity and smoothness of the loss function (Zhang et al., 2022). Additionally, for regularization-based algorithms, such as kernel regularization schemes and the Minimum Relative Entropy (MRE) algorithm, it has been demonstrated that $\beta_{n}$ can achieve a convergence rate of $\mathcal{O}(1/n)$ under certain conditions (Bousquet & Elisseeff, 2002).

Remark 5.

Convergence of the Distribution Shift Term $d_{\mathrm{TV}}(n)$. Regarding the convergence of the term $2M\left(1-(1-\alpha)^{i}\right)\alpha^{-1}d_{\mathrm{TV}}(n)$, as discussed in Remark 3, when $\alpha$ remains a non-negligible constant, attention turns to the distribution shift term $d_{\mathrm{TV}}(n)$. This term critically depends on the generative model’s capacity and quantifies the divergence between the learned distribution and the input distribution in each generation.

Theoretical studies have provided various convergence rates for $d_{\mathrm{TV}}(n)$ across different generative models. For instance, in diffusion models, $d_{\mathrm{TV}}(n)$ has been shown to converge at a rate of $\mathcal{O}\left(1/n^{1/4}\right)$ (Fu et al., 2024b). Similarly, for GANs, the convergence rate is also $\mathcal{O}\left(1/n^{1/4}\right)$ (Liang, 2021). More generally, by applying Pinsker’s inequality to relate KL divergence and TV distance, the convergence rates for other models, such as Bias potential models and Normalizing flows, have been explored in previous works (Yang, 2022). Additionally, we will further examine the behavior of transformer models in Theorem 3, demonstrating the flexibility of our theoretical framework across a wide range of generative models.

Remark 6.

Comparision with Previous Works. In the realm of theoretical research on the STL, where models are recursively trained on the synthetic data they generate, the foundational work was introduced by Shumailov et al. (2024) and Alemohammad et al. (2024a). They provided the initial theoretical definitions and analyzed the behavior of a simplistic multivariate Gaussian toy model in such loops. However, their analyses were limited to basic theoretical insights and lacked in-depth exploration of more complex generative models.

Recent advancements in this field have primarily come from Bertrand et al. (2024) and Fu et al. (2024b). Bertrand et al. (2024) established an upper bound on the deviation of likelihood-based model output parameters from the optimal ones, denoted as $\left\|\theta_{i}-\theta^{*}\right\|$. This was achieved by making direct assumptions on the upper bounds of both statistical and optimization errors in generative models, as outlined in their Assumption 3. In contrast, Fu et al. (2024b) derived bounds on the TV distance, addressing the distribution divergence between the synthetic data distributions produced by future models and the original real data distribution, with a specific focus on diffusion models. Our work makes significant theoretical advancements over both Bertrand et al. (2024) and Fu et al. (2024b) in several key aspects:

1. Innovative Concept of Recursive Stability. A central technical contribution of our work is the extension of the traditional notion of algorithmic stability. We define recursive stability, a crucial factor for controlling error propagation across generations. This novel concept tackles the theoretical challenges posed by non-i.i.d. data and recursive structures within STLs, while also incorporating the influence of model architectures into the generalization error. Moreover, recursive stability serves as a new measure for assessing the stability of generative models within STLs. In Theorem 2, we further establish an upper bound on the recursive stability parameter for transformers under mild conditions, underscoring the broad applicability and robustness of our framework.

2. Establishing the First Generalization Error Bound for STLs. While Bertrand et al. (2024) primarily focused on parameter deviations in generative models and Fu et al. (2024b) concentrated on distribution divergence, our work emphasizes the utility of the generated data produced by STLs. Specifically, by utilizing recursive stability, we present the first generalization error bound that quantifies the gap between the population risk on the initial real data distribution $\mathcal{D}_{0}$ and the empirical risk of the hypothesis $\mathcal{A}(\widetilde{S}_{i})$, generated by applying learning algorithms to the synthetic data produced after multiple generations of STLs. This introduces a new layer of complexity compared to prior work, as it necessitates handling not only the distribution shifts within STLs but also the challenges arising from the non-i.i.d. nature of the mixed datasets, where each generation’s data is influenced by all preceding generations.

3. A More General Framework Accounting for Model Structure. Our proposed theoretical framework is more comprehensive than previous studies. Bertrand et al. (2024) restricted their analysis to simplified likelihood-based generative models, while Fu et al. (2024b) focused specifically on diffusion models. Importantly, neither of their theoretical results accounted for the impact of different model architectures. In contrast, as discussed in Remark 5, our framework explicitly incorporates the effects of varying model structures, thereby extending its applicability to a broader range of generative models. Notably, we are the first to extend the theory of SLTs to transformer models, further broadening the scope of our approach across diverse generative model architectures.

4. Comprehensive Collapse Prevention Through Recursive Stability. In addition to the existing theoretical work, which primarily analyzes conditions to avoid model collapse based on the proportion of real data (e.g., Bertrand et al. (2024); Fu et al. (2024b)), our work extends these analyses by considering the impact of model architecture. Specifically, Theorem 1 demonstrates that under a recursive stability condition and a non-negligible constant level of real data, model collapse can be avoided across a variety of model architectures. This analysis offers broader conditions for preventing collapse by incorporating recursive stability, deepening the understanding of how model architecture affects training robustness.

Remark 7.

Proof Sketch of Theorem 1. We first decompose the generalization error of STLs into two distinct terms: (1) the cumulative distribution shift across generations, and (2) the generalization error on the mixed dataset.

Cumulative Distribution Shift: This term measures the shift between the real dataset $\mathcal{D}_{0}$ and the mixed distribution $\mathcal{D}_{i}$ after the $i$-th generation. Using the TV distance to quantify the shift introduced by the generative model, we bound the difference as:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|\leq(1-\alpha)\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i-1}}(\mathcal{A}(\widetilde{S}_{i}))\right|+2(1-\alpha)MTV(\widetilde{\mathcal{D}}_{i-1},\mathcal{D}_{i}).$$

By leveraging the recursive structure of the generative process, this cumulative distribution shift can be bounded across all generations as:

$$|R_{\mathcal{D}_{0}}(A(S_{i}))-R_{\mathcal{D}_{i}}(A(S_{i}))|\leq 2M\left(1-(1-\alpha)^{i}\right)\alpha^{-1}d_{\mathrm{TV}}(n).$$

Generalization Error on the Mixed Dataset: The second term quantifies the generalization error when training on the mixed dataset $\widetilde{S}_{i}$, which consists of both real and synthetic data. Our goal is to establish a moment bound on the generalization error, which can be decomposed as follows:

$$\|\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\|_{p}+\|(1-\alpha)R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{i,1-\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\|_{p}.$$

In this context, $S_{0,\alpha}$ represents a proportion $\alpha$ of the $n$ data points in $S_{0}$, leading to a total of $n\times\alpha$ data points. Similarly, $S_{i,1-\alpha}$ denotes a subset of the synthetic dataset $S_{i}$, where $S_{i,1-\alpha}\subseteq S_{i}$ and its size is $(1-\alpha)\times|S_{i}|$. For each term, we leverage the uniform stability $\beta_{n}$ of the learning algorithm $\mathcal{A}$ and the recursive stability $\gamma_{n}^{i}$ of the generative model to address the non-i.i.d. nature of the mixed dataset. The mixed dataset exhibits conditional independence (Zheng et al., 2023), with synthetic data conditioned on the initial real dataset $S_{0}$, allowing the application of recursive techniques to derive the moment bound. Subsequently, Lemma 8 and Lemma 9 are utilized to derive the high-probability bound for the final result.

5 Theoretical Analysis of Transformers in In-Context Learning

In this section, we first present the transformer in in-context learning (ICL) and its settings within SLTs in Section 5.1. In Section 5.2, we prove that it satisfies recursive stability, followed by the derivation of the generalization error bound for transformers in ICL in Section 5.3. Finally, in Section 5.4, we explore the scenario of synthetic data augmentation and investigate the associated trade-offs.

5.1 Settings of Transformer in In-context Learning

In-Context Learning Setting. ICL involves a transformer model processing a sequence of input-output examples to perform inference without parameter updates. Unlike traditional supervised learning, where a model is trained on a fixed dataset and then makes predictions, ICL allows the model to adapt on-the-fly to new queries based on the provided examples. We denote a prompt, containing $n$ in-context examples followed by the ($n+1$)-th query input, as $S_{0}=\left(\bm{z}_{1},\bm{z}_{2},\ldots,\bm{z}_{n},\bm{x}_{n+1}\right),$ where $\left(\bm{z}_{i}\right)_{i=1}^{n}=\left(\bm{x}_{i},\bm{y}_{i}\right)_{i=1}^{n}\in\mathcal{Z}=\mathcal{X}\times\mathcal{Y}$ represents i.i.d. in-context samples, and $\bm{x}_{n+1}\in\mathcal{X}$ is the query input whose label we want to predict. The transformer model, denoted as $\mathrm{TF}(\cdot)$, takes the prompt $S_{0}$ as input and outputs the predicted label $\hat{\bm{y}}_{n+1}$ for the query $\bm{x}_{n+1}$: $\hat{\bm{y}}_{n+1}=\mathrm{TF}(S_{0})$.

Recursive Data Generation in STLs with ICL. We extend the traditional ICL setting to an STL, where the transformer recursively generates new data using its own ICL predictions. Starting with an initial real dataset $S_{0}$, this serves as the initial real in-context examples for the transformer. The process begins by sampling the first generation queries $\left\{\bm{x}_{1,j}\right\}_{j=1}^{n}$ i.i.d. from the input distribution $\mathcal{X}$. Each query $\bm{x}_{1,j}$ is incorporated into the in-context examples from $S_{0}$ as a new query $\bm{x}_{0,n+1}$, and the transformer predicts the corresponding label $\hat{\bm{y}}_{1,j}$. This produces a synthetic dataset $S_{1}$, consisting of inputs $\left\{\bm{x}_{1,j}\right\}_{j=1}^{n}$ and their predicted labels $\left\{\hat{\bm{y}}_{1,j}\right\}_{j=1}^{n}$. A mixed dataset $\widetilde{S}_{j}$ is then formed and used as the in-context examples for the next generation. This process continues, with each generation producing a new synthetic dataset $S_{j+1}$ based on the updated mixed dataset $\widetilde{S}_{j}$.

5.2 Recursive Stability of In-Context Learning with Transformers

In this section, we demonstrate that transformers exhibit recursive stability within the ICL framework. Following the ICL setting from Li et al. (2023), we show that the model effectively controls error propagation from perturbations in the initial real dataset, ensuring stability across the STLs.

Theorem 2.

Let $S_{0},S_{0}^{\prime}$ be two initial real datasets that only differ at the inputs $\bm{z}_{j}=\left(\bm{x}_{j},\bm{y}_{j}\right)$ and $\bm{z}_{j}^{\prime}=$ $\left(\bm{x}_{j}^{\prime},\bm{y}_{j}^{\prime}\right)$ where $1\leq j\leq n$. Assume the inputs and labels lie within the unit Euclidean ball in $\mathbb{R}^{d}$. Represent the prompts $S_{0}$ and $S_{0}^{\prime}$ as matrices $\bm{Z}_{0},\bm{Z}_{0}^{\prime}\in\mathbb{R}^{(2n+1)\times d}$. Let $\mathrm{TF}(\cdot)$ be an $L$-layer transformer. Given $\bm{Z}_{0}$ as the initial input, the $k$-th layer applies MLPs and self-attention, producing the output:

$$\left.\bm{Z}_{k}=\operatorname{Parallel\_\operatorname{MLPs}(ATTN}\left(\bm{Z}_{k-1}\right)\right)\text{ where }\operatorname{ATTN}(\bm{Z}):=\operatorname{softmax}\left(\bm{Z}\bm{W}\bm{Z}^{\top}\right)\bm{Z}\bm{V}.$$

Assume $\mathrm{TF}$ is normalized as $\|\bm{V}\|\leq 1,\|\bm{W}\|\leq B_{W}$ and $\operatorname{MLPs}$ obey $\operatorname{MLP}(\bm{z})=\operatorname{ReLU}(\bm{M}\bm{z})$ with $\|\bm{M}\|\leq 1$. Let $\mathrm{TF}$ output the last token of the final layer $\bm{Z}_{L}$ that corresponds to the query $\bm{x}_{j,n+1}$. Let $n$ represent the sample size of the mixed dataset $\widetilde{S}_{j}$, where $\widetilde{S}_{j}=\alpha S_{0}+(1-\alpha)S_{j}$ for $1\leq j\leq i$. Then, we obtain:

$$\left\|\operatorname{TF}(\widetilde{S}_{i})-\operatorname{TF}(\widetilde{S}_{i}^{\prime})\right\|_{\ell_{2}}\lesssim(1-\alpha)^{i}\frac{\widetilde{B}_{W}^{(i+1)L}}{2n+1},$$

where $\widetilde{B}_{W}=\left(1+2B_{W}\right)e^{2B_{W}}$ and $\widetilde{S}_{i}^{\prime}$ denotes the mixed dataset at the $i$-th generation in the STL when the initial real dataset is $S_{0}^{\prime}$. Additionally, if the measure $d$ for the recursive stability parameter in Definition 2 is taken as the $\ell_{2}$ norm, then the recursive stability $\gamma_{n}^{i}\lesssim(1-\alpha)^{i}\frac{\widetilde{B}_{W}^{(i+1)L}}{2n+1}$.

Remark 8.

Controlling Exponential Growth with Real Data Proportion. In this remark, we further investigate the influence of the proportion of real data $\alpha$ on the recursive stability of transformers. As outlined in Theorem 2, the upper bound of the recursive stability parameter includes a term that grows exponentially with the number of generations $i$ in the STL, specifically $\widetilde{B}_{W}^{(i+1)L}$. However, we show that even a constant proportion of real data, $\alpha$, is sufficient to control this growth.

Specifically, setting $\alpha=\Omega(1-\widetilde{B}_{W}^{-((i+1)L)/i})$, we establish that the recursive stability parameter in Theorem 2 satisfies $\gamma_{n}^{i}\lesssim\frac{1}{2n+1}$. Additionally, as the number of generations $i$ in the STL approaches infinity, the proportion $\alpha$ asymptotically converges to $1-\widetilde{B}_{W}^{-L}$. Notably, the depth $L$ is typically small in practical settings. For example, studies on LLM performance in STLs, such as Briesch et al. (2023), often employ models with $L=6$. Furthermore, techniques like layer normalization effectively constrain the norm of weights $B_{W}$, ensuring numerical stability during training. Thus, with a constant real data proportion $\alpha$ independent of the STL generation number $i$, the exponential growth term $\widetilde{B}_{W}^{(i+1)L}$ can be effectively controlled, ensuring that $\gamma_{n}^{i}=\mathcal{O}(1/n)$.

5.3 Generalization Bound for Transformers in In-Context Learning

In this section, we investigate the behavior of transformers under the ICL framework in STLs. We select SGD as the learning algorithm $\mathcal{A}$ and consider a binary task with $\mathcal{Y}=\{0,1\}$. Applying our general theoretical framework from Theorem 1, we derive the generalization error bound by addressing the terms $\beta_{n}$ and $d_{\mathrm{TV}}(n)$ using recent results on SGD (Zhang et al., 2022) and ICL (Zhang et al., 2023). The recursive stability parameter $\gamma_{n}^{i}$ is obtained from Theorem 2. We assume that the loss function $\ell(\cdot;z)$ is $\kappa$-smooth and $\rho$-Lipschitz, which are standard assumptions in related works (Hardt et al., 2016; Lei & Ying, 2020), with formal definitions provided in Appendix A.1. Examples include logistic and Huber losses. We now present the generalization error bound:

Theorem 3.

Consider an $L$-layer transformer under the setting described in Theorem 2. Let $n$ represent the sample size of the mixed dataset $\widetilde{S}_{j}$, where $\widetilde{S}_{j}=\alpha S_{0}+(1-\alpha)S_{j}$ for $1\leq j\leq i$. Suppose that the loss function $\ell(\cdot;\bm{z})$ is $\kappa$-smooth, $\rho$-Lipschitz and bounded by $M>0$ for every $\bm{z}$. Let $\mathcal{A}(\widetilde{S}_{i})$ denote the output after running SGD for $T\gtrsim n$ iterations with a step size $\eta_{t}=\mathcal{O}(\frac{1}{\kappa t})$ on the mixed dataset $\widetilde{S}_{i}$. Then, for any $\delta\in(0,1)$, with probability at least $1-\delta$, the following holds:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|\lesssim n^{-1/2}\log(n)M\rho^{2}\alpha\sqrt{1-\alpha}\log\frac{1}{\delta}$$

$$\quad+n^{-1}\log^{2}(n)\rho^{2}((1-\alpha)\widetilde{B}_{W}^{L})^{i}\alpha\log(\frac{1}{\delta})+n^{-1/4}\alpha^{-1}M\left(1-(1-\alpha)^{i}\right)\log(\frac{1}{\delta}).$$ (2)

Remark 9.

In this remark, we provide a detailed explanation of the theoretical results of Theorem 3. As discussed earlier in Remark 8, $\alpha$ is set to $1-\widetilde{B}_{W}^{-L}$. To enhance clarity and focus on the primary results, we omit constant terms and the $\log(1/\delta)$ factor. Consequently, the bound in Theorem 3 can be expressed as follows:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|\lesssim n^{-1/2}\log(n)+n^{-1}\log^{2}(n)+n^{-1/4}.$$

In this bound, the terms $n^{-1/2}\log(n)+n^{-1}\log^{2}(n)$ correspond to the generalization error on the mixed dataset, while the term $n^{-1/4}$ represents the cumulative distribution shift across generations, which is primarily governed by the learnability of the generative model.

It is evident from this result that the generative model’s capacity plays a crucial role in the performance within the STLs. The ability of the generative model to maintain distributional fidelity over multiple generations directly impacts the generalization error and determines how well the model can control the propagation of errors across generations.

5.4 Synthetic Data Augmentation

The previous theorem addresses the scenario where the training dataset is unintentionally contaminated by synthetic data, leading to STLs. In contrast, many researchers intentionally incorporate synthetic data to augment the real dataset, also creating STLs. Next, we explore this synthetic data augmentation scenario, where each generation’s synthetic data is added to the mixed dataset, i.e., $\widetilde{S}_{i}=\sum_{j=0}^{i}S_{j}$.

Theorem 4.

Consider an $L$-layer transformer under the setting described in Theorem 2. Let $n$ and $\lambda n$ represent the sample size of the real dataset $S_{0}$ and the synthetic dataset $S_{j}$, respectively, where $1\leq j\leq i$. The mixed dataset $\widetilde{S}_{i}$ is denoted as $\sum_{j=0}^{i}S_{j}$. Suppose that the loss function $\ell(\cdot;\bm{z})$ is $\kappa$-smooth, $\rho$-Lipschitz and bounded by $M>0$ for every $\bm{z}$. Let $\mathcal{A}(\widetilde{S}_{i})$ denote the output after running SGD for $T\gtrsim n$ iterations with a step size $\eta_{t}=\mathcal{O}(\frac{1}{\kappa t})$ on the mixed dataset $S_{i}$. Then, for any $\delta\in(0,1)$, with probability at least $1-\delta$, the following holds:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|\lesssim n^{-\frac{1}{4}}\log((1+i\lambda)n)M\log\frac{1}{\delta}$$

$$\quad+n^{-1}\frac{\rho^{2}}{(1+i\lambda)^{2}}\log((1+i\lambda)n)i!\widetilde{B}_{W}^{(i+1)L}\log\frac{1}{\delta}+n^{-\frac{1}{2}}\frac{Mi}{1+i\lambda}\sqrt{\log\frac{1}{\delta}}.$$

Remark 10.

Analyzing the Trade-off in Synthetic Data Augmentation for STLs. In this remark, we examine the trade-off between generalization and distribution shifts from increased synthetic data, providing insights into optimal synthetic data size. At each generation, $\lambda n$ synthetic data points are added to the mixed dataset. We analyze how the coefficient $\lambda$, representing the scale of synthetic data augmentation, affects the generalization error in STLs. From the bound in Theorem 4, we observe that the Cumulative Distribution Shift Across Generations term is expressed as:

$$n^{-\frac{1}{4}}\log((1+i\lambda)n)M\log(1/\delta).$$

As the coefficient $\lambda$ increases, the cumulative distribution shift correspondingly grows, thereby amplifying the associated error. This behavior aligns with intuition, as an increase in $\lambda$ reduces the proportion of real data within the mixed dataset at each generation. Consequently, this reduction in real data leads to a greater divergence between the mixed distribution and the true underlying distribution, exacerbating the deviation and compounding the error across successive generations. In contrast, for the Generalization Error on Mixed Distributions term:

$$n^{-1}\frac{\rho^{2}}{(1+i\lambda)^{2}}\log((1+i\lambda)n)i!\widetilde{B}_{W}^{(i+1)L}\log\frac{1}{\delta}+n^{-\frac{1}{2}}\frac{Mi}{1+i\lambda}\sqrt{\log\frac{1}{\delta}}.$$

We observe that as $\lambda$ increases, the corresponding error decreases. This outcome is consistent with theoretical intuition, as augmenting the dataset with synthetic data effectively enlarges the mixed dataset. A larger dataset provides a more comprehensive representation of the mixed distribution, which in turn reduces the generalization error associated with this distribution. By incorporating more synthetic data, the mixed dataset better approximates the underlying mixed distribution, leading to improved generalization performance.

From the above discussion, we can conclude that the inclusion of synthetic data introduces a trade-off: on one hand, it increases the error from the cumulative distribution shift, while on the other, it reduces the generalization error on the mixed distribution. This trade-off has been explored theoretically in Fu et al. (2024b), though they primarily provided theoretical intuition. In contrast, our work explicitly decomposes the error into two terms, offering a deeper understanding of this trade-off and its implications for model performance in STLs. As for the optimal augmentation coefficient $\lambda^{*}$, it must satisfy the following condition:

$$\lambda^{*}=\inf_{\lambda} \Big{\{}n^{-\frac{1}{4}}\log((1+i\lambda)n)M\log(1/\delta)$$

$$\lesssim n^{-1}\frac{\rho^{2}}{(1+i\lambda)^{2}}\log((1+i\lambda)n)i!\widetilde{B}_{W}^{(i+1)L}\log\frac{1}{\delta}+n^{-\frac{1}{2}}\frac{Mi}{1+i\lambda}\sqrt{\log\frac{1}{\delta}}\Big{\}}.$$

Unfortunately, obtaining a closed-form solution for $\lambda^{*}$ from this equation proves to be analytically intractable. However, we can derive the relationship between $\lambda^{*}$, the size of the real dataset $n$ from the above equation. Specifically, by omitting irrelevant constants and the $\log(1/\delta)$ term, we obtain that $\lambda^{*}$ should satisfy the following expression:

$$\frac{i!\widetilde{B}_{W}^{(i+1)L}}{n^{3/4}(1+i\lambda^{*})^{2}}+\frac{i}{n^{1/4}(1+i\lambda^{*})\log((1+i\lambda^{*})n)}=\mathcal{O}(1).$$

We observe an important trend: the value of $\lambda^{*}$ increases as the size of the real dataset $n$ decreases. This aligns with theoretical intuition, as a smaller real dataset struggles to adequately represent the underlying distribution, leading to higher generalization error. Consequently, more synthetic data is required to control the generalization error of each generation on the mixed distribution. Conversely, when the real dataset is sufficiently large, the need for synthetic data augmentation diminishes.

6 Conclusion

As real-world data becomes increasingly scarce and existing datasets are progressively contaminated with synthetic content, STLs have emerged as a necessary strategy. STLs enable generative models to recursively train on a mix of real and synthetic data. However, empirical outcomes have varied significantly, revealing the need for a theoretical foundation to guide their successful application.

In this work, we introduced recursive stability as a key technical innovation and established the first generalization error bounds for STLs, which consider the impact of different model architectures. Our analysis demonstrated that preventing model collapse requires two critical conditions: maintaining a non-negligible proportion of real data and ensuring that models satisfy recursive stability. Furthermore, we were the first to extend this framework to transformers in in-context learning, showing that they also satisfy recursive stability and establish their generalization error bounds. Finally, we explored the trade-off introduced by synthetic data augmentation, balancing generalization improvement with potential distributional shifts. These contributions provide new insights into enhancing the stability and performance of generative models in STLs.

Appendix A Appendix

A.1 Auxiliary Definitions

Below, we present some essential definitions.

Definition 3.

(Lipschitz and Smoothness). Let constants $\kappa,\rho>0$. Consider the function $\ell:\mathcal{W}\times\mathcal{Z}\rightarrow\mathbb{R}$. We define the following properties:

• Lipschitz Continuity: The loss $\ell$ is said to be $\rho$-Lipschitz continuous if $\|\ell(\bm{w}_{1},\bm{z})-\ell(\bm{w}_{2},\bm{z})\|\leq\rho\|\bm{w}_{1}-\bm{w}_{2}\|$ for any $\bm{w}_{1},\bm{w}_{2},\bm{z}$.

• Smoothness: The loss $\ell$ is said to be $\kappa$-Smooth if $\|\nabla_{\bm{w}}\ell(\bm{w}_{1},\bm{z})-\nabla_{\bm{w}}\ell(\bm{w}_{2},\bm{z})\|\leq\kappa\|\bm{w}_{1}-\bm{w}_{2}\|$ for any $\bm{w}_{1},\bm{w}_{2},\bm{z}$.

A.2 Expansion to Gaussian Mixture Models

We adopt the setup from prior works Zheng et al. (2023) and consider a binary classification task where $Y=\{-1,1\}$. Given a vector $\mu\in\mathbb{R}^{d}$ with $\|\mu\|_{2}=1$ and noise variance $\sigma^{2}>0$, the data distribution is specified as follows: $y\sim\text{uniform}\{-1,1\}$ and $x\mid y\sim\mathcal{N}(y\mu,\sigma^{2}I_{d})$. We define the conditional generative model using parameters $\mu_{y}$ and $\sigma_{k}^{2}$, where $y\in\{-1,1\}$ and $k\in[d]$. For $n$ data points, let $n_{y}$ represent the number of samples in class $y$. The parameters of the Gaussian mixture model are then learned as:

$$\hat{\mu}_{y}=\frac{\sum_{y_{i}=y}x_{i}}{n_{y}},\quad\hat{\sigma}_{k}^{2}=\sum_{y}\frac{n_{y}}{n}\frac{\sum_{y_{i}=y}(x_{ik}-\hat{\mu}_{yk})^{2}}{n_{y}-1}.$$

Then we can generate new samples from the distribution: $y\sim\text{uniform}\{-1,1\}$ and $x\mid y\sim\mathcal{N}(\hat{\mu}_{y},\Sigma)$, where $\Sigma=\text{diag}(\sigma_{1}^{2},\dots,\sigma_{d}^{2})$. Additionally, the learning algorithm functions as a linear classifier, parameterized by $\theta\in\mathbb{R}^{d}$, with predictions given by: $\hat{y}=\text{sign}(\theta^{\top}\mathbf{x})$. The loss function is defined as:

$$\ell(\theta,(x,y))=\frac{1}{2\sigma^{2}}(x-y\theta)^{\top}(x-y\theta).$$

Thus, the output is $\hat{\theta}=\frac{1}{m}\sum_{i=1}^{m}y_{i}x_{i}.$

In this setting, we demonstrate recursive stability for the Gaussian mixture model as follows:

Theorem 5.

Let $S_{0},S_{0}^{\prime}$ denote two initial real datasets differing by a single example. Let $n$ represent the sample size of the mixed dataset $\tilde{S}_{j}$, where $\tilde{S}_{j}=\alpha S_{0}+(1-\alpha)S_{j}$ for $1\leq j\leq i$. Choose $m=\mathcal{O}(\sqrt{n})$. Consider the previously described sampling and learning steps, where real data samples are drawn from the Gaussian Mixture Model distribution $\mathcal{D}$, and the synthetic data for the $i$-th generation is generated from the learned Gaussian Mixture distribution of the $i$-th generation. Then with probability at least $1-\delta$, we have:

$$\gamma_{n}^{i}\lesssim n^{-1/2}\alpha^{-1}(1-(1-\alpha)^{i})\log(nd/\delta),$$ (3)

where the measure for the recursive stability parameter is taken as the KL divergence.

As $\alpha$ approaches 0, indicating that no real data is incorporated during each generation of training, we observe

$$\gamma_{n}^{i}\lesssim in^{-1/2}\log\frac{nd}{\delta},$$

which suggests a linear accumulation of errors. This finding aligns closely with the theoretical insights presented in Shumailov et al. (2024); Alemohammad et al. (2024a), where a Gaussian model trained without real data demonstrated a linear divergence in variance. Thus, this underscores the validity of our theoretical results, confirming that the derived bound is meaningful and not vacuous.

Moreover, by leveraging the generalization error bound established in Theorem 1, we derive the following:

Theorem 6.

Consider the Gaussian Mixture Model in the setting outlined above. Let $n$ represent the sample size of the mixed dataset $\tilde{S}_{j}$, where $\tilde{S}_{j}=\alpha S_{0}+(1-\alpha)S_{j}$ for $1\leq j\leq i$. Suppose the loss function is defined as $\ell(\theta,(\mathbf{x},y))=\frac{1}{2\sigma^{2}}(\mathbf{x}-y\theta)^{\top}(\mathbf{x}-y\theta)$. Let $\mathcal{A}(\tilde{S}_{i})$ denote the output of applying the linear classifier described above to the mixed dataset $\tilde{S}_{i}$. Then, for any $\delta\in(0,1)$, with probability at least $1-\delta$, the following holds:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|\lesssim n^{-1/2}(d+\log(n/\delta))\log n\log(1/\delta)$$

$$\quad+n^{-1/4}(1-(1-\alpha)^{i})\alpha^{-1}(d+\log(n/\delta))\sqrt{d\log(nd/\delta)}.$$ (4)

We observe that when $\alpha$ is set to a constant (e.g., $\alpha=0.1$), the generalization error can be effectively controlled, preventing model collapse. This result aligns with the experimental findings in Alemohammad et al. (2024a) for Gaussian models.

A.3 Additional Comparison with Related Work on Theorem 1

Dohmatob et al. (2024a) examined a linear regression setting, focusing solely on statistical approximation error without addressing the functional approximation error described in Shumailov et al. (2024). They did not consider incorporating real data to prevent collapse and demonstrated a linear dependency of degradation on the generation number in the case of fully synthetic data. Similarly, Alemohammad et al. (2024a) and Shumailov et al. (2024) provided theoretical insights using simple Gaussian models without incorporating real data, proving that the variance diverges linearly with the generation number. Seddik et al. (2024) explored a linear softmax classifier and, while also neglecting functional approximation error, demonstrated that adding real data can mitigate model collapse. Marchi et al. (2024) used asymptotic analysis to study parameter variance, assuming an infinite number of training generations and considering scenarios where the generative model is controlled via a “temperature” parameter. They proved that parameter variance is bounded under these conditions.

In contrast, our work addresses a much more complex and realistic scenario by introducing the novel concept of recursive stability and providing the first generalization analysis for STLs. Our analysis accounts for statistical approximation error, functional approximation error, and optimization error during the training of generative models. Unlike the settings explored in prior theoretical works, such as linear regression (Dohmatob et al., 2024a; Gerstgrasser et al., 2024), Gaussian models (Alemohammad et al., 2024a; Shumailov et al., 2024), or asymptotic assumptions (Marchi et al., 2024), our framework accommodates more complex generative model architectures, such as transformers. Specifically, we reveal how both model architecture and the ratio of real to synthetic data influence the success of STLs. For example, in Theorem 3, we demonstrate how our general generalization bound applies to transformer-based generative models, providing a theoretical framework that aligns with practical and more sophisticated use cases.

Additionally, while Marchi et al. (2024) assumed an infinite number of training generations for their asymptotic analysis, we consider finite generations, which is more practical since most experimental setups limit generations to fewer than 10 (as noted in Shumailov et al. (2024)). Moreover, our results confirm that when $\alpha=0$ (i.e., no real data is used), the last term in our bound, representing the Cumulative Distribution Shift ($d_{\text{TV}}(n)M(1-(1-\alpha)^{i})\alpha^{-1}$), grows linearly. This finding aligns with the theoretical results of Dohmatob et al. (2024a); Alemohammad et al. (2024a); Shumailov et al. (2024); Fu et al. (2024b). Furthermore, we show that introducing even a constant proportion of real data significantly mitigates model collapse, aligning with experimental findings by Alemohammad et al. (2024a) and Bertrand et al. (2024).

A.4 Additional Comparison with Related Work on Theorem 4

Gerstgrasser et al. (2024) also explored the use of accumulating data to prevent model collapse. They considered a simple linear regression setting without accounting for the dynamic process of training generative models, focusing solely on statistical approximation error. They demonstrated that under the assumption of fixed synthetic data quality matching the original real data, statistical approximation error can be controlled.

By contrast, our work addresses a much more complex and realistic scenario, incorporating the dynamic behavior of transformer-based generative models, learning algorithms, and both statistical and functional approximation errors. Additionally, we allow for dynamic regulation of synthetic data size via a $\lambda$ coefficient, enabling us to identify the optimal synthetic dataset size for avoiding model collapse in these more challenging settings.

A.5 Auxiliary Lemmas

In this section, we begin by introducing a set of auxiliary theorems that will be utilized in the subsequent proofs.

Lemma 7 (McDiarmid’s Inequality).

Consider independent random variables $Z_{1},\cdots,Z_{n}\in\mathcal{Z}$ and a mapping $\phi:\mathcal{Z}^{n}\rightarrow\mathbb{R}$. If, for all $i\in\{1,\cdots,n\}$, and for all $z_{1},\cdots,z_{n},z_{i}^{\prime}\in\mathcal{Z}$, the function $\phi$ satisfies

$$\left|\phi\left(z_{1},\cdots,z_{i-1},z_{i},z_{i+1},\cdots,z_{n}\right)-\phi\left(z_{1},\cdots,z_{i-1},z_{i}^{\prime},z_{i+1},\cdots,z_{n}\right)\right|\leq c,$$

then,

$$P\left(|\phi\left(Z_{1},\cdots,Z_{n}\right)-\mathbb{E}\phi\left(Z_{1},\ldots,Z_{n}\right)\geq t|\right)\leq 2\exp\left(\frac{-2t^{2}}{nc^{2}}\right).$$

Furthermore, for any $p\geq 2$,

$$\left\|\phi\left(Z_{1},\ldots,Z_{n}\right)-\mathbb{E}\left[\phi\left(Z_{1},\ldots,Z_{n}\right)\right]\right\|_{p}\leq 2\sqrt{np}c.$$

Lemma 8.

((Bousquet et al., 2020)). Let $\bm{z}=\left(Z_{1},\ldots,Z_{n}\right)$ be a vector of independent random variables each taking values in $\mathcal{Z}$, and let $g_{1},\ldots,g_{n}$ be some functions $g_{i}:\mathcal{Z}^{n}\rightarrow\mathbb{R}$ such that the following holds for any $i\in[n]$ :

• $\left|\mathbb{E}\left[g_{i}(\bm{z})\mid Z_{i}\right]\right|\leq M$,

• $\mathbb{E}\left[g_{i}(\bm{z})\mid\bm{z}^{\backslash i}\right]=0$,

• $g_{i}$ has a bounded difference $\beta$ with respect to all variables except the $i$-th variable, that is, for all $j\neq i,\bm{z}=\left(Z_{1},\ldots,Z_{n}\right)$ and $\bm{z}^{j}=\left(Z_{1},\ldots,Z_{j}^{\prime},\ldots,Z_{n}\right)\in\mathbb{R}^{n}$, we have $\left|g_{i}(\bm{z})-g_{i}\left(\bm{z}^{j}\right)\right|\leq\beta$.

Then, for any $p\geq 2$,

$$\left\|\sum_{i=1}^{n}g_{i}(\bm{z})\right\|_{p}\leq 12\sqrt{2}pn\beta\log n+4M\sqrt{pn}.$$

Lemma 9.

If $\|Y\|_{p}\leq\sqrt{p}a+pb$ for any $p\geq 1$, then for any $\delta\in(0,1)$, with probability at least $1-\delta$,

$$|Y|\leq e\left(a\sqrt{\log\left(\frac{e}{\delta}\right)}+b\log\left(\frac{e}{\delta}\right)\right).$$

In addition, we introduce the definition of the Total Variation (TV) distance as follows:

Definition 4 (Total Variation Distance).

Given two probability distributions $p$ and $q$ over a multidimensional space $\mathbb{R}^{d}$, the Total Variation Distance between $p$ and $q$ is:

$$TV(p,q)=\frac{1}{2}\int_{\mathbb{R}^{d}}|p(\bm{z})-q(\bm{z})|\,d\bm{z}.$$

A.6 Proof of Theorem 1

In this Section, we prove Theorem 1 by first decomposing the generalization error into two components: the Cumulative Distribution Shift Across Generations and the Generalization Error on Mixed Distributions. We then proceed to bound the Cumulative Distribution Shift Across Generations by leveraging the properties of the generative model and recursive techniques. For the Generalization Error on Mixed Distributions, we follow the framework of Zheng et al. (2023), leveraging the fact that within the mixed dataset $\widetilde{S}_{i}$, the set $S_{i}$ satisfies the conditional i.i.d. assumption when $S_{0}$ is fixed. Combined with moment bounds, this allows us to effectively bound the generalization error.

The main proof is as follows:

Proof of Theorem 1.

We begin by decomposing the generalization error as follows:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|\leq\underbrace{\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|}_{\text{Cumulative distribution shift across generations}}+\underbrace{\left|R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|}_{\text{Generalization error on mixed distributions}}.$$

Upper Bounding Cumulative Distribution Shift Term

For the term $\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|$, we first note that $\widetilde{\mathcal{D}}_{i}=\alpha\mathcal{D}_{0}+(1-\alpha)\mathcal{D}_{i}$. Therefore, we obtain:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|$$

$$=\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i})-(1-\alpha)R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|$$

$$=(1-\alpha)\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i})-R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|.$$ (5)

Furthermore, we can further decompose it as follows:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i})-R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|\leq\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i-1}}(\mathcal{A}(\widetilde{S}_{i}))\right|+\left|R_{\widetilde{\mathcal{D}}_{i-1}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|.$$ (6)

By substituting inequality 6 into inequality 5, we obtain:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|$$

$$\leq(1-\alpha)\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i-1}}(\mathcal{A}(\widetilde{S}_{i}))\right|+(1-\alpha)\left|R_{\widetilde{\mathcal{D}}_{i-1}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|.$$ (7)

Then, for the term $|R_{\widetilde{\mathcal{D}}_{i-1}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|$, we have:

$$\left|R_{\widetilde{\mathcal{D}}_{i-1}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right| =\Bigg{|}\int_{\bm{z}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z})\left(\mathbb{P}_{\widetilde{\mathcal{D}}_{i-1}}(\bm{z})-\mathbb{P}_{\mathcal{D}_{i}}(\bm{z})\right)d\bm{z}\Bigg{|}$$

$$\leq\int_{\bm{z}}\biggl{|}\ell(\mathcal{A}(\widetilde{S}),\bm{z})\left(\mathbb{P}_{\widetilde{\mathcal{D}}_{i-1}}(\bm{z})-\mathbb{P}_{\mathcal{D}_{i}}(\bm{z})\right)\biggr{|}d\bm{z}$$

$$\leq M\int_{\bm{z}}\Bigl{|}\mathbb{P}_{\widetilde{\mathcal{D}}_{i-1}}(\bm{z})-\mathbb{P}_{\mathcal{D}_{i}}(\bm{z})\Bigr{|}d\bm{z}$$

$$=2MTV\left(\widetilde{\mathcal{D}}_{i-1},\mathcal{D}_{i}\right).$$ (8)

Incorporating inequality 8 into inequality 7, we arrive at:

$$|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|$$

$$\leq(1-\alpha)|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i-1}}(\mathcal{A}(\widetilde{S}_{i}))|+2(1-\alpha)MTV\left(\widetilde{\mathcal{D}}_{i-1},\mathcal{D}_{i}\right).$$ (9)

Next, we apply recursive techniques to address the problem further. First, we obtain

$$|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i-1}}(\mathcal{A}(\widetilde{S}_{i}))|$$

$$\leq(1-\alpha)|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i-2}}(\mathcal{A}(\widetilde{S}_{i}))|+2(1-\alpha)MTV\left(\widetilde{\mathcal{D}}_{i-2},\mathcal{D}_{i-1}\right).$$ (10)

Plugging inequality 10 into inequality 9into the inequality, we obtain that:

$$|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|$$

$$\leq(1-\alpha)^{2}|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i-2}}(\mathcal{A}(\widetilde{S}_{i}))|+2(1-\alpha)^{2}MTV\left(\widetilde{\mathcal{D}}_{i-2},\mathcal{D}_{i-1}\right)+2(1-\alpha)MTV\left(\widetilde{\mathcal{D}}_{i-1},\mathcal{D}_{i}\right).$$

By recursion, we obtain:

$$|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|$$

$$\leq(1-\alpha)^{i-1}|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{1}}(\mathcal{A}(\widetilde{S}_{i}))|+2(1-\alpha)^{i-1}MTV\left(\widetilde{\mathcal{D}}_{1},\mathcal{D}_{2}\right)+...+2(1-\alpha)MTV\left(\widetilde{\mathcal{D}}_{i-1},\mathcal{D}_{i}\right)$$

$$\leq 2(1-\alpha)^{i}MTV\left(\mathcal{D}_{0},\mathcal{D}_{1}\right)+2(1-\alpha)^{i-1}MTV\left(\widetilde{\mathcal{D}}_{1},\mathcal{D}_{2}\right)+...+2(1-\alpha)MTV\left(\widetilde{\mathcal{D}}_{i-1},\mathcal{D}_{i}\right).$$

Let $n_{0}$ represent the sample size of the real dataset $S_{0}$, and let $n_{i}$ denote the sample size of the mixed dataset $\widetilde{S}_{i}$ in the $i$-th generation. Thus, $TV\left(\widetilde{\mathcal{D}}_{j},\mathcal{D}_{j+1}\right)$ can be written as a function of $n_{j}$. Assuming that the sample size for each generation’s dataset is identical, i.e., $n_{0}=n_{1}=\cdots=n_{i}=n$, and that the TV distance for each generation is of the same order, denoted by $d_{\mathrm{TV}}(n)$, we can derive the following result:

$$|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))| \leq 2Md_{\mathrm{TV}}(n)\left[(1-\alpha)^{i}+(1-\alpha)^{i-1}+...+(1-\alpha)\right]$$

$$=2M\left(1-(1-\alpha)^{i}\right)\alpha^{-1}d_{\mathrm{TV}}(n).$$ (11)

Then we obtain:

$$|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|\leq|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|+|R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|$$

$$\leq 2M\left(1-(1-\alpha)^{i}\right)\alpha^{-1}d_{\mathrm{TV}}(n)+|R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|.$$ (12)

Upper Bounding Generalization Error on Mixed Distributions Term

Next, we turn our attention to the term $|R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|$. Our primary objective is to establish a moment bound for this expression.

$$\left\|R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right\|_{p}$$

$$=\left\|\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))+(1-\alpha)R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{i,1-\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\right\|_{p}$$

$$\leq\underbrace{\left\|\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\right\|_{p}}_{\text{Term 1}}+\underbrace{\left\|(1-\alpha)R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{i,1-\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\right\|_{p}}_{\text{Term 2}}.$$ (13)

The newly sampled dataset, denoted as $S_{0,\alpha}$, is a subset of the original dataset $S_{0}$, where $S_{0,\alpha}\subseteq S_{0}$ and its size is $\alpha\times\left|S_{0}\right|$. Specifically, $S_{0,\alpha}$ contains a proportion $\alpha$ of the $n$ data points in $S_{0}$, resulting in a total of $n\times\alpha$ data points. Similarly, $S_{i,1-\alpha}$ is a subset of the synthetic dataset $S_{i}$, where $S_{i,1-\alpha}\subseteq S_{i}$, and its size is $(1-\alpha)\times\left|S_{i}\right|$. Specifically, $S_{i,1-\alpha}$ contains a proportion $1-\alpha$ of the $n$ data points in $S_{i}$, resulting in $n\times(1-\alpha)$ data points.

We observe that for any function $f(S)$, if there exists a bound $\|f\|_{p}\left(S_{j}\right)\leq C$ for some subset $S_{j}\subseteq S$, then we have the following:

$$\|f\|_{p}=\left(\mathbb{E}\mathbb{E}\left[|f|^{p}\mid S_{j}\right]\right)^{1/p}\leq\left(\mathbb{E}\left[C^{p}\right]\right)^{1/p}\leq C.$$

Fix $S_{0}$, then data in $S_{i}$ are independent. We use this property and Lemma 8 to bound the Term 2. We introduce functions $f_{j}(S_{i,1-\alpha})$ which play the same role as $g_{j}$’s in Lemma 8 as

$$f_{j}(S_{i,1-\alpha})=\mathbb{E}_{\bm{z}_{i,j}^{\prime}\sim\mathcal{D}_{i}}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{i}}\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z})-\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z}_{i,j})\right],$$

where $\bm{z}_{i,j}$ is the $j$-th data in $S_{i,1-\alpha}$, and $S_{i,1-\alpha}^{j}$ obtained by replacing $\bm{z}_{i,j}$ by $\bm{z}_{i,j}^{\prime}.$ Next, we prove that $f_{j}$ satisfies the three conditions outlined in Lemma 8. First, we demonstrate condition $|f_{j}|\leq M$.

$$|f_{j}| =\left|\mathbb{E}_{\bm{z}_{i,j}^{\prime}\sim\mathcal{D}_{i}}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{i}}\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z})-\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z}_{i,j})\right]\right|$$

$$\leq\mathbb{E}_{\bm{z}_{i,j}^{\prime}\sim\mathcal{D}_{i}}\mathbb{E}_{\bm{z}\sim\mathcal{D}_{i}}\left|\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z})-\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z}_{i,j})\right|.$$

$$\leq M$$

We then continue by proving conditions $\mathbb{E}[f_{j}|S_{i,1-\alpha}^{\setminus j}]=0$:

$$\mathbb{E}\left[f_{j}\mid S_{i,1-\alpha}^{\backslash j}\right]$$

$$=\mathbb{E}_{\bm{z}_{i,j}\sim\mathcal{D}_{i}}\left[\mathbb{E}_{\bm{z}_{i,j}^{\prime}\sim\mathcal{D}_{i}}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{i}}\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z})-\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z}_{i,j})\right]\mid S_{i,1-\alpha}^{\backslash j}\right]$$

$$=\mathbb{E}_{\bm{z}_{i,j}^{\prime}\sim\mathcal{D}_{i}}\left[\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{i}}\ell\left(\mathcal{A}\left(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}\right),\bm{z}\right)-\mathbb{E}_{\bm{z}_{i,j}\sim\mathcal{D}_{i}}\ell\left(\mathcal{A}\left(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}\right),\bm{z}_{i,j}\right)\right]\mid S_{i,1-\alpha}^{\backslash j}\right]$$

$$=\mathbb{E}_{\bm{z}_{i,j}^{\prime}\sim\mathcal{D}_{i}}\left[0\mid S_{i,1-\alpha}^{\backslash j}\right]=0.$$

Finally, we prove that $f_{j}$ has a bounded difference $2\beta_{n}$ with respect to all variables except the $j$-th variable. Let $t\neq j$, then we obtain:

$$|f_{j}\left(S_{i,1-\alpha}\right)- f_{j}\left(S_{i,1-\alpha}^{t}\right)|$$

$$= |\mathbb{E}_{\bm{z}_{i,j}^{\prime}\sim\mathcal{D}_{i}}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{i}}\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z})-\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z}_{i,j})\right]$$

$$-\mathbb{E}_{\bm{z}_{i,j}^{\prime}\sim\mathcal{D}_{i}}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{i}}\ell(\mathcal{A}(S_{0,\alpha}\cup(S_{i,1-\alpha}^{t})^{j}),\bm{z})-\ell(\mathcal{A}(S_{0,\alpha}\cup(S_{i,1-\alpha}^{t})^{j}),\bm{z}_{i,j})\right]\mid$$

$$\leq \left|\mathbb{E}_{\bm{z}_{i,j}^{\prime}\sim\mathcal{D}_{i}}\mathbb{E}_{\bm{z}\sim\mathcal{D}_{i}}\left[\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z})-\ell(\mathcal{A}(S_{0,\alpha}\cup(S_{i,1-\alpha}^{t})^{j}),\bm{z})\right]\right|$$

$$+\left|\mathbb{E}_{\bm{z}_{i,j}^{\prime}\sim\mathcal{D}_{i}}\left[\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z}_{i,j})-\ell(\mathcal{A}(S_{0,\alpha}\cup(S_{i,1-\alpha}^{t})^{j}),\bm{z}_{i,j})\right]\right|$$

$$\leq \beta_{n}+\beta_{n}=2\beta_{n}.$$

Therefore, for any fixed $S_{0}$, by Lemma 8, for any $p\geq 2$, we have

$$\left\|\sum_{j=1}^{n(1-\alpha)}f_{j}\left(S_{i,1-\alpha}\right)\right\|_{p}\lesssim pn(1-\alpha)\beta_{n}\log(n(1-\alpha))+M\sqrt{pn(1-\alpha)}.$$ (14)

We note that the difference between Term 2 and $\sum_{j=1}^{n(1-\alpha)}f_{j}$ is minimal. Consequently, for any fixed $S_{0}$, we can bound Term 2 using inequality 14 as follows.

$$\left\|(1-\alpha)R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{i,1-\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\right\|_{p}$$

$$=\left\|(1-\alpha)R_{\mathcal{D}_{i}}(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}))-\frac{1}{n}\sum_{j=1}^{n(1-\alpha)}\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}),\bm{z}_{i,j})\right\|_{p}\qquad\qquad\text{Fix}\ S_{0,\alpha}$$

$$=\left\|\frac{1}{n}\sum_{j=1}^{n(1-\alpha)}\left(\mathbb{E}_{\bm{z}\sim\mathcal{D}_{i}}\ell\left(\mathcal{A}\left(S_{0,\alpha}\cup S_{i,1-\alpha}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left(S_{0,\alpha}\cup S_{i,1-\alpha}\right),\bm{z}_{i,j}\right)\right)\right\|_{p}$$

$$\leq\frac{1}{n}\left\|\sum_{j=1}^{n(1-\alpha)}\left(\mathbb{E}_{\bm{z}_{i,j}^{\prime}\sim\mathcal{D}_{i}}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{i}}\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z})-\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z}_{i,j})\right]\right)\right\|_{p}+(1-\alpha)\left\|2\beta_{n}\right\|_{p}$$

$$=\frac{1}{n}\left\|\sum_{j=1}^{n(1-\alpha)}f_{j}\left(S_{i,1-\alpha}\right)\right\|_{p}+(1-\alpha)\left\|2\beta_{n}\right\|_{p}$$

$$\lesssim p(1-\alpha)\beta_{n}\log(n(1-\alpha))+M\sqrt{\frac{p(1-\alpha)}{n}}+2(1-\alpha)\beta_{n}$$

$$\lesssim p(1-\alpha)\beta_{n}\log(n(1-\alpha))+M\sqrt{\frac{p(1-\alpha)}{n}}.$$

Next, for Term 2, we derive the following result:

$$\left\|(1-\alpha)R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{i,1-\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\right\|_{p}\lesssim p(1-\alpha)\beta_{n}\log(n(1-\alpha))+M\sqrt{\frac{p(1-\alpha)}{n}}.$$ (15)

Now, we use a similar idea to bound Term 1 $\left\|\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\right\|_{p}$. We decompose it as the following.

$$\left\|\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\right\|_{p}$$

$$\leq\underbrace{\left\|(\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i}))-\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}}(\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i}))\right\|_{p}}_{\text{Term 3}}$$

$$+\underbrace{\left\|\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}}(\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i}))\right\|}_{\text{Term 4}}.$$ (16)

We proceed by bounding each term. Specifically, Term 3 can be bounded using McDiarmid’s inequality, as outlined in Lemma 7, and Term 4 can be bounded by applying Lemma 8.

To bound Term 3, we begin by fixing $S_{0,\alpha}$ and utilizing the conditional independence property of $S_{i}$ once again. In order to apply Lemma 8, we must show that $\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})$ exhibits a bounded difference with respect to $S_{i,1-\alpha}$ when $S_{0,\alpha}$ is fixed. This expression can be formulated as follows.

$$\left|\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}),\bm{z}_{i})\right.$$

$$\left.-\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}))+\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}),\bm{z}_{i})\right|$$

$$\leq\alpha\left|R_{\mathcal{D}_{0}}(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}))-R_{\mathcal{D}_{0}}(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}))\right|$$

$$+\frac{1}{n}\left|\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell(\mathcal{A}(S_{0,\alpha}\cup S_{i,1-\alpha}),\bm{z}_{i})-\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell\left(\mathcal{A}\left(S_{0,\alpha}\cup S_{i,1-\alpha}^{j}\right),\bm{z}_{i}\right)\right|$$

$$\leq\alpha\beta_{n}+\alpha\beta_{n}=2\alpha\beta_{n}.$$

Thus, by Mcdiarmid Inequality, we have

$$\left\|(\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i}))-\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}}(\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i}))\right\|_{p}$$

$$\leq 4\sqrt{n(1-\alpha)p}\alpha\beta_{n}\lesssim\sqrt{n(1-\alpha)p}\alpha\beta_{n}.$$ (17)

We now introduce a set of functions and apply Lemma 8 once more to bound Term 4. Specifically, we define $h_{j}(S)$, which serves a similar role to the $g_{i}$ ’s in Lemma 8, as follows:

$$h_{j}(S_{0,\alpha})$$

$$= \mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left(S_{0,\alpha}^{j}\right)}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left(S_{0,\alpha}^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left(S_{0,\alpha}^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)\right]$$ (18)

where $\bm{z}_{0,j}$ denote the $j$-th data point in $S_{0,\alpha}$, and $S_{0,\alpha}^{j}$ represent the dataset obtained by replacing $\bm{z}_{0,j}$ with $\bm{z}_{0,j}^{\prime}$. Additionally, it is important to note that $S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left(S_{0,\alpha}^{j}\right)$ indicates that $S_{i,1-\alpha}$ is the synthetic dataset generated after the self-consuming loop, following $i$-generations, and obtained by modifying a single data point from the initial real dataset $S_{0}$. This complex scenario can be addressed using the recursive stability we have defined for the self-consuming loop in Definition 2. Moreover, similar to the process above, we observe that $\left|h_{j}\right|\leq M$ and $\mathbb{E}\left[h_{j}\mid S_{0,\alpha}^{\backslash j}\right]=0$. More intricately, we will now prove that $h_{j}$ exhibits a bounded difference. This will be demonstrated as follows.

$$|h_{j}(S_{0,\alpha})-h_{j}\left(S_{0,\alpha}^{t}\right)|$$

$$= \mid\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left(S_{0,\alpha}^{j}\right)}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left(S_{0,\alpha}^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left(S_{0,\alpha}^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)\right]$$

$$- \mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left((S_{0,\alpha}^{t})^{j}\right)}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)\right]$$

$$\leq \mid\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left(S_{0,\alpha}^{j}\right)}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left(S_{0,\alpha}^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left(S_{0,\alpha}^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)\right]$$

$$- \mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left(S_{0,\alpha}^{j}\right)}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)\right]|$$ (19)

$$+ |\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left(S_{0,\alpha}^{j}\right)}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)\right]$$

$$- \mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left((S_{0,\alpha}^{t})^{j}\right)}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)\right]|.$$ (20)

We can bound equation 19 by applying the concept of uniform stability, resulting in an upper bound of $2\beta_{n}$. Regarding equation 20, for ease of notation, let us represent $\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)$ as $Q$. Consequently, we obtain the following:

$$|\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left(S_{0,\alpha}^{j}\right)}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)\right]$$

$$-\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left((S_{0,\alpha}^{t})^{j}\right)}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)\right]|$$

$$=\left|\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\left[\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left(S_{0,\alpha}^{j}\right)}Q-\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left((S_{0,\alpha}^{t})^{j}\right)}Q\right]\right|$$

$$=\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\left|\int_{S_{i,1-\alpha}}\left(\mathbb{P}\left(S_{i,1-\alpha}\mid S^{j}_{0,\alpha}\right)-\mathbb{P}\left(S_{i,1-\alpha}\mid\left(S^{t}_{0,\alpha}\right)^{j}\right)\right)QdS_{i,1-\alpha}\right|$$

$$\leq\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\left[\int_{S_{i,1-\alpha}}\left|\left(\mathbb{P}\left(S_{i,1-\alpha}\mid S^{j}_{0,\alpha}\right)-\mathbb{P}\left(S_{i,1-\alpha}\mid\left(S^{t}_{0,\alpha}\right)^{j}\right)\right)Q\right|dS_{i,1-\alpha}\right]$$

$$\leq M\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\left[\int_{S_{i,1-\alpha}}\left|\left(\mathbb{P}\left(S_{i,1-\alpha}\mid S^{j}_{0,\alpha}\right)-\mathbb{P}\left(S_{i,1-\alpha}\mid\left(S^{t}_{0,\alpha}\right)^{j}\right)\right)\right|dS_{i,1-\alpha}\right]$$

$$\leq 2M\sup_{j}TV\left(\mathcal{D}_{i}^{n(1-\alpha)}(S_{0}^{j}),\mathcal{D}_{i}^{n(1-\alpha)}(S_{0})\right)$$

$$=2M\gamma_{n}^{i}.$$ (21)

Thus, $h_{j}$ exhibits a bounded difference of $2\beta_{n}+2M\gamma_{n}^{i}$ with respect to all variables except the $j$-th variable. By applying Lemma 8, we obtain the following:

$$\left\|\sum_{j=1}^{n\alpha}h_{j}(S_{0,\alpha})\right\|_{p} \leq 12\sqrt{2}pn\alpha\left(2\beta_{n}+2M\gamma_{n}^{i}\right)\log(n\alpha)+4M\sqrt{pn\alpha}$$

$$\lesssim pn\alpha\left(\beta_{n}+M\gamma_{n}^{i}\right)\log(n\alpha)+M\sqrt{pn\alpha}.$$

We observe that the difference between Term 4 and $\left\|\sum_{j=1}^{n\alpha}h_{j}\left(S_{0,\alpha}\right)\right\|_{p}$ is negligible. Thus, we can bound Term 4 as follows:

$$\left\|\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}}\left[\alpha R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell\left(\mathcal{A}\left(\widetilde{S}_{i}\right),\bm{z}_{i}\right)\right]\right\|_{p}$$

$$=\left\|\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}}\left[R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\ell\left(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i}\right)\right]\right\|_{p}$$

$$\leq\left\|\frac{1}{n}\sum_{j=1}^{n\alpha}\left(\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left(S_{0,\alpha}^{j}\right)}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left(S_{0,\alpha}^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left(S_{0,\alpha}^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)\right]\right)\right\|_{p}$$

$$+\left\|2\alpha\beta_{n}+2\alpha M\gamma_{n}^{i}\right\|_{p}$$

$$=\left\|\frac{1}{n}\sum_{j=1}^{n\alpha}h_{j}(S_{0,\alpha})\right\|_{p}+\left\|2\alpha\beta_{n}+2\alpha M\gamma_{n}^{i}\right\|_{p}$$

$$\lesssim p\alpha\left(\beta_{n}+M\gamma_{n}^{i}\right)\log(n\alpha)+M\sqrt{p\alpha n^{-1}}+\alpha\beta_{n}+\alpha M\gamma_{n}^{i}$$

$$\lesssim p\alpha\left(\beta_{n}+M\gamma_{n}^{i}\right)\log(n\alpha)+M\sqrt{p\alpha n^{-1}}.$$

By substituting the above inequality and inequality 17 into the decomposition 16, we obtain:

$$\left\|\alpha R_{\mathcal{D}_{0}}\left(\mathcal{A}\left(\widetilde{S}_{i}\right)\right)-\frac{1}{n}\sum_{\bm{z}_{i}\in S_{0,\alpha}}\ell\left(\mathcal{A}\left(\widetilde{S}_{i}\right),\bm{z}_{i}\right)\right\|_{p}$$

$$\lesssim\sqrt{n(1-\alpha)p}\alpha\beta_{n}+p\alpha\left(\beta_{n}+M\gamma_{n}^{i}\right)\log(n\alpha)+M\sqrt{p\alpha n^{-1}}$$

$$\lesssim\sqrt{p}(\sqrt{(1-\alpha)n}\alpha\beta_{n}+M\sqrt{\alpha n^{-1}})+p\alpha\left(\beta_{n}+M\gamma_{n}^{i}\right)\log(n\alpha).$$ (22)

Plug inequalities 22 and 15 into the inequality 57, then we obtain:

$$\|R_{\widetilde{\mathcal{D}}_{i}} (\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\|_{p}$$

$$\lesssim p(1-\alpha)\beta_{n}\log(n(1-\alpha))+M\sqrt{p(1-\alpha)n^{-1}}+\sqrt{p}(\sqrt{(1-\alpha)n}\alpha\beta_{n}+M\sqrt{\alpha n^{-1}})$$

$$+p\alpha\left(\beta_{n}+M\gamma_{n}^{i}\right)\log(n\alpha)$$

$$= \sqrt{p}\left(\sqrt{(1-\alpha)n}\alpha\beta_{n}+Mn^{-1/2}(\sqrt{1-\alpha}+\sqrt{\alpha})\right)$$

$$+p\left((1-\alpha)\beta_{n}\log(n(1-\alpha))+\alpha\left(\beta_{n}+M\gamma_{n}^{i}\right)\log(n\alpha)\right).$$ (23)

By applying Lemma 8, we can derive a bound on the generalization error with respect to the mixed distribution.$\left|R_{\widetilde{\mathcal{D}}_{i}}\left(\mathcal{A}\left(\widetilde{S}_{i}\right)\right)-\widehat{R}_{\widetilde{S}_{i}}\left(\mathcal{A}\left(\widetilde{S}_{i}\right)\right)\right|$ as follows.

$$\left|R_{\widetilde{\mathcal{D}}_{i}}\left(\mathcal{A}\left(\widetilde{S}_{i}\right)\right)-\widehat{R}_{\widetilde{S}_{i}}\left(\mathcal{A}\left(\widetilde{S}_{i}\right)\right)\right|$$

$$\lesssim\left(\sqrt{(1-\alpha)n}\alpha\beta_{n}+Mn^{-1/2}(\sqrt{1-\alpha}+\sqrt{\alpha})\right)\sqrt{\log\left(\frac{1}{\delta}\right)}$$

$$+\left((1-\alpha)\beta_{n}\log(n(1-\alpha))+\alpha\left(\beta_{n}+M\gamma_{n}^{i}\right)\log(n\alpha)\right)\log\left(\frac{1}{\delta}\right).$$

Finally, we conclude that:

$$\left|R_{\mathcal{D}_{0}}\left(A\left(\widetilde{S}_{i}\right)\right)-\widehat{R}_{\widetilde{S}_{i}}\left(A\left(\widetilde{S}_{i}\right)\right)\right|$$

$$\leq\left|R_{\mathcal{D}_{0}}\left(A\left(\widetilde{S}_{i}\right)\right)-R_{\widetilde{\mathcal{D}}_{i}}\left(A\left(\widetilde{S}_{i}\right)\right)\right|+\left|R_{\widetilde{\mathcal{D}}_{i}}\left(A\left(\widetilde{S}_{i}\right)\right)-\widehat{R}_{\widetilde{S}_{i}}\left(A\left(\widetilde{S}_{i}\right)\right)\right|$$

$$\leq 2M\left(1-(1-\alpha)^{i}\right)\alpha^{-1}d_{\mathrm{TV}}(n)$$

$$+\left((1-\alpha)\beta_{n}\log(n(1-\alpha))+\alpha\left(\beta_{n}+M\gamma_{n}^{i}\right)\log(n\alpha)\right)\log\left(\frac{1}{\delta}\right)$$

$$+\left(\sqrt{(1-\alpha)n}\alpha\beta_{n}+Mn^{-1/2}(\sqrt{1-\alpha}+\sqrt{\alpha})\right)\sqrt{\log\left(\frac{1}{\delta}\right)}.$$ (24)

∎

A.7 Proof of Theorem 2

In this section, we prove that transformers in in-context learning exhibit recursive stability. Specifically, we utilize the framework and lemmas from Li et al. (2023), combined with recursive techniques, to establish the proof.

Lemma 10 (Li et al. (2023)).

Let $\bm{z},\bm{\varepsilon}\in\mathbb{R}^{n}$ be vectors obeying $\|\bm{z}\|_{\ell_{\infty}},\|\bm{z}+\bm{\varepsilon}\|_{\ell_{\infty}}\leq c$. Then, there exists a constant $C=C(c)$, such that

$$\|\operatorname{softmax}(\bm{z})\|_{\ell_{\infty}}\leq e^{2c}/n\quad\text{ and }\quad\|\operatorname{softmax}(\bm{z})-\operatorname{softmax}(\bm{z}+\bm{\varepsilon})\|_{\ell_{1}}\leq e^{2c}\|\bm{\varepsilon}\|_{\ell_{1}}/n.$$

Proof of Theorem 2.

. Let $\bm{Z}=\left[\bm{z}_{1},\ldots,\bm{z}_{n}\right]^{\top}$ and $\bm{E}=\left[\bm{\varepsilon}_{1},\ldots,\bm{\varepsilon}_{n}\right]^{\top}$ be the input and perturbation matrices respectively. Given that the tokens $\bm{z}_{i}$ lie in the unit ball, and assuming $\bm{z}_{i}+\bm{\varepsilon}_{i}$ also lies in the unit ball, we can proceed with the following. For a matrix, let $\|\cdot\|_{2,p}$ denote the $\ell_{p}$-norm of the vector formed by the $\ell_{2}$-norms of its rows. Therefore, we obtain $\|\bm{Z}\|_{2,\infty}\leq 1$ and $\|\bm{\bar{Z}}\|_{2,\infty}=\|\bm{Z}+\bm{E}\|_{2,\infty}\leq$ 1. Let the attention outputs be defined as $\bm{A}=\operatorname{softmax}\left(\bm{Z}\bm{W}\bm{Z}^{\top}\right)\bm{Z}\bm{V}$ and $\bar{\bm{A}}=\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}\bar{\bm{Z}}^{\top}\right)\bar{\bm{Z}}\bm{V}$. Define the perturbation as $\bar{\bm{E}}=\bar{\bm{A}}-\bm{A}:=\left[\bar{\bm{\varepsilon}}_{1},\ldots\bar{\bm{\varepsilon}}_{n}\right]^{\top}$.

Let us examine the attention output difference $\bar{\bm{E}}=\bar{\bm{A}}-\bm{A}$, which can be further decomposed as follows:

$$\bm{\bar{E}} =\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}\bar{\bm{Z}}^{\top}\right)\bar{\bm{Z}}\bm{V}-\operatorname{softmax}\left(\bm{Z}\bm{W}\bm{Z}^{\top}\right)\bm{Z}\bm{V}$$

$$=\underbrace{\left[\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}\bar{\bm{Z}}^{\top}\right)-\operatorname{softmax}\left(\bm{Z}\bm{W}\bm{Z}^{\top}\right)\right]\bm{Z}\bm{V}}_{\bm{\bar{E}_{1}}}+\underbrace{\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}\bar{\bm{Z}}^{\top}\right)\bm{E}\bm{V}}_{\bm{\bar{E}_{2}}}.$$ (25)

We first observe that $\bm{V}$ preserves the norm, meaning that $\bm{Z}\bm{V}$ satisfies $\|\bm{Z}\bm{V}\|_{2,\infty}\leq$ $\|\bm{Z}\|_{2,\infty}\leq 1$ and $\|\bm{E}\bm{V}\|_{2,1}\leq\|\bm{E}\|_{2,1}$. Moreover, for any pair of tokens, it holds that $\left|\bm{z}_{i}^{\top}\bm{W}\bm{z}_{j}\right|\leq B_{W}$. Applying Lemma 10, we can therefore derive the following:

$$\left\|\bm{\bar{E}}_{2}\right\|_{2,1}=\left\|\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}\bar{\bm{Z}}^{\top}\right)\bm{E}\bm{V}\right\|_{2,1}\leq e^{2B_{W}}\|\bm{E}\|_{2,1}.$$ (26)

Subsequently, for $\bm{\bar{E}}_{1}$, we establish the following expression

$$\left\|\bm{\bar{E}}_{1}\right\|_{2,1} =\left\|[\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}\bar{\bm{Z}}^{\top}\right)-\operatorname{softmax}\left(\bm{Z}\bm{W}\bm{Z}^{\top}\right)]\bm{Z}\bm{V}\right\|_{2,1}$$

$$\leq\left\|\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}\bar{\bm{Z}}^{\top}\right)-\operatorname{softmax}\left(\bm{Z}\bm{W}\bm{Z}^{\top}\right)\right\|_{\ell_{1}}\|\bm{Z}\bm{V}\|_{2,\infty}$$

$$\leq\left\|\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}\bar{\bm{Z}}^{\top}\right)-\operatorname{softmax}\left(\bm{Z}\bm{W}\bm{Z}^{\top}\right)\right\|_{\ell_{1}}.$$

To advance the analysis, we introduce the $\delta$-scaled perturbation $\bm{E}^{\prime}=\delta\bm{E}=\bar{\bm{Z}}^{\prime}-\bm{Z}$, where $\delta$ is constrained within $0\leq\delta\leq 1$. Our approach involves first bounding the derivative as $\delta\rightarrow 0$, and then integrating this bound along the path of $\bm{E}$, effectively covering the interval from $\delta=0$ to $\delta=1$. Notably, as $\delta\rightarrow 0$, the quadratic terms proportional to $\delta^{2}\bm{E}$ diminish, simplifying the analysis at this limit.

$$\left\|\operatorname{softmax}\left(\bar{\bm{Z}}^{\prime}\bm{W}\bar{\bm{Z}}^{\prime\top}\right)-\operatorname{softmax}\left(\bm{Z}\bm{W}\bm{Z}^{\top}\right)\right\|_{\ell_{1}}$$

$$\leq\left\|\operatorname{softmax}(\bar{\bm{Z}}^{\prime}\bm{W}\bm{Z}^{\top})-\operatorname{softmax}(\bm{Z}\bm{W}\bm{Z}^{\top})\right\|_{\ell_{1}}+\left\|\operatorname{softmax}(\bm{Z}\bm{W}\bar{\bm{Z}}^{\prime\top})-\operatorname{softmax}(\bm{Z}\bm{W}\bm{Z}^{\top})\right\|_{\ell_{1}}.$$

To bound the latter, we focus on each row separately. Consider a row from $\bm{Z}$ and its perturbed version $\bm{Z}+\bm{E}^{\prime}$, represented by the pair $\left(\bm{z},\bm{z}+\bm{\varepsilon^{\prime}}\right)$. It follows that for any cross product, we have the guarantees $\left|\left(\bm{z}+\bm{\varepsilon^{\prime}}\right)^{\top}\bm{W}\bm{z}_{i}\right|\leq B_{W}$ and $\left|\bm{z}^{\top}\bm{W}\bm{z}_{i}\right|\leq B_{W}$. Additionally, the bounds $\left\|\bm{\varepsilon}^{\prime\top}\bm{W}\bm{Z}\right\|_{\ell_{1}}\leq B_{W}n\left\|\bm{\varepsilon}^{\prime}\right\|_{\ell_{2}}$ and $\left\|\bm{z}^{\top}\bm{W}\bm{E}^{\prime\top}\right\|_{\ell_{1}}\leq B_{W}\left\|\bm{E}^{\prime}\right\|_{2,1}$ hold. Applying the perturbation bounds provided by Lemma 10, we obtain the desired result

$$\left\|\operatorname{softmax}\left(\left(\bm{z}+\bm{\varepsilon}^{\prime}\right)^{\top}\bm{W}\bm{Z}^{\top}\right)-\operatorname{softmax}\left(\bm{z}^{\top}\bm{W}\bm{Z}^{\top}\right)\right\|_{\ell_{1}}\leq B_{W}e^{2B_{W}}\left\|\bm{\varepsilon}^{\prime}\right\|_{\ell_{2}}$$

$$\left\|\operatorname{softmax}\left(\bm{z}^{\top}\bm{W}\left(\bm{Z}+\bm{E}^{\prime}\right)^{\top}\right)-\operatorname{softmax}\left(\bm{z}^{\top}\bm{W}\bm{Z}^{\top}\right)\right\|_{\ell_{1}}\leq B_{W}e^{2B_{W}}\left\|\bm{E}^{\prime}\right\|_{2,1}/n.$$

Summing across all $n$ rows, we obtain the following:

$$\lim_{\delta\rightarrow 0}\left\|\operatorname{softmax}\left((\bm{Z}+\delta\bm{E})\bm{W}\bar{\bm{Z}}^{\top}\right)-\operatorname{softmax}\left(\bm{Z}\bm{W}\bm{Z}^{\top}\right)\right\|_{\ell_{1}}/\delta\leq 2B_{W}e^{2B_{W}}\|\bm{E}\|_{2,1}.$$

By integrating the derivative over the interval $\delta=0$ to $\delta=1$, we obtain the final expression,

$$\left\|\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}\bar{\bm{Z}}^{\top}\right)-\operatorname{softmax}\left(\bm{Z}\bm{W}\bm{Z}^{\top}\right)\right\|_{\ell_{1}}\leq 2B_{W}e^{2B_{W}}\|\bm{E}\|_{2,1}.$$ (27)

By substituting inequality 27 and inequality 26 into the decomposition 25, we derive the following result:

$$\|\bm{\bar{A}}-\bm{A}\|_{2,1}=\|\bar{\bm{E}}\|_{2,1}\leq(2B_{W}+1)e^{2B_{W}}\|\bm{E}\|_{2,1}.$$ (28)

To continue, we aim to control the output for a specific index $j$ where the input perturbation remains small, specifically $\left\|\bm{\varepsilon}_{j}\right\|_{\ell_{2}}\leq\frac{\|\bm{E}\|_{2,1}}{n}$. To address this, we will apply the same argument, focusing on the $j$-th token. For the $j$-th token (omitting subscripts for clarity), let the inputs be denoted as $\bm{z},\bm{\bar{z}},\bm{\varepsilon}=\bm{\bar{z}}-\bm{z}$, and the corresponding outputs as $\bm{a},\bar{\bm{a}},\bar{\bm{\varepsilon}}=\bar{\bm{a}}-\bm{a}$. Similar to the previous decomposition, we can derive the following:

$$\bm{\bar{\varepsilon}}=\underbrace{\bm{V}^{\top}\bm{Z}^{\top}\left[\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}^{\top}\bar{\bm{Z}}\right)-\operatorname{softmax}\left(\bm{Z}\bm{W}^{\top}\bm{Z}\right)\right]}_{\bm{\bar{\varepsilon}}_{1}}+\underbrace{\bm{V}^{\top}\bm{E}^{\top}\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}^{\top}\bar{\bm{Z}}\right)}_{\bm{\bar{\varepsilon}}_{2}}.$$ (29)

By leveraging the fact that $\left|\bm{z}_{i}^{\top}\bm{W}\bm{z}_{j}\right|\leq B_{W}$ for all $i,j$, and applying Lemma 10, we can establish a bound similar to that in equation 26. Specifically, we can constrain the terms involved as follows:

$$\left\|\bm{\bar{\varepsilon}}_{2}\right\|_{\ell_{2}}\leq\left\|\bm{E}^{\top}\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}^{\top}\bar{\bm{z}}\right)\right\|_{\ell_{2}}\leq\frac{e^{2B_{W}}}{n}\|\bm{E}\|_{2,1}.$$ (30)

Similarly, for $\bar{\bm{\varepsilon}}_{1}$, we can derive the following:

$$\left\|\bar{\bm{\varepsilon}}_{1}\right\|_{\ell_{2}} \leq\left\|\bm{Z}^{\top}\left[\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}^{\top}\bar{\bm{z}}\right)-\operatorname{softmax}\left(\bm{Z}\bm{W}^{\top}\bm{z}\right)\right]\right\|_{\ell_{2}}$$

$$\leq\|\bm{Z}\|_{2,\infty}\left\|\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}^{\top}\bar{\bm{z}}\right)-\operatorname{softmax}\left(\bm{Z}\bm{W}^{\top}\bm{z}\right)\right\|_{\ell_{1}}$$

$$\leq\left\|\operatorname{softmax}\left(\bar{\bm{Z}}\bm{W}^{\top}\bar{\bm{z}}\right)-\operatorname{softmax}\left(\bm{Z}\bm{W}^{\top}\bm{z}\right)\right\|_{\ell_{1}}.$$

Now, considering the perturbation $\bm{E}^{\prime}=\delta\bm{E}$, and letting $\delta\rightarrow 0$, we apply the triangle inequality to obtain the following result:

$$\lim_{\delta\rightarrow 0} \delta^{-1}\left\|\operatorname{softmax}\left((\bm{Z}+\delta\bm{E})\bm{W}^{\top}(\bm{z}+\delta\bm{\varepsilon})\right)-\operatorname{softmax}\left(\bm{Z}\bm{W}^{\top}\bm{z}\right)\right\|_{\ell_{1}}$$

$$\leq \lim_{\delta\rightarrow 0}\delta^{-1}\left\|\operatorname{softmax}\left((\bm{Z}+\delta\bm{E})\bm{W}^{\top}\bm{z}\right)-\operatorname{softmax}\left(\bm{Z}\bm{W}^{\top}\bm{z}\right)\right\|_{\ell_{1}}$$

$$\quad+\delta^{-1}\left\|\operatorname{softmax}\left(\bm{Z}\bm{W}^{\top}(\bm{z}+\delta\bm{\varepsilon})\right)-\operatorname{softmax}\left(\bm{Z}\bm{W}^{\top}\bm{z}\right)\right\|_{\ell_{1}}$$

$$\leq B_{W}e^{2B_{W}}\|\bm{E}\|_{2,1}/n+B_{W}e^{2B_{W}}\|\bm{\varepsilon}\|_{\ell_{2}}$$

$$\leq 2B_{W}e^{2B_{W}}\|\bm{E}\|_{2,1}/n.$$ (31)

In a similar manner to the previous steps, we can derive the following:

$$\left\|\bar{\bm{\varepsilon}}\right\|_{\ell_{2}}\leq\frac{1}{n}(2B_{W}+1)e^{2B_{W}}\|\bm{E}\|_{2,1}.$$ (32)

Next, we examine the effect of the MLP layer on the model’s behavior. Let $\left(\bm{M}_{i}\right)_{i=1}^{n}\in\mathbb{R}^{d\times d}$ represent the weights of the parallel MLPs that follow the self-attention mechanism. Given that $\left\|\bm{M}_{i}\right\|\leq 1$, we denote the MLP outputs corresponding to the self-attention results $\bm{A}$ and $\bm{\bar{A}}$ as $\bm{U}$ and $\bm{\bar{U}}$, respectively. From this, we can derive the following relationship.

Let $\phi$ denote the ReLU function, which is a 1-Lipschitz continuous activation function with $\phi(0)=0$. First, observe that each row of $\bm{U}$ is given by $\bm{u}_{i}=\phi\left(\bm{M}_{i}\bm{a}_{i}\right)$, where $\bm{M}_{i}\in\mathbb{R}^{d\times d}$ represents the weights of the MLPs. Given the properties of the ReLU function, we can derive the following bound:

$$\left\|\bm{u}_{i}\right\|_{\ell_{2}}\leq\left\|\phi\left(\bm{M}_{i}\bm{a}_{i}\right)\right\|_{\ell_{2}}\leq\left\|\bm{M}_{i}\bm{a}_{i}\right\|_{\ell_{2}}\leq\left\|\bm{a}_{i}\right\|_{\ell_{2}}\leq 1.$$

Next, we consider the difference between the perturbed and original outputs. We can express the difference as $\left\|\bm{u}_{i}-\bm{\bar{u}}_{i}\right\|_{\ell_{2}}\leq\left\|\phi\left(\bm{M_{i}}\bm{a_{i}}\right)-\phi\left(\bm{M_{i}}\bm{\bar{a}_{i}}\right)\right\|_{\ell_{2}}$, which, due to the 1-Lipschitz property of $\phi$, is further bounded by $\left\|\bm{M}_{i}\left(\bm{a}_{i}-\bm{\bar{a}}_{i}\right)\right\|_{\ell_{2}}\leq\left\|\bm{a}_{i}-\bm{\bar{a}}_{i}\right\|_{\ell_{2}}$. Finally, we obtain:

$$\left\|\bm{u}_{i}-\bm{\bar{u}}_{i}\right\|_{\ell_{2}}\leq\left\|\bm{a}_{i}-\bm{\bar{a}}_{i}\right\|_{\ell_{2}}.$$ (33)

Thus, we conclude that the perturbations in the rows of $\bm{U}$ are controlled by the corresponding perturbations in $\bm{A}$. Consequently, we establish the bound

$$\|\bm{U}-\bm{\bar{U}}\|_{2,1}\leq\|\bm{A}-\bar{\bm{A}}\|_{2,1}.$$

Thus, from inequality 28, we derive the following result:

$$\|\bm{U}-\bm{\bar{U}}\|_{2,1}\leq\left(2B_{W}+1\right)e^{2B_{W}}\|\bm{E}\|_{2,1}.$$ (34)

Furthermore, for any $i\in[n]$ such that $\left\|\bm{\varepsilon}_{i}\right\|_{\ell_{2}}\leq\frac{\|\bm{E}\|_{2,1}}{n}$, it holds that

$$\left\|\bm{u}_{i}-\bar{\bm{u}}_{i}\right\|_{\ell_{2}}\leq\frac{1}{n}(2B_{W}+1)e^{2B_{W}}\|\bm{E}\|_{2,1},$$

where $\bm{u}_{i}$ represents the $i$-th row of $\bm{U}$. With this, we have addressed the stability of the single-layer transformer. Moving forward, we will extend our analysis and focus on the stability of $L$-layer transformer. First, we can derive the following:

$$\left\|\bm{Z}_{(k)}-\bar{\bm{Z}}_{(k)}\right\|_{2,1}\leq(1+2B_{W})e^{2B_{W}}\left\|\bm{Z}_{(k-1)}-\bar{\bm{Z}}_{(k-1)}\right\|_{2,1},$$

where $1\leq k\leq L$ represents the number of layers in the transformer. Then, for $L$-layer transformer, we have the following:

$$\left\|\bm{Z}_{(L)}-\bar{\bm{Z}}_{(L)}\right\|_{2,1}\leq((1+2B_{W})e^{2B_{W}})^{L}\left\|\bm{Z}_{(0)}-\bar{\bm{Z}}_{(0)}\right\|_{2,1}$$

What remains is to perform induction on the difference between the last tokens $\bm{z}_{n}^{(i)}-\bm{z}_{n}^{\prime(i)}$. We claim that, for all layers,

$$\left\|\bm{z}_{n}^{(i)}-\bm{z}_{n}^{\prime(i)}\right\|_{\ell_{2}}\leq\frac{1}{n}((1+2B_{W})e^{2B_{W}})^{i}\left\|\bm{Z}_{(0)}-\bar{\bm{Z}}_{(0)}\right\|_{2,1}.$$

This claim holds at $i=0$ because the change in the last token is at most $\left\|\bm{Z}_{(0)}-\bar{\bm{Z}}_{(0)}\right\|_{2,1}/n$. By induction, the claim holds for all layers, and we conclude the proof by setting $i=L$, covering the entire depth of the $L$-layer transformer. Finally, we obtain:

$$\left\|\bm{z}_{n}^{(L)}-\bm{z}_{n}^{\prime(L)}\right\|_{\ell_{2}}\leq\frac{1}{n}((1+2B_{W})e^{2B_{W}})^{L}\left\|\bm{Z}_{(0)}-\bar{\bm{Z}}_{(0)}\right\|_{2,1}.$$ (35)

Next, we further analyze the self-consuming process. Let $S_{0}=[\bm{z}_{0,1},...,\bm{z}_{0,j},...,\bm{z}_{0,n}]^{\top}$ and $S_{0}^{\prime}=[\bm{z}_{0,1},...,\bm{z}_{0,j}^{\prime},...,\bm{z}_{0,n}]^{\top}$ represent two initial real datasets that differ only in their inputs, specifically $\bm{z}_{0,j}=\left(\bm{x}_{0,j},\bm{y}_{0,j}\right)$ and $\bm{z}_{0,j}^{\prime}=\left(\bm{x}_{0,j}^{\prime},\bm{y}_{0,j}^{\prime}\right)$, where $j\leq n$. Since $\|S_{0}-S_{0}^{\prime}\|_{2,1}\leq 2$, then, we have the following:

$$\left\|\mathrm{TF}\left(S_{0}\right)-\mathrm{TF}\left(S_{0}^{\prime}\right)\right\|_{\ell_{2}} \leq\frac{1}{2n+1}\left((1+2B_{W})e^{2B_{W}}\right)^{L}\|S_{0}-S_{0}^{\prime}\|_{2,1}$$ (36)

$$\leq\frac{2}{2n+1}\left((1+2B_{W})e^{2B_{W}}\right)^{L}.$$

Then, $S_{0}$ and $S_{0}^{\prime}$ are used as in-context examples, and i.i.d. queries $\{\bm{x}_{1,j}\}_{j=1}^{n}$ are sampled from $\mathcal{X}$. These queries, along with the in-context examples $S_{0}$ and $S_{0}^{\prime}$, are processed through the transformer model to predict their respective labels. As a result, the first generation of synthetic datasets, $S_{1}=[\bm{z}_{1,1},...,\bm{z}_{1,j},...,\bm{z}_{1,n}]^{\top}$ and $S_{1}^{\prime}=[\bm{z}_{1,1}^{\prime},...,\bm{z}_{1,j}^{\prime},...,\bm{z}_{1,n}^{\prime}]^{\top}$, is produced. Then we obtain:

$$\|S_{1}-S_{1}^{\prime}\|_{2,1}\leq\frac{2n}{2n+1}\left((1+2B_{W})e^{2B_{W}}\right)^{L}.$$ (37)

Given the mixed dataset $\widetilde{S}_{j}$, where $\widetilde{S}_{j}=\alpha S_{0}+(1-\alpha)S_{j}$ for $1\leq j\leq i$, we can proceed with further analysis based on the specified combination of the original dataset $S_{0}$ and the synthetic dataset $S_{j}$.

$$\|\widetilde{S}_{1}-\widetilde{S}_{1}^{\prime}\|_{2,1} \leq\alpha\|S_{0}-S_{0}^{\prime}\|_{2,1}+(1-\alpha)\|S_{1}-S_{1}^{\prime}\|_{2,1}$$

$$\leq 2\alpha+(1-\alpha)\frac{2n}{2n+1}\left((1+2B_{W})e^{2B_{W}}\right)^{L}.$$ (38)

By reintroducing the mixed datasets $\widetilde{S}_{1}$ and $\widetilde{S}_{1}^{\prime}$ as in-context examples into the transformer model, and considering the query set $\left\{\bm{x}_{2,j}\right\}_{j=1}^{n}$ as i.i.d. samples from the distribution $\mathcal{X}$, we can derive the transformer’s output according to Equation 36:

$$\left\|\mathrm{TF}\left(\widetilde{S}_{1}\right)-\mathrm{TF}\left(\widetilde{S}_{1}^{\prime}\right)\right\|_{\ell_{2}}$$

$$\leq\frac{1}{2n+1}\left((1+2B_{W})e^{2B_{W}}\right)^{L}\|\widetilde{S}_{1}-\widetilde{S}_{1}^{\prime}\|_{2,1}$$

$$\leq\frac{1}{2n+1}\left((1+2B_{W})e^{2B_{W}}\right)^{L}\left(2\alpha+(1-\alpha)\frac{2n}{2n+1}\left((1+2B_{W})e^{2B_{W}}\right)^{L}\right)$$

$$\leq(1-\alpha)\frac{2n}{(2n+1)^{2}}\left((1+2B_{W})e^{2B_{W}}\right)^{2L}+\alpha\frac{2}{2n+1}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{L}.$$ (39)

From the above expression, we can further derive that

$$\left\|S_{2}-S_{2}^{\prime}\right\|_{2,1}\leq(1-\alpha)\frac{2n^{2}}{(2n+1)^{2}}\left((1+2B_{W})e^{2B_{W}}\right)^{2L}+\alpha\frac{2n}{2n+1}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{L}.$$

Thus,

$$\|\widetilde{S}_{2}-\widetilde{S}_{2}^{\prime}\|_{2,1}$$

$$\leq\alpha\|S_{0}-S_{0}^{\prime}\|_{2,1}+(1-\alpha)\|S_{2}-S_{2}^{\prime}\|_{2,1}$$

$$\leq 2\alpha+(1-\alpha)^{2}\frac{2n^{2}}{(2n+1)^{2}}\left((1+2B_{W})e^{2B_{W}}\right)^{2L}+\alpha(1-\alpha)\frac{2n}{2n+1}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{L}.$$

Similarly, for the 2-th generation, following analogous steps, we can derive that

$$\left\|\mathrm{TF}\left(\widetilde{S}_{2}\right)-\mathrm{TF}\left(\widetilde{S}_{2}^{\prime}\right)\right\|_{\ell_{2}}$$

$$\leq\frac{1}{2n+1}\left((1+2B_{W})e^{2B_{W}}\right)^{L}\|\widetilde{S}_{2}-\widetilde{S}_{2}^{\prime}\|_{2,1}$$

$$\leq(1-\alpha)^{2}\frac{2n^{2}}{(2n+1)^{3}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{3L}+\alpha(1-\alpha)\frac{2n}{(2n+1)^{2}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{2L}$$

$$\quad+\alpha\frac{2}{2n+1}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{L}.$$ (40)

Building on the above expression, we can further deduce that

$$\left\|S_{3}-S_{3}^{\prime}\right\|_{2,1}$$

$$\leq(1-\alpha)^{2}\frac{2n^{3}}{(2n+1)^{3}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{3L}+\alpha(1-\alpha)\frac{2n^{2}}{(2n+1)^{2}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{2L}$$

$$\quad+\alpha\frac{2n}{2n+1}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{L}.$$ (41)

The discrepancy between the mixed datasets is as follows:

$$\|\widetilde{S}_{3}-\widetilde{S}_{3}^{\prime}\|_{2,1}$$

$$\leq\alpha\|S_{0}-S_{0}^{\prime}\|_{2,1}+(1-\alpha)\|S_{3}-S_{3}^{\prime}\|_{2,1}$$

$$\leq(1-\alpha)^{3}\frac{2n^{3}}{(2n+1)^{3}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{3L}+\alpha(1-\alpha)^{2}\frac{2n^{2}}{(2n+1)^{2}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{2L}$$

$$\quad+\alpha(1-\alpha)\frac{2n}{2n+1}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{L}+2\alpha.$$ (42)

Utilizing recursive techniques, we can obtain the following:

$$\|\widetilde{S}_{i}-\widetilde{S}_{i}^{\prime}\|_{2,1}$$

$$\leq(1-\alpha)^{i}\frac{2n^{i}}{(2n+1)^{i}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{iL}+\alpha(1-\alpha)^{i-1}\frac{2n^{i-1}}{(2n+1)^{i-1}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{(i-1)L}$$

$$\quad+...+\alpha(1-\alpha)^{2}\frac{2n^{2}}{(2n+1)^{2}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{2L}+\alpha(1-\alpha)\frac{2n}{2n+1}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{L}$$

$$\quad+2\alpha$$

$$\leq 2(1-\alpha)^{i}\frac{n^{i}}{(2n+1)^{i}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{iL}$$

$$\quad+2\alpha\left[1-(1-\alpha)\frac{n}{2n+1}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{L}\right]^{-1}\left[1-(1-\alpha)^{i}\frac{n^{i}}{(2n+1)^{i}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{iL}\right].$$ (43)

Ultimately, the discrepancy between the transformer outputs after $i$ generations of the self-consuming loop for $S_{0}$ and $S_{0}^{\prime}$ can be obtained as follows:

$$\left\|\mathrm{TF}\left(\widetilde{S}_{i}\right)-\mathrm{TF}\left(\widetilde{S}_{i}^{\prime}\right)\right\|_{\ell_{2}}$$

$$\leq\frac{1}{2n+1}\left((1+2B_{W})e^{2B_{W}}\right)^{L}\|\widetilde{S}_{i}-\widetilde{S}_{i}^{\prime}\|_{2,1}$$

$$\leq(1-\alpha)^{i}\frac{2n^{i}}{(2n+1)^{i+1}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{(i+1)L}+\alpha(1-\alpha)^{i-1}\frac{2n^{i-1}}{(2n+1)^{i}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{iL}$$

$$\quad+...+\alpha(1-\alpha)^{2}\frac{2n^{2}}{(2n+1)^{3}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{3L}+\alpha(1-\alpha)\frac{2n}{(2n+1)^{2}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{2L}$$

$$\quad+2\alpha\frac{1}{2n+1}\left((1+2B_{W})e^{2B_{W}}\right)^{L}$$

$$\leq 2(1-\alpha)^{i}\frac{n^{i}}{(2n+1)^{i+1}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{(i+1)L}+2\alpha\left[\frac{1}{2n+1}\left((1+2B_{W})e^{2B_{W}}\right)^{L}\right]$$

$$\times\left[1-(1-\alpha)\frac{n}{2n+1}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{L}\right]^{-1}\left[1-(1-\alpha)^{i}\frac{n^{i}}{(2n+1)^{i}}\left(\left(1+2B_{W}\right)e^{2B_{W}}\right)^{iL}\right].$$

Subsequently, given that $\widetilde{B}_{W}=(1+2B_{W})e^{2B_{W}}$, we define the measure $d$ as the $\ell_{2}$-norm to quantify the output discrepancy of the generative transformer model after $i$ iterations of the self-consuming loop, starting from the initial real datasets $S_{0}$ and $S_{0}^{\prime}$. In this context, the recursive stability parameter $\gamma_{n}^{i}$, as described in Definition 2, can be bounded by the following expression, providing a formal measure of the model’s stability across iterations:

$$\left\|\operatorname{TF}(\widetilde{S}_{i})-\operatorname{TF}(\widetilde{S}_{i}^{\prime})\right\|_{\ell_{2}}\lesssim(1-\alpha)^{i}\frac{\widetilde{B}_{W}^{(i+1)L}}{2n+1}.$$

The proof is complete.

∎

A.8 Proof of Theorem 3

In this section, building on the general theoretical framework established in Theorem 1, we provide the proof of Theorem 3 by analyzing the terms $\beta_{n}$ and $d_{\mathrm{TV}}(n)$, leveraging recent advancements in SGD (Zhang et al., 2022) and ICL (Zhang et al., 2023). The recursive stability parameter $\gamma_{n}^{i}$ is derived from Theorem 2.

Lemma 11.

(Uniform stability of SGD in the non-convex case (Zhang et al., 2022)). Assume $f$ is $\kappa$-smooth and $\rho$-Lipschitz. Running $T\gtrsim n$ iterations of $SGD$ with step size $\eta_{t}=\frac{1}{\beta t}$. Choose the stability of SGD satisfies

$$\beta_{n}\lesssim\frac{16\rho^{2}\log n}{n}.$$

Lemma 12.

(Zhang et al., 2023) Let $\mathbb{P}_{\theta}$ represent the probability distribution induced by the transformer with parameter $\theta$. Additionally, the model $\mathbb{P}_{\hat{\theta}}$ is pretrained by the algorithm:

$$\hat{\theta}=\underset{\theta\in\Theta}{\operatorname{argmin}}-\frac{1}{n}\sum_{t=1}^{n-1}\log\mathbb{P}_{\theta}\left(\bm{x}_{t+1}^{n}\mid S_{t}^{n}\right),$$

where $S_{t}^{n}=\left(\bm{x}_{1},\bm{y}_{1},\ldots\bm{x}_{t},\bm{y}_{t}\right)$. Furthermore, we consider the realizable setting, where ground truth probability distribution $\mathbb{P}(\cdot\mid S)$ and $\mathbb{P}_{\theta^{*}}(\cdot\mid S)$ are consistent for some $\theta^{*}\in\Theta$. Then, with probability at least $1-\delta$, the following inequality holds:

$$\operatorname{TV}\left(\mathbb{P}(\cdot\mid S),\mathbb{P}_{\hat{\theta}}(\cdot\mid S)\right)\lesssim\frac{1}{n^{1/2}}\log(1+n)+\frac{1}{n^{1/4}}\log(1/\delta),$$ (44)

where $\lesssim$ denotes that we omit constants that are independent of $n$ and $\delta$.

Proof of Theorem 3.

First, we note that in the setting where the transformer generates data through in-context learning, the generalization error of the self-consuming loop is given by:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|=\left|\mathbb{E}_{\bm{z}\sim\mathbb{P}(\cdot\mid S_{0})}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z})-\frac{1}{n}\sum_{\bm{z}_{i}\in\widetilde{S}_{i}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\right|.$$ (45)

Now, we are ready to prove Theorem 3. The main idea is to bound the uniform stability parameter $\beta_{n}$, the recursive stability parameter $\gamma_{n}^{i}$, and the learnability of the generative model through the total variation distance $d_{\mathrm{TV}}(n)$ as stated in Theorem 1. First, as for the bound for the total variation distance $d_{\text{TV }}(n)$ in Theorem 1. For Equation 8 in the proof of Theorem 1, we can rewrite it in the setting of in-context learning as follows:

$$\left|R_{\widetilde{\mathcal{D}}_{i-1}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right| =\left|\mathbb{E}_{\bm{z}\sim\mathbb{P}(\cdot\mid\widetilde{S}_{i-1})}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z})-\mathbb{E}_{\bm{z}\sim\mathbb{P}(\cdot\mid S_{i})}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z})\right|$$

$$=\left|\mathbb{E}_{\bm{z}\sim\mathbb{P}(\cdot\mid\widetilde{S}_{i-1})}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z})-\mathbb{E}_{\bm{z}\sim\mathbb{P}_{\hat{\theta}}(\cdot\mid\widetilde{S}_{i-1})}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z})\right|$$

$$=\Bigg{|}\int_{\bm{z}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z})\left(\mathbb{P}\left(\bm{z}\mid\widetilde{S}_{i-1}\right)-\mathbb{P}_{\hat{\theta}}\left(\bm{z}\mid\widetilde{S}_{i-1}\right)\right)d\bm{z}\Bigg{|}$$

$$\leq\int_{\bm{z}}\biggl{|}\ell(\mathcal{A}(\widetilde{S}),\bm{z})\left(\mathbb{P}\left(\bm{z}\mid\widetilde{S}_{i-1}\right)-\mathbb{P}_{\hat{\theta}}\left(\bm{z}\mid\widetilde{S}_{i-1}\right)\right)\biggr{|}d\bm{z}$$

$$\leq M\int_{\bm{z}}\Bigl{|}\mathbb{P}\left(\bm{z}\mid\widetilde{S}_{i-1}\right)-\mathbb{P}_{\hat{\theta}}\left(\bm{z}\mid\widetilde{S}_{i-1}\right)\Bigr{|}d\bm{z}$$

$$=2MTV\left(\mathbb{P}(\cdot\mid\widetilde{S}_{i-1}),\mathbb{P}_{\hat{\theta}}(\cdot\mid\widetilde{S}_{i-1})\right).$$ (46)

Where, the second equality holds because, in the $(i-1)$-th generation of the self-consuming loop, the mixed data distribution from the $(i-1)$-th generation is reintroduced as the ground truth distribution to train the transformer. As a result, the transformer outputs the synthetic data distribution for the $i$-th generation. Thus, $TV\left(\mathbb{P}(\cdot\mid\widetilde{S}_{j}),\mathbb{P}_{\hat{\theta}}(\cdot\mid\widetilde{S}_{j})\right)$ corresponds to $d_{\mathrm{TV}}(n)$ in Theorem 1. Finally, the bound for the total variation distance $d_{\text{TV }}(n)$ follows from Lemma 12.

$$d_{\text{TV }}(n)\lesssim\frac{1}{n^{1/2}}\log(1+n)+\frac{1}{n^{1/4}}\log(1/\delta).$$ (47)

Similarly, for the recursive stability parameter in the self-consuming loop, we rederive Equation 21 from the proof of Theorem 1 under the in-context learning setting:

$$|\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left(S_{0,\alpha}^{j}\right)}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)\right]$$

$$-\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left((S_{0,\alpha}^{t})^{j}\right)}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left((S_{0,\alpha}^{t})^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)\right]|$$

$$=\left|\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\left[\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left(S_{0,\alpha}^{j}\right)}\ell\left(\mathcal{A}\left(\left(S_{0,\alpha}^{t}\right)^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)\right.\right.$$

$$\left.\left.\quad-\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left((S_{0,\alpha}^{t})^{j}\right)}\ell\left(\mathcal{A}\left(\left(S_{0,\alpha}^{t}\right)^{j}\cup S_{i,1-\alpha}\right),\bm{z}\right)\right]\right|$$

$$\quad+\left|\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\left[\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left(S_{0,\alpha}^{j}\right)}\ell\left(\mathcal{A}\left(\left(S_{0,\alpha}^{t}\right)^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)\right.\right.$$

$$\left.\left.\quad-\mathbb{E}_{S_{i,1-\alpha}\sim\mathcal{D}_{i}^{n(1-\alpha)}\left((S_{0,\alpha}^{t})^{j}\right)}\ell\left(\mathcal{A}\left(\left(S_{0,\alpha}^{t}\right)^{j}\cup S_{i,1-\alpha}\right),\bm{z}_{0,j}\right)\right]\right|$$

$$\leq 2n(1-\alpha)\beta_{n}\left\|\mathrm{TF}\left(\left(S_{0,\alpha}^{t}\right)^{j}\cup S_{i-1,1-\alpha}\right)-\mathrm{TF}\left(\left(S_{0,\alpha}^{t}\right)^{j}\cup S_{i-1,1-\alpha}^{\prime}\right)\right\|_{\ell_{2}}$$

$$\lesssim 2n(1-\alpha)\beta_{n}\frac{2\widetilde{B}_{W}^{L}}{2n+1}\left[((1-\alpha)\widetilde{B}_{W}^{L})^{i-1}+\alpha\frac{1-((1-\alpha)\widetilde{B}_{W}^{L})^{i-1}}{1-(1-\alpha)\widetilde{B}_{W}^{L}}\right]$$

$$=2n(1-\alpha)\beta_{n}\gamma_{n}^{i-1}.$$ (48)

For the uniform stability parameter $\beta_{n}$ of SGD algorithm, we can derive the bound from Lemma 11. Substituting above results into Theorem 3, we obtain the following conclusion:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|$$

$$\leq\left((1-\alpha)\beta_{n}\log(n(1-\alpha))+\alpha\left(\beta_{n}+(1-\alpha)\rho^{2}\gamma_{n}^{i-1}\right)\log(n\alpha)\right)\log(\frac{1}{\delta})$$

$$\quad+\left(\sqrt{(1-\alpha)n}\alpha\beta_{n}+Mn^{-1/2}(\sqrt{1-\alpha}+\sqrt{\alpha})\right)\sqrt{\log(\frac{1}{\delta})}+2M\left(1-(1-\alpha)^{i}\right)\alpha^{-1}d_{\mathrm{TV}}(n)$$

$$\leq\beta_{n}\left[(1-\alpha)\log(n(1-\alpha))\log(\frac{1}{\delta})+\alpha\log(n\alpha)\log(\frac{1}{\delta})+\alpha\sqrt{(1-\alpha)n\log\frac{1}{\delta}}\right]$$

$$\quad+\gamma_{n}^{i-1}\alpha(1-\alpha)\rho^{2}\log(n\alpha)\log(\frac{1}{\delta})+n^{-1/2}M(\sqrt{1-\alpha}+\sqrt{\alpha})\sqrt{\log(\frac{1}{\delta})}+2d_{\mathrm{TV}}(n)M\left(1-(1-\alpha)^{i}\right)\alpha^{-1}$$

$$\lesssim n^{-1/2}\log(n)M\rho^{2}\alpha\sqrt{1-\alpha}\log\frac{1}{\delta}+n^{-1}\rho^{2}((1-\alpha)\widetilde{B}_{W}^{L})^{i}\alpha\log^{2}(n)\log\left(\frac{1}{\delta}\right)$$

$$\quad+n^{-1/4}\alpha^{-1}M\left(1-(1-\alpha)^{i}\right)\log(\frac{1}{\delta}).$$ (49)

∎

A.9 Proof of Theorem 4

In this section, we prove Theorem 4. The proof follows a similar approach to that of Theorem 3; however, it is more intricate due to the fact that the mixed dataset in Theorem 4 contains synthetic data from all previous generations. Each generation’s synthetic dataset depends on the synthetic datasets of previous generations, leading to a more complex non-i.i.d. setting. Similar to Theorem 3, we begin by decomposing the generalization error into two components: the Cumulative Distribution Shift Across Generations and the Generalization Error on Mixed Distributions.

The main proof is as follows:

Proof of Theorem 4.

We begin by decomposing the generalization error as follows:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|\leq\underbrace{\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|}_{\text{Cumulative distribution shift across generations}}+\underbrace{\left|R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|}_{\text{Generalization error on mixed distributions}}.$$

Upper Bounding Cumulative Distribution Shift Term

For the term $\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|$, we first note that $\widetilde{\mathcal{D}}_{i}=\frac{1}{1+i\lambda}\mathcal{D}_{0}+\frac{\lambda}{1+i\lambda}\mathcal{D}_{1}+\frac{\lambda}{1+i\lambda}\mathcal{D}_{2}+...+\frac{\lambda}{1+i\lambda}\mathcal{D}_{i}$. Therefore, we obtain:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|$$

$$=\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{1+i\lambda}R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i})-\frac{\lambda}{1+i\lambda}R_{\mathcal{D}_{1}}(\mathcal{A}(\widetilde{S}_{1}))-...-\frac{\lambda}{1+i\lambda}R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|$$

$$=\left|\frac{i\lambda}{1+i\lambda}R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i})-\frac{\lambda}{1+i\lambda}R_{\mathcal{D}_{1}}(\mathcal{A}(\widetilde{S}_{1}))-...-\frac{\lambda}{1+i\lambda}R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|$$

$$\leq\frac{\lambda}{1+i\lambda}\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i})-R_{\mathcal{D}_{1}}(\mathcal{A}(\widetilde{S}_{i}))\right|+...+\frac{\lambda}{1+i\lambda}\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i})-R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|$$

$$\leq\frac{\lambda}{1+i\lambda}\sum_{j=1}^{i}\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i})-R_{\mathcal{D}_{j}}(\mathcal{A}(\widetilde{S}_{i}))\right|.$$ (50)

Furthermore, we can further decompose it as follows:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i})-R_{\mathcal{D}_{j}}(\mathcal{A}(\widetilde{S}_{i}))\right|\leq\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{j-1}}(\mathcal{A}(\widetilde{S}_{i}))\right|+\left|R_{\widetilde{\mathcal{D}}_{j-1}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\mathcal{D}_{j}}(\mathcal{A}(\widetilde{S}_{i}))\right|.$$ (51)

By substituting inequality 51 into inequality 50, we obtain:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|$$

$$\leq\frac{\lambda}{1+i\lambda}\sum_{j=1}^{i}\left(\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{j-1}}(\mathcal{A}(\widetilde{S}_{i}))\right|+\left|R_{\widetilde{\mathcal{D}}_{j-1}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\mathcal{D}_{j}}(\mathcal{A}(\widetilde{S}_{i}))\right|\right).$$ (52)

Thus, from equation 46 in the proof of Theorem 3 and lemma 12, we obtain:

$$\left|R_{\widetilde{\mathcal{D}}_{j-1}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\mathcal{D}_{j}}(\mathcal{A}(\widetilde{S}_{i}))\right| \leq 2MTV\left(\mathbb{P}\left(\cdot\mid\widetilde{S}_{j-1}\right),\mathbb{P}_{\hat{\theta}}\left(\cdot\mid\widetilde{S}_{j-1}\right)\right)$$

$$\lesssim Mn_{j-1}^{-1/4}\log n_{j-1}\log(1/\delta).$$ (53)

Incorporating inequality 53 into inequality 52, we arrive at:

$$|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|$$

$$\lesssim\frac{\lambda}{1+i\lambda}\sum_{j=1}^{i}\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{j-1}}(\mathcal{A}(\widetilde{S}_{i}))\right|+\frac{\lambda}{1+i\lambda}\sum_{j=0}^{i-1}Mn_{j}^{-1/4}\log n_{j}\log(1/\delta).$$ (54)

Let $f(i)=\sum_{j=0}^{i-1}Mn_{j}^{-1/4}\log n_{j}\log(1/\delta)$, Then, we obtain:

$$|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|$$

$$\lesssim\frac{\lambda}{1+i\lambda}\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i-1}}(\mathcal{A}(\widetilde{S}_{i}))\right|+...+\frac{\lambda}{1+i\lambda}\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{1}}(\mathcal{A}(\widetilde{S}_{i}))\right|+\frac{\lambda}{1+i\lambda}f(i).$$

Similarly, we get:

$$|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i-1}}(\mathcal{A}(\widetilde{S}_{i}))|$$

$$\lesssim\frac{\lambda}{1+(i-1)\lambda}\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i-2}}(\mathcal{A}(\widetilde{S}_{i}))\right|+...+\frac{\lambda}{1+(i-1)\lambda}\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{1}}(\mathcal{A}(\widetilde{S}_{i}))\right|$$

$$\quad+\frac{\lambda}{1+(i-1)\lambda}f(i-1).$$

Then, we have

$$|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|\lesssim\frac{\lambda}{1+i\lambda}f(i)+\frac{\lambda}{1+i\lambda}\frac{\lambda}{1+(i-1)\lambda}f(i-1)+$$

$$(\frac{\lambda}{1+i\lambda}+\frac{\lambda}{1+i\lambda}\frac{\lambda}{1+(i-1)\lambda})(\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i-2}}(\mathcal{A}(\widetilde{S}_{i}))\right|+...+\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{1}}(\mathcal{A}(\widetilde{S}_{i}))\right|).$$ (55)

Thus, by applying recursive techniques, we obtain the following result:

$$|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|$$

$$\lesssim\frac{\lambda}{1+i\lambda}f(i)+\frac{\lambda}{1+i\lambda}\frac{\lambda}{1+(i-1)\lambda}f(i-1)+(\frac{\lambda}{1+i\lambda}\frac{\lambda}{1+(i-2)\lambda}+\mathcal{O}(\frac{1}{(1+i\lambda)^{2}}))f(i-2)$$

$$+...+(\frac{\lambda}{1+i\lambda}\frac{\lambda}{1+\lambda}+\mathcal{O}(\frac{1}{(1+i\lambda)}))f(1)$$

$$\lesssim\frac{\lambda}{1+i\lambda}\left[f(i)+\frac{\lambda}{1+(i-1)\lambda}f(i-1)+\frac{\lambda}{1+(i-2)\lambda}f\left(i-2\right)+...+\frac{\lambda}{1+\lambda}f(1)\right]$$

$$\lesssim M\log\frac{1}{\delta}\frac{\lambda}{1+i\lambda}\Big{[}n_{i-1}^{-\frac{1}{4}}\log(n_{i-1})+(1+\frac{\lambda}{1+(i-1)\lambda})n_{i-2}^{-\frac{1}{4}}\log(n_{i-2})+$$

$$(1+\frac{\lambda}{1+(i-1)\lambda}+\frac{\lambda}{1+(i-2)\lambda})n_{i-3}^{-\frac{1}{4}}\log(n_{i-3})+...+(1+...+\frac{\lambda}{1+\lambda})n_{0}^{-\frac{1}{4}}\log(n_{0})\Big{]}$$

$$\lesssim n^{-\frac{1}{4}}\log((1+i\lambda)n)M\log\frac{1}{\delta}.$$ (56)

Upper Bounding Generalization Error on Mixed Distributions Term

Next, we turn our attention to the term $|R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))|$. Our primary objective is to establish a moment bound for this expression.

$$\left\|R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right\|_{p}$$

$$=\Big{\|}\frac{1}{1+i\lambda}R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))+\frac{\lambda}{1+i\lambda}R_{\mathcal{D}_{1}}(\mathcal{A}(\widetilde{S}_{i}))+\frac{\lambda}{1+i\lambda}R_{\mathcal{D}_{2}}(\mathcal{A}(\widetilde{S}_{i}))...+\frac{\lambda}{1+i\lambda}R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))$$

$$\quad-\frac{1}{(1+i\lambda)n}\sum_{\bm{z}_{i}\in S_{0}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})-\frac{1}{(1+i\lambda)n}\sum_{\bm{z}_{i}\in S_{1}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})-...-\frac{1}{(1+i\lambda)n}\sum_{\bm{z}_{i}\in S_{i}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\Big{\|}_{p}$$

$$\leq\underbrace{\left\|\frac{1}{1+i\lambda}R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{(1+i\lambda)n}\sum_{\bm{z}_{i}\in S_{0}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\right\|_{p}}_{\text{Term 0}}+\underbrace{\left\|\frac{\lambda}{1+i\lambda}R_{\mathcal{D}_{1}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{(1+i\lambda)n}\sum_{\bm{z}_{i}\in S_{1}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\right\|_{p}}_{\text{Term 1}}$$

$$\quad+..+\underbrace{\left\|\frac{\lambda}{1+i\lambda}R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{(1+i\lambda)n}\sum_{\bm{z}_{i}\in S_{i}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\right\|_{p}}_{\text{Term i}}.$$ (57)

Fixing $S_{0},S_{1},\dots,S_{i-1}$, the data in $S_{i}$ are independent. Following a similar approach to the proof of Theorem 1, we utilize this property along with Lemma 8 to bound Term i. Consequently, from Equation 15 in the proof of Theorem 1, we obtain:

$$\left\|\frac{\lambda}{1+i\lambda}R_{\mathcal{D}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{(1+i\lambda)n}\sum_{\bm{z}_{i}\in S_{i}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\right\|_{p}\lesssim p\frac{\lambda}{1+i\lambda}\beta_{(1+i\lambda)n}\log(\lambda n)+\frac{M}{1+i\lambda}\sqrt{\frac{p\lambda}{n}}.$$ (58)

Next, we consider Term 0. Similar to Proof of Theorem 3, we first introduce a set of functions and apply Lemma 8 to bound Term 0. Specifically, we define $h_{j}(S)$, which serves a similar role to the $g_{i}$ ’s in Lemma 8, as follows:

$$h_{j}(S_{0})$$

$$=\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left(S_{0}^{j}\cup S_{1}\cup...\cup S_{i}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left(S_{0}^{j}\cup S_{1}\cup...\cup S_{i}\right),\bm{z}_{0,j}\right)\right],$$ (59)

where $\bm{z}_{0,j}$ denote the $j$-th data point in $S_{0}$, and $S_{0}^{j}$ represent the dataset obtained by replacing $\bm{z}_{0,j}$ with $\bm{z}_{0,j}^{\prime}$. Moreover, following the procedure above, we observe that $\left|h_{j}\right|\leq M$ and $\mathbb{E}\left[h_{j}\mid S_{0,\alpha}^{\backslash j}\right]=0$ . More intricately, we will now prove that $h_{j}$ exhibits a bounded difference. However, it is important to note that $S_{1},\ldots,S_{i}$ all depend on $S_{0}$, so when a single data point in $S_{0}$ is changed, the corresponding datasets will also change. We denote these modified datasets as $S_{1}^{\prime},\ldots,S_{i}^{\prime}$ and consequently, we have the following:

$$|h_{j}(S_{0})-h_{j}(S_{0}^{t})|$$

$$=|\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left(S_{0}^{j}\cup S_{1}\cup...\cup S_{i}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left(S_{0}^{j}\cup S_{1}\cup...\cup S_{i}\right),\bm{z}_{0,j}\right)\right]|$$

$$-\mathbb{E}_{\bm{z}_{0,j}^{\prime}\sim\mathcal{D}_{0}}\left[\mathbb{E}_{\bm{z}\sim\mathcal{D}_{0}}\ell\left(\mathcal{A}\left((S_{0}^{t})^{j}\cup S_{1}^{\prime}\cup...\cup S_{i}^{\prime}\right),\bm{z}\right)-\ell\left(\mathcal{A}\left((S_{0}^{t})^{j}\cup S_{1}^{\prime}\cup...\cup S_{i}^{\prime}\right),\bm{z}_{0,j}\right)\right]$$

$$\leq 2\beta_{(1+i\lambda)n}\Big{(}\|S_{0}^{j}-(S_{0}^{t})^{j}\|_{\ell_{2}}+\|S_{1}-S_{1}^{\prime}\|_{\ell_{2}}+...+\|S_{i}-S_{i}^{\prime}\|_{\ell_{2}}\Big{)}.$$ (60)

Thus, by applying the recursive stability established in Theorem 2, it is important to first note that in Theorem 2, the mixed dataset is defined as $\widetilde{S}_{j}=\alpha S_{0}+(1-\alpha)S_{j}$, whereas in this theorem, the mixed dataset is defined as $\widetilde{S}_{i}=\sum_{j=0}^{i}S_{j}$. Therefore, by following the proof steps outlined in Theorem 2, we can derive the following:

$$|h_{j}(S_{0})-h_{j}(S_{0}^{t})|\lesssim 2\beta_{(1+i\lambda)n}\Big{(}i!\widetilde{B}_{W}^{iL}\Big{)}.$$

Thus, we apply lemma 8:

$$\left\|\sum_{j=1}^{n}h_{j}(S_{0})\right\|_{p} \leq 12\sqrt{2}pn2\beta_{(1+i\lambda)n}\left(i!\widetilde{B}_{W}^{iL}\right)\log(n)+4M\sqrt{pn}$$

$$\lesssim p\frac{\rho^{2}}{1+i\lambda}\Big{(}i!\widetilde{B}_{W}^{iL}\Big{)}\log(n(1+i\lambda))+M\sqrt{pn}.$$

We observe that the difference between Term 0 and $\frac{1}{(1+i\lambda)n}\left\|\sum_{j=1}^{n}h_{j}(S_{0})\right\|_{p}$ is negligible. Thus, we can bound Term 0 as follows:

$$\left\|\frac{1}{1+i\lambda}R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{(1+i\lambda)n}\sum_{\bm{z}_{i}\in S_{0}}\ell(\mathcal{A}(\widetilde{S}_{i}),\bm{z}_{i})\right\|_{p}$$

$$\lesssim p\frac{\rho^{2}}{(1+i\lambda)^{2}n}\Big{(}i!\widetilde{B}_{W}^{iL}\Big{)}\log(n(1+i\lambda))+\frac{1}{1+i\lambda}M\sqrt{p/n}.$$ (61)

Using the same method, for Term $j$, where $1\leq j\leq i-1$, we can derive the following:

$$\left\|\frac{\lambda}{1+i\lambda}R_{\mathcal{D}_{j}}(\mathcal{A}(\widetilde{S}_{i}))-\frac{1}{(1+i\lambda)n}\sum_{\bm{z}_{i}\in S_{1}}\ell(\mathcal{A}(\widetilde{S}_{j}),\bm{z}_{i})\right\|_{p}$$

$$\lesssim p\frac{\rho^{2}}{(1+i\lambda)^{2}n}\Big{(}j!\widetilde{B}_{W}^{jL}\Big{)}\log(n(1+i\lambda))+\frac{1}{1+i\lambda}M\sqrt{p/n}.$$ (62)

In summary, we can finally bound the Generalization Error on the Mixed Distributions term as follows:

$$\left\|R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right\|_{p}$$

$$\lesssim p\frac{\rho^{2}}{(1+i\lambda)^{2}n}\log((1+i\lambda)n)i!\widetilde{B}_{W}^{(i+1)L}+\frac{Mi}{1+i\lambda}\sqrt{\frac{p}{n}}.$$

Then, according to Lemma 9, we obtain, with probability at least $1-\delta$:

$$\left\|R_{\widetilde{\mathcal{D}}_{i}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right\|_{p}$$

$$\lesssim\frac{\rho^{2}}{(1+i\lambda)^{2}n}\log((1+i\lambda)n)i!\widetilde{B}_{W}^{(i+1)L}\log\frac{1}{\delta}+\frac{Mi}{1+i\lambda}\sqrt{\frac{1}{n}\log\frac{1}{\delta}}.$$

Then, combine the above inequality with inequality 56, we obtain:

$$\left|R_{\mathcal{D}_{0}}(\mathcal{A}(\widetilde{S}_{i}))-\widehat{R}_{\widetilde{S}_{i}}(\mathcal{A}(\widetilde{S}_{i}))\right|$$

$$\lesssim n^{-\frac{1}{4}}\log((1+i\lambda)n)M\log\frac{1}{\delta}+\frac{\rho^{2}}{(1+i\lambda)^{2}n}\log((1+i\lambda)n)i!\widetilde{B}_{W}^{(i+1)L}\log\frac{1}{\delta}+\frac{Mi}{1+i\lambda}\sqrt{\frac{1}{n}\log\frac{1}{\delta}}$$

$$\lesssim n^{-\frac{1}{2}}\frac{Mi}{1+i\lambda}\sqrt{\log\frac{1}{\delta}}+n^{-1}\frac{\rho^{2}}{(1+i\lambda)^{2}}\log((1+i\lambda)n)i!\widetilde{B}_{W}^{(i+1)L}\log\frac{1}{\delta}$$

$$\quad+n^{-\frac{1}{4}}\log((1+i\lambda)n)M\log\frac{1}{\delta}.$$

The proof is complete.

∎

Appendix B Experiments

In this section, we present some experimental results. Specifically, we trained transformer models to in-context learn linear functions within STLs.

In these experiments, we considered the class of linear functions:

$$\mathcal{F}=\left\{f\mid f(\bm{x})=\bm{w}^{\top}\bm{x},\bm{w}\in\mathbb{R}^{d}\right\},$$

in $d=5$ dimensions. We sampled $\bm{x}_{1},\ldots,\bm{x}_{k},\bm{x}_{\text{query}}$, and $\bm{w}$ independently from the isotropic Gaussian distribution $\mathcal{N}(0,I_{d})$. For each $x_{i}$, we computed $y_{i}=\bm{w}^{\top}\bm{x}_{i}$ and constructed the prompt as:

$$P=(\bm{x}_{1},y_{1},\bm{x}_{2},y_{2},\ldots,\bm{x}_{k},y_{k},\bm{x}_{\text{query}}).$$

We employed a 12-layer, 8-head GPT-2 model with a hidden size of 256, trained on an $\mathbb{R}^{5}$ linear regression task with 40 in-context examples. Two cases were considered:

• Mixed Case: Fresh data and generated data were mixed in a 0.5 ratio.

• Full Synthetic Case: No fresh data was used.

The results of these experiments are summarized below:

$$\begin{array}[]{|c|c|c|c|c|c|c|}\hline\cr\text{Loop}&1&2&3&4&5&6\\ \hline\cr\text{Full Synthetic}&0.3817&1.4975&1.5396&2.0836&2.3912&2.8764\\ \hline\cr\text{Mixed}&0.3817&0.4208&0.4391&0.4503&0.4641&0.4702\\ \hline\cr\end{array}$$

As observed, the error accumulates progressively with more self-consuming loops, particularly in the full synthetic case, where the error grows rapidly. In contrast, maintaining a constant-sized proportion of real data effectively reduces the loss, which is consistent with our theoretical findings.