通用加噪公式

设定 $x_0$ 是原始图像,$\epsilon$ 是高斯噪声,$\epsilon \sim \mathcal{N}(0,\mathbf{I})$

观察三类扩散模型的加噪公式:

观察可发现通用形式:$p(x_t|x_0) = \mathcal{N}(x_t;s(t)x_0,s^2(t)\sigma^2(t)\mathbf{I})$

而 SDE 过程可表示为:$dx_t = f(x_t,t)dt + g(t)dw$

根据 SDE 中的推导,在 DDPM 与 SMLD 中,$f(x_t,t)=f(t)x_t$,$f(t):\mathbb{R}^1\rightarrow \mathbb{R}^1$

$$ dx_t = f(t) x_t dt + g(t) dw $$

同样,已知 SDE 的均值和方差,可以求得加噪公式的均值和协方差:

$$ \frac{d\mathbf{m}}{dt} = \mathbb{E}[f(x_t,t)]\\ \frac{d\mathbf{P}}{dt}=\mathbb{E}[f(x_t,t)(x_t-m)^T]+\mathbb{E}[(x_t-m)f(x_t,t)^T] + \mathbb{E}[g^2(t)] $$

因此可以继续推导:

$$ \begin{aligned} &d \mathbf{m} = \mathbb{E}[f(t)x_t]dt = f(t) \mathbf{m} dt\\ &\mathbf{m}(t) = Ae^{\int_0^1f(r)dr}\\ &\text{代入系数解得: } A = x_0\\ &\mathbf{m}(t) = e^{\int_0^1f(r)dr} x_0\\ &\text{因此可得: }s(t) = e^{\int_0^t f(r)dr} \end{aligned} $$

发现 s(t) 和函数 f 是相关甚至是等价的。同理分析协方差的情况:

$$ \begin{aligned} &\frac{d\mathbf{P}}{dt} = f(t)\mathbb{E}[x_tx_t^T-x_t\mathbf{m}^T+x_tx_t^T-\mathbf{m}x_t^T]+g^2(t)\\ &注意到\quad \mathbf{P} = \mathbb{E}[(x_t-\mathbf{m})(x_t-\mathbf{m})^T]\\ &\mathbb{E}[\mathbf{m}x_t^T] = m\mathbb{E}[x_t^T]\\ &\mathbb{E}[\mathbf{m}\mathbf{m}^T]=\mathbf{m}\mathbf{m}^T = \mathbf{m}\mathbb{E}[x^T_t]=\mathbb{E}[\mathbf{m}x^T_t] = \mathbb{E}[x_t\mathbf{m}^T]\\ &\mathbf{P} = \mathbb{E}[x_tx_t^T-\mathbf{m}\mathbf{m}^T]\\ &\mathbb{E}[x_tx_t^T-x_t\mathbf{m}^T+x_tx_t^T-\mathbf{m}x_t^T] = 2 \mathbf{P}\\ &\text{因此最初的式子可写成: }\ \ \frac{d\mathbf{P}}{dt} = 2f(t) \mathbf{P} + g^2(t) \end{aligned} $$

求解一阶非齐次线性常微分方程:

$$ \mathbf{P}(t) = e^{\int_0^t 2f(r)dr}\int_0^tg^2(r)e^{\int_0^t-2f(r)dr}dr $$

跟 s(t) 公式比较,可以发现:

$$ s^2(t)\sigma^2(t) = s^2(t)\int_0^t \frac{g^2(r)}{s^2(r)}dr \\ \therefore \ \sigma^2(t)=\int_0^t \frac{g^2(r)}{s^2(r)}dr $$