自编码器 (Autoencoder)Autoencoders

一台 640×480 的彩色相机，每帧给出 921 600 个数。把这样的向量直接交给控制器或强化学习算法，几乎学不动：输入维度巨大，相邻像素高度相关，而决定任务的量（物体位置、姿态、夹爪开合）只有几个。我们希望先把观测压成一个短向量 \(\mathbf z\)，让它保留“场景处在什么状态”，丢掉其余的东西，再在 \(\mathbf z\) 上做规划、学策略、检测异常。

18.1 节的 PCA 已经是一种压缩：找一个 \(k\) 维线性子空间，使数据投影上去之后丢掉的方差最少。但 PCA 只会用平直的子空间。一个圆盘在桌面上平移，这些图像构成一张只有 2 个自由度的“曲面”，可它在像素空间里弯得厉害：圆盘挪开一个直径，新旧两张图几乎不重叠，彼此近似正交。用平面去贴这种曲面，需要很多维。

自编码器（autoencoder，AE）是 PCA 的非线性推广：一个编码器网络 \(f_\phi\) 把 \(\mathbf x\) 映成短编码 \(\mathbf z\)，一个解码器网络 \(g_\theta\) 再把 \(\mathbf z\) 还原成 \(\hat{\mathbf x}\)，用 16.3 节的反向传播一起训练，目标是让 \(\hat{\mathbf x}\) 尽量接近 \(\mathbf x\)。标签就是输入本身，所以它是自监督的：机器人随手录下的视频就是训练数据。它也是本章后面几种方法的起点：VAE（19.1 节）给编码加上概率解释，使它能生成；对比学习（18.3 节）和 Masked Modeling / JEPA（18.4 节）换掉“逐像素还原”这个目标；去噪自编码器则直接通向扩散模型（19.6 节）。

A 640×480 colour camera delivers 921 600 numbers per frame. Hand such a vector straight to a controller or a reinforcement-learning algorithm and it barely learns: the input is huge, neighbouring pixels are strongly correlated, and the quantities that matter for the task (object position, pose, gripper opening) are only a few. We would like to first squeeze the observation into a short vector \(\mathbf z\) that keeps "what state the scene is in" and drops the rest, and then plan, learn policies and detect anomalies on \(\mathbf z\).

The PCA of Lesson 18.1 is already a compression: find the \(k\)-dimensional linear subspace onto which the data can be projected with the least lost variance. But PCA only knows flat subspaces. Slide a disc across a table and the images form a "surface" with only 2 degrees of freedom, yet that surface is badly curved in pixel space: move the disc by one diameter and the old and new images hardly overlap, so they are nearly orthogonal. Fitting such a surface with flat pieces takes many dimensions.

An autoencoder (AE) is the nonlinear generalization of PCA: an encoder network \(f_\phi\) maps \(\mathbf x\) to a short code \(\mathbf z\), a decoder network \(g_\theta\) maps \(\mathbf z\) back to \(\hat{\mathbf x}\), and both are trained together by the backpropagation of Lesson 16.3 so that \(\hat{\mathbf x}\) comes as close to \(\mathbf x\) as possible. The label is the input itself, so it is self-supervised: video the robot records anyway is training data. It is also the starting point of several later methods: the VAE (Lesson 19.1) gives the code a probabilistic meaning so that it can generate; contrastive learning (Lesson 18.3) and masked modeling / JEPA (Lesson 18.4) replace the "rebuild every pixel" objective; and the denoising autoencoder leads straight to diffusion models (Lesson 19.6).

自编码器的结构像一个沙漏：宽的输入，窄的中间，宽的输出。

An autoencoder is shaped like an hourglass: a wide input, a narrow middle, a wide output.

中间那层叫瓶颈（bottleneck），它的输出 \(\mathbf z\) 叫编码或隐变量（latent code）。如果 \(k\ge D\)，网络只要学会“原样照抄”就能把误差降到零，什么也没学到；正是因为 \(k\) 很小，网络被迫找出数据里最省字数的描述方式。如果数据点其实都落在一条曲线上，一个数（沿曲线走了多远）就够了。

下面是最小的例子：机器人示教时夹爪在桌面上走过的一条弯曲路径，160 个二维位置（单位 m，带 3 cm 噪声）。编码器是 2→24→1 的小网络（tanh），解码器是 1→24→2。PCA 只能用一条直线来表示一维结构；自编码器的解码器 \(\hl{dec}{g_\theta}\) 把整条一维的“编码轴”弯成一条曲线，训练就是让这条曲线去贴合数据。

The middle layer is the bottleneck, and its output \(\mathbf z\) is the code or latent. If \(k\ge D\) the network could drive the error to zero by just copying its input, having learned nothing; it is precisely because \(k\) is small that the network is forced to find the most economical description of the data. If all data points actually lie on one curve, a single number (how far along the curve) is enough.

Here is the smallest example: a curved path traced by a robot's gripper on a table during a demonstration, 160 planar positions (in metres, with 3 cm of noise). The encoder is a small 2→24→1 network (tanh), the decoder 1→24→2. PCA can only represent one-dimensional structure by a straight line; the autoencoder's decoder \(\hl{dec}{g_\theta}\) bends the whole one-dimensional "code axis" into a curve, and training makes that curve hug the data.

在弧线上，默认种子大约 300 步时误差约为 PCA 的 11%，1000 步时约 5%，最后停在 3% 到 4% 左右，已经接近噪声下限（约 PCA 的 2%）：剩下的误差基本是垂直于路径的噪声，任何一维描述都去不掉。S 形曲线更有意思：前一千步左右误差一直停在 PCA 的水平（解码器还是一条直线），然后突然下降到约 8%。这种“平台期再突降”在训练自编码器时很常见，不要过早判断它学不会。

On the arc, with the default seed, the error is about 11% of PCA's after 300 steps, about 5% after 1000, and settles around 3% to 4%, close to the noise floor (about 2% of PCA's): what remains is essentially noise perpendicular to the path, which no one-dimensional description can remove. The S-curve is more interesting: for roughly the first thousand steps the error stays at PCA's level (the decoder is still a straight line), then it suddenly drops to about 8%. This "plateau, then a sudden drop" is common when training autoencoders; do not conclude too early that it cannot learn.

现在换成真正的高维输入。俯视相机看着桌上的一个圆盘（半径 1.8 像素，边缘有 1 像素的过渡），图像只有 12×12 = 144 个像素。圆盘的位置 \((p_x,p_y)\) 是唯一在变的量，所以这些图像构成像素空间 \(\R^{144}\) 里的一张二维曲面。训练集是 300 张随机位置的图，另有 60 张测试图只用来评估。

自编码器是 144→32→\(K\)→32→144（tanh 隐层，sigmoid 输出把像素限制在 0 到 1），用 Adam、每批 16 张训练。对照组是同一训练集上的 PCA：保留前 \(k\) 个主成分再重建。两者的误差都是测试图上每个像素的均方误差。

Now switch to genuinely high-dimensional input. A top-down camera looks at a disc on a table (radius 1.8 pixels with a 1-pixel soft edge), in images of only 12×12 = 144 pixels. The disc position \((p_x,p_y)\) is the only thing that changes, so the images form a two-dimensional surface inside pixel space \(\R^{144}\). The training set is 300 images at random positions; 60 more test images are used only for evaluation.

The autoencoder is 144→32→\(K\)→32→144 (tanh hidden layers, a sigmoid output that keeps pixels between 0 and 1), trained by Adam on batches of 16. The baseline is PCA on the same training set: keep the first \(k\) principal components and reconstruct. Both errors are the per-pixel mean squared error on the test images.

按默认种子，3000 步后 \(K=2\) 的测试误差约 0.0073，而 PCA 要用约 13 个主成分（0.0068）才能做到，PCA-2 是 0.0363，只解释了训练集 31% 的方差。原因正是第 1 节说的：圆盘在不同位置的图像彼此几乎不重叠，像素空间里的这张曲面“卷曲”得厉害，任何平面都贴不住它。

还有一个反直觉的现象：再试 \(K=1\)、\(4\)、\(8\)。\(K=1\) 当然不够（4000 步后测试误差仍约 0.019），但 \(K=4\) 在 3000 步时只有约 0.0008，比 \(K=2\) 低一个数量级，虽然数据只有 2 个自由度。理论上 2 个数足够，可是要把一张二维曲面无折叠地摊平到二维编码上，优化很难找到；多给几维，网络可以用“弯着放”的编码绕开这个困难。FIG 4 会看到 \(K=2\) 的编码里确实有折痕。

With the default seed, after 3000 steps \(K=2\) has a test error of about 0.0073, which PCA needs about 13 components to match (0.0068); PCA-2 sits at 0.0363 and explains only 31% of the training variance. The reason is the one from Section 1: images of the disc at different places barely overlap, so this surface in pixel space is tightly "curled up" and no flat subspace can hug it.

There is also a counter-intuitive effect: try \(K=1\), \(4\) and \(8\). \(K=1\) is of course too few (test error still about 0.019 after 4000 steps), but \(K=4\) reaches about 0.0008 at 3000 steps, an order of magnitude below \(K=2\), even though the data have only 2 degrees of freedom. In theory 2 numbers suffice, but laying a two-dimensional surface flat onto a two-dimensional code without folds is hard for the optimizer to find; with a few spare dimensions the network can use a code that lies "bent" and avoid the difficulty. FIG 4 shows that the \(K=2\) code does contain creases.

去掉所有非线性，自编码器还剩什么？编码器变成一个矩阵 \(E\)（加偏置），解码器变成矩阵 \(W\)（加偏置），重建 \(\hat{\mathbf x}=W E\mathbf x+\mathbf c\) 必然落在 \(W\) 的列空间（平移到某一点）里。第 5 节会证明：只要第 \(k\) 和第 \(k+1\) 个特征值不相等，平方误差的最优解里 \(W\) 的列空间就是前 \(k\) 个主方向张成的子空间，误差恰好等于 PCA-\(k\) 的误差。下面的例子是两个手指关节的角度（单位 rad）：它们倾向于一起弯曲，所以数据沿一条斜线分布，就像 18.1 节的“协同”（synergy）。线性自编码器 2→1→2 用 Adam 从随机初值开始训练。

Strip out every nonlinearity and what is left of an autoencoder? The encoder becomes a matrix \(E\) (plus a bias), the decoder a matrix \(W\) (plus a bias), and the reconstruction \(\hat{\mathbf x}=W E\mathbf x+\mathbf c\) always lies in the column space of \(W\), shifted to some point. Section 5 proves that as long as the \(k\)-th and \((k+1)\)-th eigenvalues differ, at the optimum of the squared error the column space of \(W\) is the subspace spanned by the first \(k\) principal directions, and the error equals PCA-\(k\)'s exactly. The example below is two finger-joint angles (in rad): they tend to flex together, so the data spread along a slanted line, like the "synergies" of Lesson 18.1. A linear 2→1→2 autoencoder is trained by Adam from a random start.

既然解码器能把编码变回图像，能不能随便挑一个编码，“生成”一张没见过的图？下图把 FIG 2 训练出的模型的隐空间画出来（两者共用同一个模型，在哪个图里训练都行）。\(K=2\) 时每张训练图的编码是平面上的一个点；\(K>2\) 时画的是编码的前两维，其余维固定在训练编码的平均值。拖动琥珀色手柄选择一个编码，右侧显示它解码出的图像，以及这张图离“某个真实圆盘图像”有多远：在所有圆盘位置里找最接近的那张，算每像素的均方误差，小于 0.01 判为“像一个圆盘”。

Since the decoder turns codes back into images, can we pick any code and "generate" an image never seen before? The figure below draws the latent space of the model trained in FIG 2 (both figures share one model, so you can train it in either). For \(K=2\) each training image's code is a point in the plane; for \(K>2\) the plot shows the first two code dimensions with the others held at the mean of the training codes. Drag the amber handle to choose a code; the side panel shows the image it decodes to and how far that image is from "some real disc image": search all disc positions for the closest one and take the per-pixel mean squared error; below 0.01 counts as "looks like a disc".

按默认设置训练 3000 步后，我们在 node 里用同样的代码各采了 200 个编码：\(K=2\) 时约 86% 解码成圆盘，\(K=8\) 时只有约 37%；而测试图的重建在两种情况下都 100% 合格。\(K=8\) 的重建误差更低，采样反而更差：训练编码只占据 8 维空间里一张薄薄的二维“片”，按每一维独立的高斯去采，几乎总是落在片外，那里的解码器从没被训练过。即使在 \(K=2\) 时，解码器地图里也能看到远离数据的大片红区，以及数据区内部细长的红色折痕。

After 3000 steps at the default settings we sampled 200 codes each in node with the same code: for \(K=2\) about 86% decode to a disc, for \(K=8\) only about 37%, whereas reconstructions of the test images are 100% valid in both cases. \(K=8\) reconstructs better and samples worse: the training codes occupy only a thin two-dimensional "sheet" inside the 8-dimensional space, and a per-dimension Gaussian almost always lands off the sheet, where the decoder was never trained. Even for \(K=2\) the decoder map shows large red areas away from the data and thin red creases inside the data region.

真实相机有噪声。去噪自编码器（denoising AE，DAE）的改动只有一处：输入换成被破坏的版本 \(\hl{noise}{\tilde{\mathbf x}}=\mathbf x+\sigma\boldsymbol\varepsilon\)，目标仍是干净的 \(\mathbf x\)。网络没法再“照抄输入”，只能学会把偏离数据流形的点推回流形上。下面两个网络结构完全相同（144→32→144，没有窄瓶颈），同时训练：一个用干净输入（普通 AE），一个用加噪输入（DAE），都在带同样噪声的测试图上评估。

Real cameras are noisy. The denoising autoencoder (DAE) changes one thing: the input is replaced by a corrupted version \(\hl{noise}{\tilde{\mathbf x}}=\mathbf x+\sigma\boldsymbol\varepsilon\) while the target stays the clean \(\mathbf x\). The network can no longer "copy the input"; it has to learn to push points that stray from the data manifold back onto it. The two networks below have identical architectures (144→32→144, no narrow bottleneck) and train side by side: one on clean inputs (a plain AE), one on noisy inputs (the DAE); both are evaluated on test images carrying the same noise.

默认 σ = 0.3 时，什么都不做的误差是 0.091；1000 步后普通 AE 是 0.014，DAE 是 0.0043，约为前者的三分之一。普通 AE 也去掉了不少噪声，因为 32 维的隐层本身就是一个约束；但它从没见过噪声，噪声中“看起来像圆盘边缘”的部分被原样保留。代价是 DAE 在干净输入上略差（约 0.0012 对普通 AE 的 0.0003）：它学的是“输入多半带噪，要往数据上收”，对干净图也会收一点。σ 越大，两者差距越大：σ = 0.5 时 2000 步后约 0.040 对 0.010。

At the default σ = 0.3, doing nothing costs 0.091; after 1000 steps the plain AE is at 0.014 and the DAE at 0.0043, about a third of it. The plain AE removes plenty of noise too, because a 32-unit hidden layer is a constraint in itself; but it has never seen noise, and the parts of the noise that "look like a disc edge" pass straight through. The price is that the DAE is slightly worse on clean inputs (about 0.0012 against the plain AE's 0.0003): what it learned is "the input is probably noisy, pull it toward the data", and it pulls a clean image a little too. The larger σ, the larger the gap: at σ = 0.5, after 2000 steps, about 0.040 against 0.010.

给定数据 \(\mathbf x_1,\dots,\mathbf x_N\in\R^D\)，编码器 \(f_\phi:\R^D\to\R^k\)，解码器 \(g_\theta:\R^k\to\R^D\)，训练最小化重建损失

Given data \(\mathbf x_1,\dots,\mathbf x_N\in\R^D\), an encoder \(f_\phi:\R^D\to\R^k\) and a decoder \(g_\theta:\R^k\to\R^D\), training minimizes the reconstruction loss

本页所有误差都是这个“每个分量的均方误差”。选平方误差等价于假设 \(\mathbf x=\hat{\mathbf x}+\)高斯噪声，再做最大似然（4.6 节）；像素在 0 到 1 之间、接近二值的图像，也常用逐像素的二元交叉熵（伯努利似然）。梯度就是两段网络串起来的反向传播（16.3 节）：先把 \(\nabla_{\hat{\mathbf x}}\ell\) 反传过解码器，得到参数梯度和对编码的梯度 \(\partial\ell/\partial\mathbf z\)，再把 \(\partial\ell/\partial\mathbf z\) 反传过编码器：

Every error on this page is this "mean squared error per component". Choosing the squared error amounts to assuming \(\mathbf x=\hat{\mathbf x}+\) Gaussian noise and doing maximum likelihood (Lesson 4.6); for images with pixels in 0 to 1 that are nearly binary, a per-pixel binary cross-entropy (a Bernoulli likelihood) is also common. The gradient is backpropagation through the two networks chained together (Lesson 16.3): first push \(\nabla_{\hat{\mathbf x}}\ell\) back through the decoder, which gives its parameter gradients and the gradient with respect to the code, \(\partial\ell/\partial\mathbf z\); then push \(\partial\ell/\partial\mathbf z\) back through the encoder:

每一项都是 16.3 节的向量–雅可比积，代价约为一次前向传播的几倍。页面里的代码正是这样写的：解码器的 backward 返回 \(\partial\ell/\partial\mathbf z\)，再交给编码器的 backward。

Each term is a vector–Jacobian product of Lesson 16.3 and costs a small multiple of a forward pass. That is literally how the page's code is written: the decoder's backward returns \(\partial\ell/\partial\mathbf z\), which is handed to the encoder's backward.

设数据已中心化（偏置的最优值恰好做这件事），排成 \(D\times N\) 的矩阵 \(X\)，其 SVD 为 \(X=U\Sigma V^\top\)（1.12 节），\(U\) 的前 \(k\) 列 \(\hl{pc}{U_k}\) 就是前 \(k\) 个主方向。线性自编码器 \(\mathbf z=E\mathbf x\)，\(\hat{\mathbf x}=\hl{W}{W}\mathbf z\)，其中 \(E\in\R^{k\times D}\)，\(W\in\R^{D\times k}\)，总误差是

Let the data be centred (the optimal biases do exactly that) and stacked into a \(D\times N\) matrix \(X\) with SVD \(X=U\Sigma V^\top\) (Lesson 1.12); the first \(k\) columns of \(U\), \(\hl{pc}{U_k}\), are the first \(k\) principal directions. A linear autoencoder has \(\mathbf z=E\mathbf x\) and \(\hat{\mathbf x}=\hl{W}{W}\mathbf z\) with \(E\in\R^{k\times D}\) and \(W\in\R^{D\times k}\), and its total error is

不等号成立是因为 \(WEX\) 的秩不超过 \(k\)，而右边是 Eckart–Young 定理（1.12 节）给出的最佳秩 \(k\) 近似误差；右边的等价关系在 \(\sigma_k>\sigma_{k+1}\) 时成立，因为这时最佳秩 \(k\) 近似唯一，就是 \(U_kU_k^\top X\)。所以全局最优处 \(W\) 的列空间就是 \(\operatorname{span}(U_k)\)，误差等于 PCA-\(k\) 的误差。但具体的 \(W\) 不唯一：对任意可逆的 \(k\times k\) 矩阵 \(A\)，

The inequality holds because \(WEX\) has rank at most \(k\), and the right side is the best rank-\(k\) approximation error of the Eckart–Young theorem (Lesson 1.12); the equivalence on the right holds when \(\sigma_k>\sigma_{k+1}\), because then the best rank-\(k\) approximation is unique, namely \(U_kU_k^\top X\). So at the global optimum the column space of \(W\) is \(\operatorname{span}(U_k)\) and the error equals PCA-\(k\)'s. But the particular \(W\) is not unique: for any invertible \(k\times k\) matrix \(A\),

给出同一个重建。FIG 3 的“编码 ×2”就是 \(A=\tfrac12\)（\(k=1\)），所以编码的 \(k\) 个坐标一般是主成分的某种混合，而不是主成分本身。要找回主方向，可以在学到的 \(k\) 维编码上再做一次 PCA（只有 \(k\times k\)，很便宜）。Baldi 与 Hornik（1989）还证明了更强的结论：这个损失没有坏的局部极小，除全局最优外的临界点都是鞍点，这就是 FIG 3 从任何初值都能收敛的原因。一旦有非线性，这个保证就没有了（FIG 1 的平台期、FIG 2 的折痕）。

gives the same reconstruction. FIG 3's "code ×2" is \(A=\tfrac12\) (with \(k=1\)), so the \(k\) coordinates of the code are in general some mixture of principal components rather than the components themselves. To recover the principal directions, run PCA once more on the learned \(k\)-dimensional codes (only \(k\times k\), cheap). Baldi and Hornik (1989) proved even more: this loss has no bad local minima, and every critical point other than the global optimum is a saddle, which is why FIG 3 converges from any start. With nonlinearities this guarantee is gone (FIG 1's plateaus, FIG 2's creases).

DAE 最小化 \(\mathbb E_{\mathbf x,\boldsymbol\varepsilon}\|\mathbf x-r(\hl{noise}{\tilde{\mathbf x}})\|^2\)，\(r=g_\theta\circ f_\phi\)，\(\tilde{\mathbf x}=\mathbf x+\sigma\boldsymbol\varepsilon\)，\(\boldsymbol\varepsilon\sim\mathcal N(0,I)\)。对每个 \(\tilde{\mathbf x}\) 单独看，平方误差的最优解是条件均值（4.6 节的 MMSE 估计）；对高斯破坏，Tweedie 公式把它写成：

The DAE minimizes \(\mathbb E_{\mathbf x,\boldsymbol\varepsilon}\|\mathbf x-r(\hl{noise}{\tilde{\mathbf x}})\|^2\) with \(r=g_\theta\circ f_\phi\), \(\tilde{\mathbf x}=\mathbf x+\sigma\boldsymbol\varepsilon\) and \(\boldsymbol\varepsilon\sim\mathcal N(0,I)\). Taken one \(\tilde{\mathbf x}\) at a time, the minimizer of a squared error is the conditional mean (the MMSE estimate); for Gaussian corruption Tweedie's formula writes it as

其中 \(p\) 是数据的密度，\(p_\sigma\) 是它与噪声卷积后（加噪数据）的密度。所以足够大的 DAE 输出减去输入，是 \(\sigma^2\) 乘以加噪数据密度的 score：一个指向数据更密处的向量场。Vincent（2011）把这一点和 score matching 联系起来，Alain 与 Bengio（2014）分析了 \(\sigma\to0\) 的极限。这正是 19.6 节扩散模型训练的东西，只是那里同时在许多个 \(\sigma\) 上训练。

Here \(p\) is the data density and \(p_\sigma\) its convolution with the noise (the density of the noised data). So for a large enough DAE, output minus input is \(\sigma^2\) times the score of the noised data density: a vector field pointing toward denser data. Vincent (2011) connected this to score matching, and Alain and Bengio (2014) analysed the \(\sigma\to0\) limit. It is exactly what the diffusion models of Lesson 19.6 train, only there at many values of \(\sigma\) at once.

除了把编码变窄，还可以给编码加别的约束，这时隐层甚至可以比输入更宽（过完备）。稀疏自编码器惩罚隐层激活 \(\mathbf h=f_\phi(\mathbf x)\)，让每个输入只激活少数单元：

Instead of narrowing the code one can constrain it in other ways, and then the hidden layer may even be wider than the input (overcomplete). A sparse autoencoder penalizes the hidden activations \(\mathbf h=f_\phi(\mathbf x)\) so that each input switches on only a few units:

第一种是 L1 惩罚；第二种形式 \(\mathcal L''\)（sigmoid 隐层，把每个单元的平均激活 \(\hat\rho_j\) 拉向一个小目标 \(\rho\)，如 0.05）来自 Ng 的讲义。在自然图像小块上训练，稀疏编码的单元往往变成局部的边缘检测器。收缩自编码器（Rifai 等，2011）惩罚编码器的雅可比 \(\lambda\|\partial f_\phi/\partial\mathbf x\|_F^2\)，让编码对偏离流形的微小扰动不敏感，和 DAE 的效果相近。

The first is an L1 penalty; the second form, \(\mathcal L''\) (sigmoid hidden units, pulling each unit's mean activation \(\hat\rho_j\) toward a small target \(\rho\) such as 0.05), comes from Ng's lecture notes. Trained on natural image patches, the units of a sparse code tend to become localized edge detectors. A contractive autoencoder (Rifai et al., 2011) penalizes the encoder's Jacobian, \(\lambda\|\partial f_\phi/\partial\mathbf x\|_F^2\), making the code insensitive to small perturbations off the manifold, with an effect similar to the DAE's.

要生成，需要一个能采样的分布 \(p(\mathbf z)\)，并且解码器在它的大部分质量上都被训练过。普通 AE 的训练只碰到 \(N\) 个点 \(f_\phi(\mathbf x_i)\)，“编码的分布”就是这些点的经验分布，形状由优化随意决定（FIG 4 的空隙、折痕、薄片）。损失里没有任何一项惩罚“空隙处解码成什么”。VAE 的两处改动正对着这两个问题：编码器输出一个分布 \(q_\phi(\mathbf z\mid\mathbf x)\) 而不是一个点，使解码器在每个编码附近的一整团区域上都被训练；再加一项 \(\operatorname{KL}(q_\phi(\mathbf z\mid\mathbf x)\,\|\,\mathcal N(0,I))\)，让所有编码合起来铺满标准正态，于是可以从 \(\mathcal N(0,I)\) 采样（19.1 节推导 ELBO）。

Generating requires a distribution \(p(\mathbf z)\) we can sample from, with the decoder trained on most of its mass. A plain AE only ever touches the \(N\) points \(f_\phi(\mathbf x_i)\); "the code distribution" is their empirical distribution, with a shape the optimizer chose arbitrarily (FIG 4's gaps, creases and sheets). Nothing in the loss penalizes what the gaps decode to. The VAE's two changes address exactly these problems: the encoder outputs a distribution \(q_\phi(\mathbf z\mid\mathbf x)\) instead of a point, so the decoder is trained on a whole blob around each code; and a term \(\operatorname{KL}(q_\phi(\mathbf z\mid\mathbf x)\,\|\,\mathcal N(0,I))\) makes all codes together fill a standard normal, so that one can sample from \(\mathcal N(0,I)\) (Lesson 19.1 derives the ELBO).

自编码器在机器人里最直接的用途，是把相机图像变成控制器能用的低维状态。一个最简单的视觉伺服：给机器人一张“目标图像”，让它移动物体，直到当前图像和目标图像一致。用什么来衡量“一致”？最直接的是像素距离 \(\hl{dpix}{\|\mathbf x-\mathbf x_g\|^2}\)；另一个选择是编码距离 \(\hl{dlat}{\|f_\phi(\mathbf x)-f_\phi(\mathbf x_g)\|^2}\)。

下图是一个导轨上的圆盘（1 个自由度），相机图像 20×6 像素。机器人每次把圆盘左移或右移 0.1 像素，选能让距离变小的方向，两个方向都不能变小就停下（贪心下降）。我们同时训练两个 120→24→1→24→120 的自编码器：一个是普通的；另一个额外要求相邻两帧的编码相近，即损失再加 \(\lambda\,(z(\mathbf x_t)-z(\mathbf x_{t+1}))^2\)，\(\lambda=0.5\)，其中两帧的圆盘位置相差不到 0.3 像素，就像机器人录视频时连续的两帧。这就是 Jonschkowski 与 Brock 所说的“慢变”（slowness）机器人先验。

The most direct use of autoencoders in robotics is turning camera images into a low-dimensional state a controller can use. Take the simplest visual servoing: give the robot a "goal image" and let it move the object until the current image matches the goal image. How do we measure "matches"? The obvious choice is the pixel distance \(\hl{dpix}{\|\mathbf x-\mathbf x_g\|^2}\); another is the code distance \(\hl{dlat}{\|f_\phi(\mathbf x)-f_\phi(\mathbf x_g)\|^2}\).

The figure shows a disc on a rail (1 degree of freedom) in 20×6-pixel camera images. Each tick the robot moves the disc 0.1 pixel left or right, choosing the direction that lowers the distance, and stops when neither does (greedy descent). Two 120→24→1→24→120 autoencoders train side by side: a plain one, and one that also asks that the codes of neighbouring frames be close, adding \(\lambda\,(z(\mathbf x_t)-z(\mathbf x_{t+1}))^2\) with \(\lambda=0.5\) to the loss, where the two frames' disc positions differ by less than 0.3 pixel, like consecutive frames of a video the robot records. This is the "slowness" robotic prior of Jonschkowski and Brock.

像素距离的伺服从起点 3 出发一步也没动；我们从导轨上各处起步试过，它最多挪 0.3 像素，停在一个让圆盘自身图像的平方和最小的亚像素位置（见第 8 节第 5 题）。默认种子（3）训练 2000 步后，普通 AE 的编码在圆盘位置约 7.7 像素处跳变，从起点 3 出发的伺服停在约 7.7 处；慢变 AE 的编码是单调的，130 步走到目标 16。我们在 node 里试了种子 1 到 10：普通 AE 有 7 个种子出现了跳变或折返，慢变 AE 在所有 10 个种子上都没有跳变，对所有相距 6 像素以上的起点–目标组合都能到达。两者的重建误差都低于 0.01（默认种子下约 0.006 和 0.003），说明重建误差根本看不出编码能不能用来控制。按“新种子”可以自己看这种差异。

The pixel-distance servo does not move at all from start 3; trying starts all along the rail, it moves 0.3 pixel at most and stops at a sub-pixel position where the disc's own image has the smallest sum of squares (see question 5 of Section 8). With the default seed (3), after 2000 steps the plain AE's code jumps at a disc position of about 7.7 pixels, and the servo from start 3 stops at about 7.7; the slow AE's code is monotone and the servo reaches the goal at 16 in 130 ticks. We tried seeds 1 to 10 in node: the plain AE's code jumped or turned back for 7 of them, while the slow AE's code never jumped for any of the 10 and reached the goal from every start–goal pair at least 6 pixels apart. Both reconstruct well (errors below 0.01; about 0.006 and 0.003 with the default seed), so reconstruction error simply does not tell whether a code is usable for control. Press "New seed" to see the variation yourself.