MLP 与 CNNMLPs & CNNs

机器人每时每刻都要把一串读数变成一个判断或一个动作：七个关节角 → “这个姿势会不会撞到架子？”；关节的位置误差和速度历史 → “电机实际会出多大力矩？”；一张 640×480 的相机图像 → “杯子在哪里，夹爪该往哪边走？”。16.1 节的线性模型和逻辑回归只能画直线（或者在你手工设计好的特征上画直线）；而上面这些映射弯弯曲曲，人也写不出合适的特征。神经网络的承诺是：特征也一起学。

16.3 节已经解决了“怎么学”：只要计算图由可微的小块组成，反向传播就能以大约三次前向计算的代价（一次前向加约两倍的反向）算出损失对所有参数的梯度，再交给 6.3 节的 Adam。剩下的问题是“学什么样的函数族”，也就是网络架构。架构决定了网络在见到数据之前就“相信”什么，这叫归纳偏置（inductive bias）。本节讲两种最基本、在机器人里也最常见的架构：

• 多层感知机（MLP）：对输入向量的各个分量一视同仁，什么也不假设。它是本体感觉（关节角、速度、力）上的策略、动力学模型、价值函数的标准选择，也是后面 Transformer、扩散模型里反复出现的积木。
• 卷积网络（CNN）：假设输入是一张图，同一个图案出现在哪里都是同一个图案。这个假设让它参数少、数据效率高，十多年来一直是机器人视觉编码器的主力（ResNet-18 至今仍常见于模仿学习策略里）。

At every instant a robot turns a list of readings into a judgement or an action: seven joint angles → "will this pose hit the shelf?"; a joint's position error and velocity history → "what torque will the motor really produce?"; one 640×480 camera image → "where is the cup, and which way should the gripper go?". The linear and logistic models of Lesson 16.1 can only draw straight lines (or straight lines on features you designed by hand); the maps above are curved, and nobody can write down the right features for them. The promise of neural networks is: learn the features too.

Lesson 16.3 already solved "how to learn": as long as the computational graph is made of differentiable pieces, backpropagation gives the gradient of the loss with respect to every parameter for about the cost of three forward passes (one forward pass plus a backward pass about twice as expensive), and the Adam of Lesson 6.3 takes it from there. What remains is "which family of functions to learn", that is, the architecture. The architecture decides what the network "believes" before it has seen any data, its inductive bias. This lesson covers the two most basic architectures, which are also the most common in robotics:

• Multilayer perceptrons (MLPs) treat all components of the input vector alike and assume nothing. They are the standard choice for policies, dynamics models and value functions on proprioception (joint angles, velocities, forces), and the building block that keeps reappearing inside transformers and diffusion models later on.
• Convolutional networks (CNNs) assume the input is an image and that a pattern is the same pattern wherever it appears. That assumption makes them small and data-efficient, and for over a decade they have been the workhorse visual encoders of robotics (a ResNet-18 is still a common sight inside imitation-learning policies).

一个神经元做两件事：先算输入的加权和 \(z=\mathbf w^\top\mathbf x+b\)（和 16.1 节的线性模型一模一样），再把 \(z\) 送进一个非线性的激活函数。今天最常用的是 ReLU：\(\operatorname{ReLU}(z)=\max(0,z)\)，负的砍成 0，正的原样通过。一维输入时，\(\operatorname{ReLU}(x-k)\) 的图像就是一根在 \(x=k\) 处“折了一下”的铰链：左边平躺，右边斜上。

一层有 \(N\) 个这样的神经元，输出层再把它们按权重加起来，结果是一条在 \(N\) 个地方折弯的折线。折点越多，折线就能越贴近任何一条光滑或不光滑的曲线。下面的目标都来自机器人：单个连杆的重力矩 \(\cos q\)、带硬限位的关节（限位前是软弹簧，撞上限位后突然变硬），以及库仑加黏性摩擦（在速度为零处跳变）。折点（下方蓝色圆圈）可以拖动；每次改动后，输出层的权重用最小二乘（1.13 节）直接解出，相当于把隐藏层固定住、只训练最后一层。

A neuron does two things: it forms a weighted sum of its inputs \(z=\mathbf w^\top\mathbf x+b\) (exactly the linear model of Lesson 16.1), then feeds \(z\) through a nonlinear activation function. The most common one today is the ReLU, \(\operatorname{ReLU}(z)=\max(0,z)\): negatives are cut to 0, positives pass unchanged. With one input, the graph of \(\operatorname{ReLU}(x-k)\) is a hinge that "bends" at \(x=k\): flat on the left, sloping up on the right.

A layer of \(N\) such neurons, summed with weights by the output layer, gives a polyline that bends in \(N\) places. The more bends, the more closely the polyline can follow any curve, smooth or not. The targets below all come from robots: the gravity torque \(\cos q\) of a single link, a joint with hard stops (a soft spring until it hits the limit, then suddenly stiff), and Coulomb-plus-viscous friction (which jumps at zero velocity). The kinks (blue circles at the bottom) can be dragged; after every change, the output-layer weights are solved directly by least squares (Lesson 1.13), which amounts to freezing the hidden layer and training only the last one.

7.2 节用采样画出了 C-障碍物：对每个关节角组合 \(\mathbf q=(q_1,q_2)\) 问一次“两根连杆碰到障碍物了吗？”。一次几何检测不贵，但规划器（14.2 节的 RRT、14.3 节的轨迹优化）每秒要问成千上万次，轨迹优化还想要“离碰撞有多近”的梯度。一个常见的做法是用一个小网络学出碰撞概率 \(p(\mathbf q)\)：查询只是一次前向计算，对 \(\mathbf q\) 的梯度由反向传播免费送上。

下图是一个真实在你浏览器里训练的 MLP。数据是 500 个随机关节角，标签由几何检测给出（臂长 1.0 m 和 0.8 m，障碍物是一个圆）；损失是 16.1 节的二元交叉熵，优化器是 Adam（学习率 0.01，每批 32 个样本）。测试集是覆盖整个 \([-\pi,\pi]^2\) 的 36×36 网格。先用默认的 2 层×16 单元网络按“训练”，再试试别的架构。把 C-空间里的琥珀色手柄拖来拖去，左边的手臂会跟着动；把左边的红色障碍物拖到别处，标签会立刻重算，网络接着学。

Lesson 7.2 drew the C-obstacle by sampling: for each joint-angle pair \(\mathbf q=(q_1,q_2)\) it asked "do the two links touch the obstacle?". One geometric check is cheap, but a planner (RRT in Lesson 14.2, trajectory optimization in 14.3) asks thousands of times a second, and trajectory optimization would also like the gradient of "how close to collision". A common trick is to learn the collision probability \(p(\mathbf q)\) with a small network: a query is one forward pass, and the gradient with respect to \(\mathbf q\) comes free from backpropagation.

The figure below is an MLP really training in your browser. The data are 500 random joint-angle pairs labelled by the geometric check (links of 1.0 m and 0.8 m, a circular obstacle); the loss is the binary cross-entropy of Lesson 16.1 and the optimizer Adam (learning rate 0.01, batches of 32). The test set is a 36×36 grid covering all of \([-\pi,\pi]^2\). Press "Train" with the default 2 layers × 16 units first, then try other architectures. Drag the amber handle in C-space and the arm on the left follows; drag the red obstacle on the left somewhere else and the labels are recomputed at once while the network keeps learning.

万能逼近定理只要一个隐藏层，那为什么大家都用很多层？因为同样数量的神经元，串起来比并排放能表示复杂得多的函数。一维里最干净的例子是“帐篷函数” \(g\)：两个 ReLU 就能拼出来，它把 \([0,1]\) 先拉上去再折回来。把 \(g\) 自己和自己复合 \(L\) 次（一层的输出喂给下一层），每复合一次，每一段单调的折线都会被再对折一次，折线段数翻倍：\(L\) 层、每层 2 个单元，一共 \(2L\) 个单元，就有 \(2^L\) 段。而一个只有一层、\(n\) 个单元的 ReLU 网络，在一维上最多只有 \(n+1\) 段，要表示同一个函数至少需要 \(2^L-1\) 个单元（Telgarsky 2016）。

The universal approximation theorem needs only one hidden layer, so why does everyone use many? Because the same number of neurons chained in series can represent far more complex functions than placed side by side. The cleanest example in one dimension is the "tent map" \(g\): two ReLUs build it, and it stretches \([0,1]\) up and folds it back down. Compose \(g\) with itself \(L\) times (each layer's output feeds the next) and every composition folds each monotone piece once more, doubling the number of linear pieces: \(L\) layers of 2 units, \(2L\) units in total, give \(2^L\) pieces. A ReLU network with a single layer of \(n\) units has at most \(n+1\) pieces in one dimension, so it needs at least \(2^L-1\) units to represent the same function (Telgarsky 2016).

把一张 224×224 的彩色图像拉成向量喂给 MLP，输入有 150,528 维，第一层哪怕只有 1000 个单元也要 1.5 亿个权重。更糟的是，MLP 不知道相邻像素是邻居，也不知道左上角的杯子和右下角的杯子是同一种东西，它得在每个位置分别学一遍。卷积层用两个假设修好了这两点：

• 局部性：一个输出只看输入里一小块 \(k\times k\) 的窗口（比如 3×3）。
• 权重共享：所有位置用同一个窗口权重（卷积核）。核在图像上滑动，每到一处算一次加权和，得到一张“特征图”。

于是参数个数和图像大小无关：一个 3×3 核只有 9 个权重加 1 个偏置。权重共享带来一个漂亮的性质，平移等变：输入平移，输出跟着平移同样的量，“先平移再卷积”等于“先卷积再平移”。下图的输入是一张 16×16 的“相机裁剪”：桌面上一个箱子。试几个经典的核：Sobel-x 对竖直的边（左右亮度变化）有反应，Sobel-y 对水平的边（比如桌沿）有反应。

Flatten a 224×224 colour image into a vector for an MLP and the input has 150,528 dimensions; even a first layer of only 1000 units needs 150 million weights. Worse, the MLP does not know that neighbouring pixels are neighbours, nor that a cup in the top-left corner and a cup in the bottom-right are the same thing: it must learn it separately at every position. A convolutional layer fixes both with two assumptions:

• Locality: an output looks only at a small \(k\times k\) window of the input (say 3×3).
• Weight sharing: every position uses the same window weights (the kernel). The kernel slides over the image, computes a weighted sum at each place, and produces a "feature map".

The parameter count therefore does not depend on the image size: a 3×3 kernel has 9 weights plus 1 bias. Weight sharing brings a beautiful property, translation equivariance: shift the input and the output shifts by the same amount; "shift then convolve" equals "convolve then shift". The input below is a 16×16 "camera crop" of a box on a table. Try the classic kernels: Sobel-x reacts to vertical edges (brightness changing left to right), Sobel-y to horizontal ones (such as the table edge).

一层 3×3 卷积只看得见 3×3 的邻域，但卷积层可以叠：第二层的一个输出看的是第一层的 3×3 个输出，而它们每个又看 3×3 个像素，于是第二层“看见”5×5 的输入。一个输出单元最终依赖的那块输入区域叫它的感受野。只叠步长 1 的卷积，感受野每层只长 \(k-1\) 个像素，要看清整只杯子得叠很多层；所以 CNN 会用步长 2 的卷积或池化来降采样：每降一次，后面每层的“一步”在输入上就跨两倍远，感受野按指数增长。另一种办法是空洞卷积（dilated convolution）：核的元素之间隔开 \(d\) 个像素，分辨率不降，感受野同样指数增长（WaveNet 和时间卷积网络这样做；17.2 节则用循环网络和状态空间模型处理很长的序列）。下图是一维的示意，从顶层选一个单元，看它的感受野。

One 3×3 convolution sees only a 3×3 neighbourhood, but convolutions stack: an output of the second layer looks at 3×3 outputs of the first, each of which looks at 3×3 pixels, so the second layer "sees" 5×5 of the input. The input region an output unit ultimately depends on is its receptive field. Stacking only stride-1 convolutions grows the receptive field by just \(k-1\) pixels per layer, and seeing a whole cup would take many layers; so CNNs downsample with stride-2 convolutions or pooling: after each downsampling, one "step" of every later layer spans twice as far in the input, and the receptive field grows exponentially. Another way is the dilated convolution: the kernel taps are spread \(d\) pixels apart, the resolution stays, and the receptive field still grows exponentially (WaveNet and temporal convolutional networks do this; Lesson 17.2 handles long sequences with recurrent and state-space models instead). The figure is a one-dimensional sketch: pick a unit in the top layer and see its receptive field.

一台机器人从相机里截出料盘的 12×12 小图，要判断里面的销钉是立着（竖直）、躺着（水平）还是歪着（对角）。图里有像素噪声，销钉亮度也略有变化。为了把归纳偏置的作用放大，我们故意让训练集里的销钉只出现在左半边，测试集全在右半边。

两个网络看同一批数据、用同一个 Adam 训练：

• CNN：4 个 3×3 卷积核 + ReLU → 每个特征图取全局最大值（全局最大池化，回答“这个图案在图里任何地方出现过吗”）→ 一个 4→3 的线性层。共 55 个参数。
• MLP：144 个像素 → 16 个 ReLU → 3 类。共 2371 个参数。

CNN 的卷积在左边学到的“竖条检测器”在右边一样有效（等变），最大池化再把位置扔掉（不变）；MLP 的每个权重都绑定在某个像素上，右半边的像素在训练时从来没亮过，它对它们一无所知。本课程共用的网络工具库里没有卷积，本页把它内联实现：最大池化的梯度只流回取到最大值的那个位置，卷积核的梯度是该位置的输入窗口乘上传回来的梯度（第 7 节）。

A robot crops a 12×12 patch of a parts tray from its camera and must decide whether the peg in it is standing (vertical), lying (horizontal) or tilted (diagonal). There is pixel noise, and the peg's brightness varies a little. To magnify the effect of inductive bias we deliberately put the training pegs only in the left half of the patch; the test pegs are all in the right half.

Two networks see the same mini-batches and are trained by the same Adam:

• CNN: 4 kernels of 3×3 + ReLU → the global maximum of each feature map (global max pooling, answering "did this pattern appear anywhere?") → one linear layer 4→3. 55 parameters in all.
• MLP: 144 pixels → 16 ReLUs → 3 classes. 2371 parameters in all.

The "vertical-bar detector" the CNN learns on the left works just as well on the right (equivariance), and the max pooling then discards the position (invariance); every weight of the MLP is tied to one pixel, the right-half pixels never lit up during training, and it knows nothing about them. There is no convolution in the shared network library, so this page implements it inline: the max-pool gradient flows back only to the position that attained the maximum, and the kernel gradient is that position's input window times the gradient sent back (Section 7).

FIG 6 的 CNN 只有一层。真实的 CNN 是“卷积 → 非线性 → 降采样”重复很多次：早期的层学到边和角，后面的层把它们组合成部件和物体，最后的特征图分辨率很低、通道很多，每个位置的感受野覆盖大半张图。LeNet-5（LeCun 等 1998）用这个配方认手写数字；AlexNet（Krizhevsky 等 2012）把它放到 GPU 和 ImageNet 上，开启了深度学习的这一轮浪潮；VGG（Simonyan 与 Zisserman 2015）说明堆叠 3×3 小核就够了。

再往深走就撞墙了：He 等人（2016）观察到，在 CIFAR-10 上 56 层的普通 CNN 连训练误差都比 20 层的高，问题不在过拟合，而在于优化。他们的修法极其简单：让每个块学习对输入的修正量，而不是整个映射，\(\mathbf h_{l+1}=\mathbf h_l+F(\mathbf h_l)\)。如果某个块什么也不该做，只需让 \(F\to0\)，恒等映射成了“默认值”；反向传播时梯度沿着这条“捷径”直接流回去，不会被几十个雅可比连乘消磨掉（16.3 节）。这就是 ResNet，它让 152 层的网络可以训练，并赢得了 ILSVRC 2015。残差连接后来成了几乎所有深网络的标配，17.3 节的 Transformer 块里你会再次见到它。

The CNN of FIG 6 has a single layer. Real CNNs repeat "convolve → nonlinearity → downsample" many times: early layers learn edges and corners, later layers combine them into parts and objects, and the final feature maps have low resolution, many channels, and a receptive field covering most of the image at every position. LeNet-5 (LeCun et al. 1998) used this recipe to read handwritten digits; AlexNet (Krizhevsky et al. 2012) moved it onto GPUs and ImageNet and started the current wave of deep learning; VGG (Simonyan & Zisserman 2015) showed that stacks of small 3×3 kernels are enough.

Going deeper hit a wall: He et al. (2016) observed that on CIFAR-10 a plain 56-layer CNN had a higher training error than a 20-layer one, so the problem was not overfitting but optimization. Their fix is strikingly simple: let each block learn a correction to its input instead of the whole map, \(\mathbf h_{l+1}=\mathbf h_l+F(\mathbf h_l)\). If a block should do nothing it only needs \(F\to0\): the identity becomes the default. In backpropagation the gradient flows straight back along this "shortcut" instead of being worn down by a product of dozens of Jacobians (Lesson 16.3). This is the ResNet; it made 152-layer networks trainable and won ILSVRC 2015. Residual connections went on to become standard in nearly every deep network; you will meet them again inside the transformer block of Lesson 17.3.

一个 \(L\) 层的 MLP 是交替的仿射映射和逐元素激活 \(\sigma\)：

An \(L\)-layer MLP alternates affine maps and an elementwise activation \(\sigma\):

参数 \(\theta=\{W^{(l)},\mathbf b^{(l)}\}\)，\(W^{(l)}\in\R^{n_l\times n_{l-1}}\)，总数 \(\sum_l n_l(n_{l-1}+1)\)：FIG 2 默认网络 \([2,16,16,1]\) 有 \(16\cdot3+16\cdot17+1\cdot17=337\) 个。第一层的每一行 \(\mathbf w_i^\top\mathbf x+b_i=0\) 是输入空间里的一个超平面（FIG 2 的青色直线），ReLU 在它一侧为 0、另一侧线性增长；所以一个 ReLU 网络是分片线性函数，每一片是这些超平面切出来的一个区域。最后一层不加激活（回归）或接 sigmoid / softmax（分类，16.1 节）。

为什么必须有非线性：若 \(\sigma\) 是恒等，\(W^{(2)}(W^{(1)}\mathbf x+\mathbf b^{(1)})+\mathbf b^{(2)}=(W^{(2)}W^{(1)})\mathbf x+(W^{(2)}\mathbf b^{(1)}+\mathbf b^{(2)})\)，再多层也只是一个仿射映射。常用激活：ReLU \(\max(0,z)\)（导数是 0 或 1，便宜，但单元可能“死掉”：如果它对所有数据都输出 0，梯度也恒为 0）；tanh 和 sigmoid（光滑，但两端饱和、梯度变小）；GELU、SiLU 这类光滑版 ReLU 常见于 Transformer。FIG 2 里 tanh 和 ReLU 都能学会，只是速度不同。

The parameters are \(\theta=\{W^{(l)},\mathbf b^{(l)}\}\) with \(W^{(l)}\in\R^{n_l\times n_{l-1}}\), \(\sum_l n_l(n_{l-1}+1)\) in total: the default \([2,16,16,1]\) network of FIG 2 has \(16\cdot3+16\cdot17+1\cdot17=337\). Each row of the first layer, \(\mathbf w_i^\top\mathbf x+b_i=0\), is a hyperplane in input space (the cyan lines of FIG 2); the ReLU is 0 on one side and grows linearly on the other, so a ReLU network is a piecewise-linear function, each piece being one region cut out by these hyperplanes. The last layer has no activation (regression) or a sigmoid / softmax (classification, Lesson 16.1).

Why the nonlinearity is essential: if \(\sigma\) were the identity, \(W^{(2)}(W^{(1)}\mathbf x+\mathbf b^{(1)})+\mathbf b^{(2)}=(W^{(2)}W^{(1)})\mathbf x+(W^{(2)}\mathbf b^{(1)}+\mathbf b^{(2)})\), and any number of layers would still be one affine map. Common activations: ReLU \(\max(0,z)\) (derivative 0 or 1, cheap, but a unit can "die": if it outputs 0 on all data its gradient is 0 for good); tanh and sigmoid (smooth, but saturating at both ends, which shrinks gradients); smooth ReLU variants such as GELU and SiLU are common in transformers. In FIG 2 both tanh and ReLU learn the task, at different speeds.

定理（Cybenko 1989 对 sigmoid；Hornik 等 1989；Leshno 等 1993 推广到任何非多项式的激活）：对紧集 \(K\subset\R^d\) 上的任意连续函数 \(f\) 和任意 \(\varepsilon>0\)，存在一个单隐藏层网络 \(\hat f\) 使 \(\max_{\mathbf x\in K}|f(\mathbf x)-\hat f(\mathbf x)|<\varepsilon\)。一维的 ReLU 情形可以直接构造，FIG 1 就是这个构造：

Theorem (Cybenko 1989 for sigmoids; Hornik et al. 1989; extended by Leshno et al. 1993 to any non-polynomial activation): for any continuous \(f\) on a compact set \(K\subset\R^d\) and any \(\varepsilon>0\) there is a one-hidden-layer network \(\hat f\) with \(\max_{\mathbf x\in K}|f(\mathbf x)-\hat f(\mathbf x)|<\varepsilon\). In one dimension with ReLUs it can be built by hand, and FIG 1 is that construction:

第一行：分段线性插值的每个折点都是一个 ReLU，系数 \(c_i\) 是那里斜率的变化量（\(s_i\) 是第 \(i\) 个折点右侧那一段的斜率）。第二行是插值误差的经典上界（\(\tilde f\) 是在 \(N\) 个等距折点上的分段线性插值，\(f\) 二阶可导），间距 \(h\) 减半误差变成四分之一，这就是 FIG 1 光滑目标的 \(N^{-2}\)；最小二乘拟合的均方根误差只会更小。若 \(f\) 有跳变，任何连续的 \(\hat f\) 在跳变附近都要差出跳变高度的一半，最大误差不会趋于 0，均方根误差只按 \(N^{-1/2}\) 下降。

定理没说的有三件事。(1) 多少单元：在 \(d\) 维里，同样的构造需要的单元数随 \(d\) 指数增长（维数灾难），除非函数有可利用的结构（光滑、组合式、低维流形），深度正是利用组合结构的方式。(2) 能不能学到：存在一组好权重，不代表梯度下降能找到它。(3) 能不能泛化：FIG 6 的 MLP 完美拟合了训练集，却在新位置上失败，见 16.2 节。

First line: every kink of the piecewise-linear interpolant is one ReLU, and the coefficient \(c_i\) is the change of slope there (\(s_i\) is the slope of the piece just right of kink \(i\)). The second line is the classic bound on the interpolation error (\(\tilde f\) is the piecewise-linear interpolant on \(N\) evenly spaced kinks, \(f\) twice differentiable): halve the spacing \(h\) and the error drops to a quarter, which is the \(N^{-2}\) of the smooth target in FIG 1; the least-squares fit can only have a smaller RMS error. If \(f\) jumps, every continuous \(\hat f\) is off by half the jump somewhere near it, so the maximum error does not tend to 0 and the RMS error falls only like \(N^{-1/2}\).

Three things the theorem does not say. (1) How many units: in \(d\) dimensions the same construction needs a number of units growing exponentially with \(d\) (the curse of dimensionality), unless the function has structure to exploit (smoothness, compositionality, a low-dimensional manifold); depth is how compositional structure is exploited. (2) Whether it can be learned: that good weights exist does not mean gradient descent finds them. (3) Whether it generalizes: the MLP of FIG 6 fits its training set perfectly and still fails at new positions; see Lesson 16.2.

\(g\) 把 \([0,a]\) 线性拉到 \([0,1]\)，再把 \([a,1]\) 折回 \([1,0]\)。如果 \(g^{\circ(L-1)}\) 有 \(m\) 段、每段都单调地扫过整个 \([0,1]\)，那么再复合一次，每段都恰好在取值 \(a\) 处多出一个折点，段数变成 \(2m\)：归纳得 \(g^{\circ L}\) 有 \(2^L\) 段。另一方面，\(n\) 个单元的单隐藏层 ReLU 网络是 \(n\) 个铰链之和，一维上至多 \(n\) 个折点、\(n+1\) 段，所以表示 \(g^{\circ L}\) 至少需要 \(2^L-1\) 个单元。更一般地，宽 \(w\)、深 \(L\) 的 ReLU 网络的线性区域数可以随 \(L\) 指数增长，而浅网络只随宽度多项式增长。

\(g\) stretches \([0,a]\) linearly onto \([0,1]\) and folds \([a,1]\) back onto \([1,0]\). If \(g^{\circ(L-1)}\) has \(m\) pieces, each sweeping monotonically across all of \([0,1]\), then one more composition adds exactly one kink inside each piece (where its value equals \(a\)), giving \(2m\) pieces; by induction \(g^{\circ L}\) has \(2^L\). On the other hand, a one-hidden-layer ReLU network with \(n\) units is a sum of \(n\) hinges: at most \(n\) kinks and \(n+1\) pieces in one dimension, so it needs at least \(2^L-1\) units to represent \(g^{\circ L}\). More generally, the number of linear regions of a ReLU network of width \(w\) and depth \(L\) can grow exponentially in \(L\), whereas a shallow network's grows only polynomially with width.

输入有 \(C_{\rm i}\) 个通道（RGB 是 3），输出有 \(C_{\rm o}\) 个通道，核大小 \(k\)，步长 \(s\)，零填充 \(p\)：

With \(C_{\rm i}\) input channels (3 for RGB), \(C_{\rm o}\) output channels, kernel size \(k\), stride \(s\) and zero padding \(p\):

注意三点。(1) 深度学习里的“卷积”其实是互相关：核没有翻转（4.7 节贝叶斯滤波里的卷积、信号处理里的卷积都会翻转）。核是学出来的，翻不翻都一样，所以没人在意。(2) 参数个数与图像尺寸 \(n\) 无关：64 进 64 出的 3×3 卷积层有 \(64(64\cdot9+1)=36{,}928\) 个参数，无论图是 64×64 还是 640×480；把同样大小的 64×64×64 特征图用全连接层映射到同样大小，需要 \((64^3)^2\approx6.9\times10^{10}\) 个权重。(3) 把卷积写成矩阵乘法，它是一个巨大但稀疏、并且沿对角线重复的矩阵（Toeplitz 结构）：卷积层就是一个“被约束得很厉害”的全连接层，约束本身就是先验。

平移等变：记 \(T_{\mathbf d}\) 为把图像平移 \(\mathbf d\) 个像素，则步长 1 时（忽略边界）\(K\star(T_{\mathbf d}X)=T_{\mathbf d}(K\star X)\)；把上式里的 \(j\) 换成 \(j-d\) 立即得到。步长 \(s\) 时，只有 \(\mathbf d\) 是 \(s\) 的整数倍才有 \(K\star_s(T_{\mathbf d}X)=T_{\mathbf d/s}(K\star_s X)\)，其他平移在输出上没有对应的整数平移，FIG 4 的误差就来自这里（这种“混叠”效应可参见 Zhang 2019）。边界也会破坏等变：零填充的那一圈对靠边的物体和对中间的物体不一样，FIG 6 因此让销钉离边至少 1 个像素。

池化：\(2\times2\) 最大池化取每个 2×2 块的最大值，相当于步长 2 的降采样，对块内的小位移不敏感。全局池化对整张特征图取最大（或平均），\(z_c=\max_{i,j}\hl{pool}{Y[c,i,j]}\)，得到对平移不变的特征：等变的卷积 + 不变的池化 = 不变的分类器，FIG 6 的 CNN 正是这样。它的梯度：\(\partial z_c/\partial Y[c,i,j]\) 只在取到最大值的 \((i^*,j^*)\) 为 1，所以 \(\partial L/\partial K[c,u,v]=\frac{\partial L}{\partial z_c}\,\mathbb 1[Y>0]\,X[i^*+u,j^*+v]\)。

Three remarks. (1) The "convolution" of deep learning is really a cross-correlation: the kernel is not flipped (the convolution of the Bayes filter in Lesson 4.7, and of signal processing, does flip). Since the kernel is learned, flipped or not makes no difference and nobody minds. (2) The parameter count does not depend on the image size \(n\): a 3×3 layer with 64 channels in and out has \(64(64\cdot9+1)=36{,}928\) parameters whether the image is 64×64 or 640×480; mapping a 64×64×64 feature map to one of the same size with a fully connected layer would take \((64^3)^2\approx6.9\times10^{10}\) weights. (3) Written as a matrix product, a convolution is a huge but sparse matrix that repeats along its diagonals (Toeplitz structure): a convolutional layer is a heavily constrained fully connected layer, and the constraint is the prior.

Translation equivariance: let \(T_{\mathbf d}\) shift an image by \(\mathbf d\) pixels; with stride 1 (ignoring borders) \(K\star(T_{\mathbf d}X)=T_{\mathbf d}(K\star X)\), which follows at once by substituting \(j-d\) for \(j\) above. With stride \(s\), only shifts \(\mathbf d\) that are multiples of \(s\) give \(K\star_s(T_{\mathbf d}X)=T_{\mathbf d/s}(K\star_s X)\); other shifts have no integer counterpart in the output, and that is the error in FIG 4 (on this aliasing see Zhang 2019). Borders break equivariance too: the ring of zero padding treats an object at the edge differently from one in the middle, which is why FIG 6 keeps its pegs at least 1 pixel from the border.

Pooling: \(2\times2\) max pooling keeps the maximum of each 2×2 block, a stride-2 downsampling that ignores small shifts inside the block. Global pooling takes the maximum (or mean) over the whole feature map, \(z_c=\max_{i,j}\hl{pool}{Y[c,i,j]}\), giving a feature that is invariant to translation: equivariant convolution + invariant pooling = an invariant classifier, which is exactly the CNN of FIG 6. Its gradient: \(\partial z_c/\partial Y[c,i,j]\) is 1 only at the maximizing \((i^*,j^*)\), so \(\partial L/\partial K[c,u,v]=\frac{\partial L}{\partial z_c}\,\mathbb 1[Y>0]\,X[i^*+u,j^*+v]\).

\(j_l\) 是第 \(l\) 层相邻两个单元在输入上相隔的像素数（“跳距”），\(d_l\) 是空洞率。步长 1、\(k=3\)：\(r_L=1+2L\)，线性增长；每层步长 2：\(r_L=1+2(2^L-1)\)，4 层是 31；空洞率 \(1,2,4,8\)：同样是 31，但分辨率不降。这也解释了 VGG 的选择：三层 3×3 的感受野是 7×7，和一层 7×7 一样，却只有 \(27C^2\) 而不是 \(49C^2\) 个权重，中间还多了两次非线性。需要注意，这是理论感受野，有效感受野（真正对输出有明显影响的区域）通常小得多，中心像素的影响最大（Luo 等 2016）。

\(j_l\) is the distance in input pixels between neighbouring units of layer \(l\) (the "jump"), and \(d_l\) is the dilation. Stride 1 with \(k=3\): \(r_L=1+2L\), linear growth; stride 2 at every layer: \(r_L=1+2(2^L-1)\), 31 after 4 layers; dilations \(1,2,4,8\): also 31, without losing resolution. This also explains the choice in VGG: three 3×3 layers have a 7×7 receptive field like one 7×7 layer, but \(27C^2\) instead of \(49C^2\) weights, plus two extra nonlinearities in between. Note that this is the theoretical receptive field; the effective receptive field (the region that noticeably affects the output) is usually much smaller and dominated by the centre (Luo et al. 2016).

普通网络的雅可比是 \(\prod_m \partial F/\partial\mathbf h_m\)，几十个小于 1 的因子相乘就消失了；残差网络展开后含有恒等项 \(I\)，总有一条梯度通路不经过任何权重。ResNet 的块 \(F\) 通常是“卷积–批归一化–ReLU–卷积–批归一化”，更深的版本用 1×1–3×3–1×1 的“瓶颈”块减少计算。批归一化（BatchNorm）用一个小批量的均值和方差归一化每个通道，推理时改用训练时的滑动平均，这在机器人上是个常见坑：批量很小或训练与部署的数据分布不同时，统计量不准，所以不少机器人策略改用不依赖批量的 GroupNorm 或 LayerNorm（归一化层见 16.3 节）。

A plain network's Jacobian is \(\prod_m \partial F/\partial\mathbf h_m\), and dozens of factors below 1 make it vanish; the residual product, expanded, contains the identity term \(I\), so there is always a gradient path through no weights at all. A ResNet block \(F\) is usually "conv–batch norm–ReLU–conv–batch norm", and deeper versions use 1×1–3×3–1×1 "bottleneck" blocks to save compute. Batch normalization normalizes each channel with the mean and variance of a mini-batch and switches to running averages from training at inference; this is a common trap in robotics: with tiny batches, or when deployment data differ from training data, the statistics are off, so many robot policies use batch-independent GroupNorm or LayerNorm instead (normalization layers: Lesson 16.3).

当 MLP 的输入是低维坐标（关节角、时间、空间位置）时，输入怎么编码会决定它能学多细的东西。FIG 2 的 \((\cos q,\sin q)\) 解决了角度的周期性；更一般的做法是把坐标送进一组不同频率的正弦：

When the input of an MLP is a low-dimensional coordinate (joint angle, time, spatial position), how it is encoded decides how fine a detail the network can learn. The \((\cos q,\sin q)\) of FIG 2 took care of the periodicity of angles; the general recipe feeds the coordinate through sines of several frequencies:

\(B\) 的元素可以随机取自 \(\mathcal N(0,\sigma^2)\)，也可以沿每个坐标轴取 \(2^k\)。直接吃坐标的 MLP 有“谱偏置”：先学低频，高频学得极慢。Tancik 等（2020）证明随机傅里叶特征能缓解它，\(\sigma\) 控制网络能表达的最高频率；NeRF 用的是按 \(2^k\) 递增的频率。12.4 节的 FIG 3 已经现场演示了这个效果，本节不再重复；17.3 节 Transformer 的正弦位置编码也是同一个思想。

The entries of \(B\) can be drawn from \(\mathcal N(0,\sigma^2)\) or set to \(2^k\) along each axis. An MLP fed raw coordinates has a "spectral bias": it learns low frequencies first and high frequencies extremely slowly. Tancik et al. (2020) showed that random Fourier features fix this, with \(\sigma\) controlling the highest frequency the network can express; NeRF uses frequencies growing as \(2^k\). FIG 3 of Lesson 12.4 already demonstrated the effect live, so it is not repeated here; the sinusoidal positional encoding of the transformer in Lesson 17.3 is the same idea.

CNN 输出的是特征图，控制器要的却是数字：物体在图像的哪个位置、夹爪该往哪边动。最常见的接口有两种：全局平均池化（得到一个向量，丢掉了位置）和空间 softmax（spatial soft-argmax，Levine 等 2016）：对每个通道的特征图做 softmax，得到一个“注意力”分布 \(p_{ij}\)，再求期望坐标 \(\hat{\mathbf u}=\sum p_{ij}(j,i)\)。每个通道于是变成一个可微的二维“关键点”，正好适合喂给控制策略。温度 \(T\) 决定分布有多尖：太高，所有像素差不多一样重，关键点被拉向图像中心；太低，几乎就是 argmax，只能落在整数像素上，梯度也几乎没有用处了。

A CNN outputs feature maps, but a controller wants numbers: where the object is in the image, which way the gripper should move. Two interfaces are most common: global average pooling (a vector, with the position thrown away) and the spatial softmax (spatial soft-argmax, Levine et al. 2016): take a softmax over each channel's feature map to get an "attention" distribution \(p_{ij}\), then the expected coordinate \(\hat{\mathbf u}=\sum p_{ij}(j,i)\). Each channel becomes a differentiable 2-D "keypoint", ideal input for a control policy. The temperature \(T\) sets how peaked the distribution is: too high and all pixels weigh about the same, pulling the keypoint towards the image centre; too low and it is almost an argmax, stuck on whole pixels and no longer usefully differentiable.

再打开干扰物：温度高时关键点落在两个物体中间，那里什么也没有（\(T=0.3\) 时约在 (7.3, 7.9)）；温度低时它跳到稍高的那个峰上，这里恰好是干扰物。这和 22.1 节将要讲的“均方误差把多峰的示范平均掉”是同一类问题：对一个多峰分布取期望，结果可能落在任何一个峰之外。

Now switch the distractor on: at high temperature the keypoint lands between the two objects, where there is nothing (about (7.3, 7.9) at \(T=0.3\)); at low temperature it jumps to whichever bump is a hair higher, here the distractor. This is the same kind of problem as the "MSE averages multimodal demonstrations" of Lesson 22.1: the expectation of a multimodal distribution can land outside every mode.