Part I · 第 6 章优化 · 6.3Part I · Ch. 6 Optimization · 6.3

梯度下降家族The Gradient Descent Family

蒙着眼站在山坡上，只能用脚感受脚下的坡度，怎样走到谷底？朝最陡的下坡方向迈一步，再感受，再迈一步。这就是梯度下降。它的几个"亲戚"（动量、Nesterov、RMSProp、Adam）只是在"步子怎么迈"上多动了一点脑筋，却往往决定了你是 10 步到达，还是 10000 步还在山腰打转。 Blindfolded on a hillside, able to feel only the slope under your feet, how do you reach the valley floor? Step in the steepest downhill direction, feel again, step again. That is gradient descent. Its relatives (momentum, Nesterov, RMSProp, Adam) only think a little harder about how to step, yet that often decides whether you arrive in 10 steps or are still circling the hillside after 10,000.

约 35 分钟~35 min 前置：3.2 梯度 · 3.5 Hessian · 6.1 什么是优化问题Prereqs: 3.2 Gradient · 3.5 Hessian · 6.1 Optimization problems 5 个交互5 interactives

1为什么机器人需要它Why robots need this

训练一个视觉-语言-动作模型 (VLA) 或扩散策略 (diffusion policy)，就是调整几亿个参数，让"预测动作与示范动作之间的误差"变小。让机械臂末端到达指定位置的数值逆运动学 (inverse kinematics)，是调整几个关节角，让"末端与目标之间的距离"变小。轨迹优化、相机标定、点云配准也一样，都是在一个代价函数上找最低点。

这些问题规模天差地别，但最常用的求解办法都是同一个：沿着梯度往下走。6.1 节讲了怎样把问题写成"最小化 $f(\x)$"；这一节讲怎样真的走下去，以及为什么同样是往下走，有人 10 步就到了，有人走了一万步还在山腰。

Training a vision-language-action model (VLA) or a diffusion policy means adjusting hundreds of millions of parameters so that "the error between predicted and demonstrated actions" shrinks. Numerical inverse kinematics adjusts a handful of joint angles so that "the distance between the gripper and the target" shrinks. Trajectory optimization, camera calibration and point-cloud registration are the same story: find the lowest point of a cost function.

These problems differ wildly in size, yet the workhorse solver is the same: walk downhill along the gradient. Lesson 6.1 showed how to phrase a problem as "minimize $f(\x)$"; this lesson is about actually walking down, and why one walker arrives in 10 steps while another is still on the hillside after ten thousand.

2直觉：蒙眼下山Intuition: walking downhill blindfolded

你看不到整座山，只能感觉脚下的坡度。3.2 节说过，梯度 $\nabla f$ 指向上坡最陡的方向，它的长度就是坡有多陡。所以最自然的规则是：朝 $-\nabla f$ 迈一步，步长正比于坡度。比例系数 $\eta$ 叫学习率 (learning rate)，也叫步长。

步子大小是第一个难题。步子太小，走到天黑也到不了；步子太大，一脚跨过谷底，落在对面更高的坡上，然后越跳越高。先在一维里感受一下。

You cannot see the whole mountain, only feel the slope underfoot. From Lesson 3.2, the gradient $\nabla f$ points in the direction of steepest ascent, and its length is how steep that is. So the natural rule is: step along $-\nabla f$, with a step proportional to the slope. The proportionality constant $\eta$ is the learning rate (or step size).

Step size is the first hard problem. Too small and you will not arrive before nightfall; too large and you leap clean over the valley floor onto the higher slope opposite, then bounce higher and higher. Feel it in one dimension first.

FIG 1一维梯度下降：步长的分寸1D gradient descent: getting the step right

点击曲线附近任意位置（或拖动空心圆）设定起点。紫色短线是脚下的切线，它的斜率就是 $f'(x)$；黄色箭头是下一步 $-\eta f'(x)$。抛物线 $f=\tfrac12 a x^2$ 模式下，右侧色条标出了三个区间：平稳收敛、来回跳但收敛、发散。 Click anywhere near the curve (or drag the hollow circle) to set the start. The purple segment is the tangent underfoot; its slope is $f'(x)$. The amber arrow is the next step, $-\eta f'(x)$. In parabola mode, $f=\tfrac12 a x^2$, the color bar on the right marks three regimes: smooth convergence, bouncing but converging, and divergence.

在抛物线上找一个学习率，让小球每步都跳到对面，却仍然收敛（至少走 4 步）。它和 $1/a$ 有什么关系？On the parabola, find a learning rate where the ball jumps to the other side every step yet still converges (take at least 4 steps). How does it relate to $1/a$?
把学习率调大，让它发散。再把曲率 $a$ 调大一倍：临界学习率怎么变？Raise the learning rate until it diverges. Now double the curvature $a$: what happens to the critical learning rate?

在抛物线上，一步更新是 $x \leftarrow x - \eta a x = (1-\eta a)\,x$：每一步都把 $x$ 乘上同一个数。这个数的绝对值小于 1 才会收敛，于是 $0 < \eta < 2/a$。曲率越大，能用的步长越小。记住这句话，下面所有的故事都从它开始。

到了二维，第二个难题出现了：一个方向陡、另一个方向缓的狭长山谷。陡的方向要求小步长，缓的方向却需要大步长才走得动，一个 $\eta$ 顾不了两头。梯度下降家族的其他成员，就是为这个问题想出的不同办法：

On the parabola one update is $x \leftarrow x - \eta a x = (1-\eta a)\,x$: every step multiplies $x$ by the same number. It converges only if that number has magnitude below 1, so $0 < \eta < 2/a$. The steeper the curvature, the smaller the step you can afford. Keep that sentence in mind; everything below grows out of it.

In two dimensions a second problem appears: a long narrow valley, steep in one direction and gentle in the other. The steep direction demands a small step; the gentle one needs a big step to make progress. One $\eta$ cannot serve both. The rest of the family are different answers to exactly this problem:

	形象比喻Picture	它额外记住了什么What it remembers
GD	蒙眼的人，每步只看脚下A blindfolded walker who only feels the ground underfoot	什么也不记Nothing
Momentum	一个沉重的铁球，有惯性A heavy ball with inertia	速度：过去梯度的累积A velocity: the running sum of past gradients
Nesterov	会"看路"的铁球：先按惯性滑一下，再看那里的坡A ball that looks ahead: coast first, then feel the slope there	速度，外加在"预计落点"处求梯度A velocity, plus the gradient taken at the predicted position
RMSProp	每个坐标各配一个油门：常年陡的方向收着走，常年平的方向放开走A separate throttle per coordinate: ease off where it is always steep, open up where it is always flat	每个坐标梯度平方的滑动平均A running average of each coordinate's squared gradient
Adam	铁球 + 分坐标油门Heavy ball plus per-coordinate throttles	梯度的滑动平均和梯度平方的滑动平均Running averages of the gradient and of its square

核心洞见Key insight

梯度只告诉你局部哪边是下坡，不告诉你最低点在哪。所有改进都在回答同一个问题：只凭一路上摸到的坡度，怎样把步子迈得更聪明？动量用历史抵消来回摆动，自适应方法用统计给每个方向配不同的步长。

The gradient tells you only which way is downhill locally, never where the bottom is. Every improvement answers one question: using nothing but the slopes felt along the way, how can you step more cleverly? Momentum uses history to cancel back-and-forth swinging; adaptive methods use statistics to give each direction its own step size.

3动手玩：优化器赛跑Play: the optimizer race

下面是一张"地形图"：底色里青色越浓的地方越低，等高线标出等高的位置。点击地图任意位置设定起点，五个优化器会同时出发。右侧可以开关每个优化器、看它的实时更新公式；下方是损失曲线（对数坐标，越往下越接近最小值）。换一种地形、拖一拖滑块，看看谁能赢。

Below is a terrain map: the stronger the cyan tint, the lower the ground, and the contour lines join points of equal height. Click anywhere on the map to set a start point and all five optimizers set off at once. On the right you can toggle each optimizer and watch its live update formula; underneath is the loss curve (log scale: lower is closer to the minimum). Switch terrains, drag the sliders, and see who wins.

FIG 2优化器赛跑Optimizer race

实时公式Live update

两个学习率是分开的：GD、动量、Nesterov 的一步是 $\eta\times$ 梯度，所以 $\eta$ 的合适大小取决于坡有多陡；RMSProp 和 Adam 先把梯度"归一化"，一步大约就是 $\eta_{\text{adapt}}$ 这么长。RMSProp 的 $\rho$ 固定为 0.9。切换地形时学习率会恢复为该地形的推荐值。 There are two learning rates. For GD, momentum and Nesterov a step is $\eta\times$ gradient, so a good $\eta$ depends on how steep the terrain is; RMSProp and Adam normalize the gradient first, so a step is roughly $\eta_{\text{adapt}}$ long. RMSProp's $\rho$ is fixed at 0.9. Switching terrain restores that terrain's suggested learning rates.

试一试Try this

在 Rosenbrock 香蕉形山谷里，让 Adam 到达最小值 (1, 1)。再看看 GD 在弯弯的谷底爬得有多慢。In the Rosenbrock banana valley, get Adam to the minimum at (1, 1). Then watch how slowly GD crawls along the curved valley floor.
让 GD 发散。在二次碗上，临界学习率是多少？（提示：最陡方向的曲率是 2。）Make GD diverge. On the quadratic bowl, what is the critical learning rate? (Hint: the steepest curvature is 2.)
在二次碗上把条件数 κ 调到 30 以上，让 Momentum 在 GD 还在路上时就先收敛。On the quadratic bowl, push the condition number κ above 30 and let Momentum converge while GD is still on its way.

多玩几种地形，你会发现没有哪个优化器总是赢。在"双谷"地形上，动量有时能冲过中间的小山包，掉进更深的那个谷；在"鞍点"地形上，GD 会在几乎平坦的鞍点附近犹豫很久，Adam 因为把小梯度"放大"了，往往最先逃出来。

Try several terrains and you will find that no optimizer always wins. On "two minima", momentum can sometimes roll over the hump into the deeper valley; on "saddle", GD dithers for a long time near the almost-flat saddle point, while Adam, which effectively amplifies small gradients, often escapes first.

4为什么会走"之"字：条件数Why the zig-zag: the condition number

在狭长的二次碗里，GD 为什么不直奔谷底？因为梯度垂直于等高线（3.2 节）。在一个扁椭圆上，除了两条主轴，垂直于等高线的方向几乎都指向"对面的谷壁"，而不是指向中心。

两个方向的曲率之比叫条件数 $\kappa = \lambda_{\max}/\lambda_{\min}$，也就是 3.5 节 Hessian 最大和最小特征值之比。κ 越大，碗越扁，GD 的"之"字越密，需要的步数大约随 κ 线性增长。拖动起点和 κ 看看。

Why does GD not head straight for the bottom of a long, narrow bowl? Because the gradient is perpendicular to the contour lines (Lesson 3.2). On a flat ellipse, except along the two principal axes, the perpendicular direction points mostly at the opposite valley wall, not at the center.

The ratio of the two curvatures is the condition number $\kappa = \lambda_{\max}/\lambda_{\min}$: the ratio of the largest to smallest Hessian eigenvalue from Lesson 3.5. The larger κ, the flatter the bowl, the denser GD's zig-zag, and the number of steps grows roughly linearly in κ. Drag the start point and κ to see it.

FIG 3梯度垂直于等高线，不指向最小值The gradient is perpendicular to the contour, not aimed at the minimum

收敛步数 vs. κ（对数坐标）Steps to converge vs. κ (log–log)

GD 用它的最优步长 $\eta^\star = 2/(\lambda_{\max}+\lambda_{\min})$；动量用 $\eta = 1/\lambda_{\max}$、$\beta = (1-1/\sqrt\kappa)^2$，恰好让最缓的方向处于临界阻尼（第 5、6 节）。收敛的标准是离最小值的距离缩小到起点的千分之一。黄色箭头是 $-\nabla f$，青色虚线是"真正该去的方向"。右图中实线是理论值，圆点是左图实际走的步数。 GD uses its best step $\eta^\star = 2/(\lambda_{\max}+\lambda_{\min})$; momentum uses $\eta = 1/\lambda_{\max}$ and $\beta = (1-1/\sqrt\kappa)^2$, which critically damps the gentlest direction (Sections 5 and 6). "Converged" means the distance to the minimum shrank to one thousandth of the start. The amber arrow is $-\nabla f$; the dashed cyan line is where you actually want to go. On the right, solid lines are theory and dots are the step counts from the left panel.

注意右图两条线的斜率：GD 那条是 1（步数 ∝ κ），动量那条是 1/2（步数 ∝ √κ）。κ = 100 时，这就是几百步和几十步的差别。神经网络和机器人代价函数的 κ 动辄成千上万，这个差别就是"一晚上训完"和"一周训完"的差别。

Look at the slopes of the two lines on the right: GD's is 1 (steps ∝ κ), momentum's is 1/2 (steps ∝ √κ). At κ = 100 that is the difference between hundreds of steps and dozens. Neural-network and robot cost functions routinely have κ in the thousands or more, where this is the difference between training overnight and training for a week.

5第二个视角：动量是一颗有摩擦的铁球A second view: momentum is a heavy ball with friction

把 GD 想成"没有质量"的点：每一步的速度完全由当前的坡度决定，上一步怎么走的，它全忘了。动量法给这个点加上质量：速度 $\v$ 先保留上一步的一部分（比例 $\beta$），再加上新的坡度。

在狭长山谷里，横穿山谷方向的梯度一正一负，累积起来互相抵消；沿着谷底方向的梯度始终同号，累积起来越滚越快。这正是 2.5 节的弹簧-阻尼系统：$1-\beta$ 像摩擦，曲率像弹簧刚度。摩擦太小，球会在谷里来回荡（欠阻尼）；摩擦太大，球爬得像 GD 一样慢（过阻尼）；刚刚好的时候，最快停到底（临界阻尼）。

Think of GD as a massless point: its velocity is set entirely by the current slope, and it forgets how it moved a moment ago. Momentum gives the point mass: the velocity $\v$ keeps a fraction $\beta$ of the previous step, then adds the new slope.

In a narrow valley, the across-valley gradients alternate in sign and cancel when accumulated; the along-valley gradients always share a sign and accumulate, so the ball picks up speed. This is the spring–damper of Lesson 2.5: $1-\beta$ acts as friction and curvature as spring stiffness. Too little friction and the ball sloshes back and forth (underdamped); too much and it crawls like GD (overdamped); just right and it settles fastest (critically damped).

FIG 4山谷里的铁球A heavy ball in a ravine

两个方向上的位置随步数变化Position along each direction vs. step

山谷沿 x 方向很缓（曲率 0.05），沿 y 方向很陡（曲率 2），κ = 40。紫色箭头是速度（这一步的位移），黄色箭头是这一步新加的 $-\eta\nabla f$，两者都放大了 2 倍（太长时截短）。点击地图可以换起点。右侧曲线就是两个方向上的"弹簧振动"：β = 0 时 x 方向像过阻尼一样爬，β 太大时两个方向都在荡。 The valley is gentle along x (curvature 0.05) and steep along y (curvature 2): κ = 40. The purple arrow is the velocity (this step's displacement); the amber arrow is the freshly added $-\eta\nabla f$; both are magnified 2× (and clipped when very long). Click the map to change the start. The curves on the right are the two "spring oscillations": with β = 0 the x direction crawls like an overdamped spring; with β too large both directions slosh.

调节 β（也可以动 η），让铁球在 60 步以内停到谷底（离最小值小于 0.01）。默认的 β = 0.95 和 β = 0 都做不到。Tune β (and η if you like) so the ball settles at the bottom (within 0.01 of the minimum) in under 60 steps. Neither the default β = 0.95 nor β = 0 manages it.

两种视角Two views

算法视角：动量是"把过去的梯度做指数加权平均"，权重按 $\beta^k$ 衰减，有效记忆大约 $1/(1-\beta)$ 步。物理视角：动量是对带阻尼的牛顿方程 $\ddot{\x} + \gamma\dot{\x} = -\nabla f(\x)$ 的离散化。前者告诉你怎么写代码，后者告诉你 β 该怎么调。

Algorithmic view: momentum is an exponentially weighted average of past gradients, with weights decaying as $\beta^k$, so its memory is about $1/(1-\beta)$ steps. Physical view: momentum discretizes the damped Newton equation $\ddot{\x} + \gamma\dot{\x} = -\nabla f(\x)$. The first tells you how to write the code; the second tells you how to tune β.

6再看数学The math

提示：把鼠标悬停在公式中带底色的符号上，FIG 2 和 FIG 4 中对应的元素会被高亮（梯度方向、步长、速度、Adam 的轨迹）。Tip: hover the highlighted symbols in the formulas; the matching elements in FIG 2 and FIG 4 light up (gradient directions, steps, velocities, Adam's path).

五条更新规则The five update rules

记 $\mathbf g_k = \nabla f(\x_k)$。梯度下降：

Write $\mathbf g_k = \nabla f(\x_k)$. Gradient descent:

$$ \x_{k+1} = \x_k - \hl{lr}{\eta}\, \hl{grad}{\cv{\nabla f(\x_k)}} $$

动量（heavy ball, Polyak 1964）：速度是梯度的累积，位置沿速度走。

Momentum (heavy ball, Polyak 1964): the velocity accumulates gradients and the position follows the velocity.

$$ \hl{vel}{\cw{\v_{k+1}}} = \hl{beta}{\beta}\,\cw{\v_k} + \cv{\mathbf g_k}, \qquad \x_{k+1} = \x_k - \eta\, \cw{\v_{k+1}} $$

Nesterov：先按惯性"预支"一步，在预计落点处求梯度，再修正。冲过头时它能更早刹车。

Nesterov: borrow a step of coasting first, take the gradient at the predicted point, then correct. It brakes earlier when overshooting.

$$ \cw{\v_{k+1}} = \beta\,\cw{\v_k} + \cv{\nabla f(\x_k - \eta\beta\,\v_k)}, \qquad \x_{k+1} = \x_k - \eta\, \cw{\v_{k+1}} $$

RMSProp 和 Adam：用梯度平方的滑动平均 $\mathbf s$ 给每个坐标单独定步长（平方、开方、除法都按分量进行）。Adam 还对梯度本身做滑动平均 $\mathbf m$，并做偏差修正：$\mathbf m, \mathbf s$ 从 0 开始，前几步会偏小，除以 $1-\beta^{k}$ 把它们拉回正常大小。

RMSProp and Adam: a running average $\mathbf s$ of squared gradients gives every coordinate its own step size (squares, roots and division are element-wise). Adam also averages the gradient itself, $\mathbf m$, and applies bias correction: because $\mathbf m, \mathbf s$ start at 0 they are too small for the first few steps, and dividing by $1-\beta^{k}$ restores their proper size.

$$ \text{RMSProp:}\quad \mathbf s_{k+1} = \rho\,\mathbf s_k + (1-\rho)\,\mathbf g_k^{2}, \qquad \x_{k+1} = \x_k - \frac{\eta}{\sqrt{\mathbf s_{k+1}}+\epsilon}\,\mathbf g_k $$ $$ \hl{adam}{\ck{\text{Adam:}}}\quad \begin{aligned} \mathbf m_{k+1} &= \beta_1 \mathbf m_k + (1-\beta_1)\,\mathbf g_k, & \hat{\mathbf m} &= \mathbf m_{k+1}/(1-\beta_1^{\,k+1}) \\ \mathbf s_{k+1} &= \beta_2 \mathbf s_k + (1-\beta_2)\,\mathbf g_k^{2}, & \hat{\mathbf s} &= \mathbf s_{k+1}/(1-\beta_2^{\,k+1}) \\ \x_{k+1} &= \x_k - \eta\,\hat{\mathbf m}\,/\,(\sqrt{\hat{\mathbf s}}+\epsilon) \end{aligned} $$

Adam 的一步大致是 $\eta \cdot \hat m/\sqrt{\hat s}$。如果梯度方向稳定，$\hat m/\sqrt{\hat s}\approx \pm 1$，每个坐标都走约 $\eta$ 那么远，与坡的陡缓无关。这就是为什么 FIG 2 里 Adam 的学习率要单独设置：它的单位是"参数的单位"，而不是"参数 / 梯度"。

An Adam step is roughly $\eta \cdot \hat m/\sqrt{\hat s}$. If the gradient direction is steady, $\hat m/\sqrt{\hat s}\approx \pm 1$ and every coordinate moves about $\eta$, regardless of steepness. That is why Adam gets its own learning rate in FIG 2: its units are "units of the parameter", not "parameter per gradient".

二次函数上的精确分析Exact analysis on a quadratic

任何光滑函数在最小值附近都像一个二次碗 $f(\x)=\tfrac12\x^\top H\x$（3.5 节的泰勒展开），所以二次函数上的结论处处适用。此时 $\nabla f = H\x$，GD 的一步是 $\x_{k+1} = (I-\eta H)\,\x_k$。在 $H$ 的特征向量坐标系里，各个方向互不干扰，第 $i$ 个分量每步乘以 $1-\eta\lambda_i$，和 FIG 1 的一维情形一模一样：

Near a minimum every smooth function looks like a quadratic bowl $f(\x)=\tfrac12\x^\top H\x$ (the Taylor expansion of Lesson 3.5), so results on quadratics apply everywhere. Here $\nabla f = H\x$ and a GD step is $\x_{k+1} = (I-\eta H)\,\x_k$. In the eigenvector coordinates of $H$ the directions decouple, and component $i$ is multiplied by $1-\eta\lambda_i$ every step, exactly the 1D story of FIG 1:

$$ c_{i,k+1} = (1-\hl{lr}{\eta}\,\lambda_i)\,c_{i,k} \quad\Longrightarrow\quad \text{converges} \iff 0 < \hl{lr}{\eta} < \frac{2}{\lambda_{\max}} $$

最慢的方向决定总速度。让最陡和最缓两个方向的收缩率一样好，得到最优步长和收敛率：

The slowest direction sets the overall pace. Balancing the steepest and gentlest directions gives the best step size and its rate:

$$ \eta^\star = \frac{2}{\lambda_{\max}+\lambda_{\min}}, \qquad \|\x_k\| \le \Big(\frac{\hl{kappa}{\kappa}-1}{\hl{kappa}{\kappa}+1}\Big)^{k}\|\x_0\|, \qquad \hl{kappa}{\kappa} = \frac{\lambda_{\max}}{\lambda_{\min}} $$

κ 很大时 $(\kappa-1)/(\kappa+1)\approx 1-2/\kappa$，误差缩小 $e$ 倍需要约 $\kappa/2$ 步。动量把每个方向变成一个二阶线性递推（特征方程 $z^2-(1+\beta-\eta\lambda)z+\beta=0$，FIG 4 右侧实时显示），选 $\eta = 4/(\sqrt{\lambda_{\max}}+\sqrt{\lambda_{\min}})^2$、$\beta = \big(\tfrac{\sqrt\kappa-1}{\sqrt\kappa+1}\big)^2$ 时，所有方向都恰好处在临界附近，收敛率变成：

For large κ, $(\kappa-1)/(\kappa+1)\approx 1-2/\kappa$: shrinking the error by a factor $e$ takes about $\kappa/2$ steps. Momentum turns each direction into a second-order linear recurrence (characteristic equation $z^2-(1+\beta-\eta\lambda)z+\beta=0$, shown live beside FIG 4). With $\eta = 4/(\sqrt{\lambda_{\max}}+\sqrt{\lambda_{\min}})^2$ and $\beta = \big(\tfrac{\sqrt\kappa-1}{\sqrt\kappa+1}\big)^2$, every direction sits at or just inside the critical boundary and the rate becomes:

$$ \text{rate}_{\text{momentum}} = \frac{\sqrt{\kappa}-1}{\sqrt{\kappa}+1} \approx 1-\frac{2}{\sqrt{\kappa}} \qquad\Longrightarrow\qquad \text{steps} \propto \sqrt{\kappa}\ \ \text{instead of}\ \ \kappa $$

直觉上：GD 的步长被最陡方向"卡住"，在最缓的方向上每步只能前进 $\eta\lambda_{\min}\approx 2/\kappa$ 的比例；动量在缓的方向上积累速度，把这个比例放大到约 $2/\sqrt\kappa$，同时在陡的方向上靠符号交替的梯度相互抵消，保持稳定。

Intuitively: GD's step is capped by the steepest direction, so along the gentlest direction each step covers only a fraction $\eta\lambda_{\min}\approx 2/\kappa$ of the remaining distance. Momentum builds up speed along the gentle direction, raising that fraction to about $2/\sqrt\kappa$, while in the steep direction the alternating-sign gradients cancel and keep it stable.

7机器人中的它：用梯度下降解逆运动学In robotics: inverse kinematics by gradient descent

一个三连杆平面机械臂，关节角 $\mathbf q = (q_1, q_2, q_3)$，正运动学 $\mathrm{FK}(\mathbf q)$ 给出末端位置。想让末端到达目标 $\mathbf t$，就最小化距离的平方：

Take a three-link planar arm with joint angles $\mathbf q = (q_1, q_2, q_3)$; forward kinematics $\mathrm{FK}(\mathbf q)$ gives the gripper position. To bring the gripper to a target $\mathbf t$, minimize the squared distance:

$$ C(\mathbf q) = \tfrac12\,\big\|\,\underbrace{\mathrm{FK}(\mathbf q) - \mathbf t}_{\ci{\mathbf e}}\,\big\|^2, \qquad \nabla_{\mathbf q} C = J(\mathbf q)^\top\,\ci{\mathbf e} $$

梯度来自链式法则：$\mathbf e$ 对 $\mathbf q$ 的导数就是机器人雅可比 $J$（3.3、3.4 节），于是 $\nabla C = J^\top\mathbf e$。它的含义很直观：每个关节"感受"到的力，等于误差向量在这个关节能推动末端的方向上的投影。拖动黄色目标，看机械臂一步步追过去。

The gradient comes from the chain rule: the derivative of $\mathbf e$ with respect to $\mathbf q$ is the robot Jacobian $J$ (Lessons 3.3 and 3.4), so $\nabla C = J^\top\mathbf e$. The meaning is intuitive: each joint "feels" the error vector projected onto the direction in which that joint can push the gripper. Drag the amber target and watch the arm chase it step by step.

FIG 5数值逆运动学Numerical inverse kinematics

连杆长 1.0、0.8、0.6，虚线圆是工作空间边界（半径 2.4）。红色虚线是误差 $\mathbf e$，淡色的手臂是之前的迭代。目标放到圈外时，机械臂会伸直指向它，代价停在一个大于 0 的值：梯度为零，但目标没有到达。右下角是代价随迭代的变化（对数坐标）。 Link lengths 1.0, 0.8, 0.6; the dashed circle is the workspace boundary (radius 2.4). The dashed red line is the error $\mathbf e$; faint arms are earlier iterations. Put the target outside the circle and the arm straightens to point at it while the cost stalls above 0: the gradient is zero but the target is not reached. Bottom right: cost vs. iteration (log scale).

把目标拖到离基座 0.8 以内，让机械臂把自己"折"起来够到它（误差小于 0.01）。哪个优化器最快？Drag the target within 0.8 of the base and let the arm fold itself up to reach it (error below 0.01). Which optimizer is fastest?

真实系统里In real systems

工业级 IK 求解器（如 KDL、TRAC-IK）很少用纯梯度下降，而是用阻尼最小二乘 / Levenberg–Marquardt（6.6 节），它利用了 $J^\top J$ 这个曲率信息，几步就收敛。但在神经网络训练里，参数动辄上亿，算不起 Hessian，Adam / AdamW 就是默认选择：从 ACT、Diffusion Policy 到 RT-2、OpenVLA 一类的 VLA 模型，训练脚本里几乎都写着 AdamW。轨迹优化、可微仿真、NeRF / 3D Gaussian Splatting 场景重建，也大量使用 Adam。

Production IK solvers (KDL, TRAC-IK) rarely use plain gradient descent; they use damped least squares / Levenberg–Marquardt (Lesson 6.6), which exploits the curvature information in $J^\top J$ and converges in a few steps. But in neural-network training, with parameters in the hundreds of millions, the Hessian is unaffordable and Adam / AdamW is the default: from ACT and Diffusion Policy to VLAs like RT-2 and OpenVLA, the training scripts nearly all say AdamW. Trajectory optimization, differentiable simulation and NeRF / 3D Gaussian Splatting reconstruction lean heavily on Adam too.

8常见误区Common pitfalls

误区 1：损失变成 NaN，以为是代码 bugPitfall 1: the loss turns NaN and you assume a code bug

最常见的原因就是学习率太大（$\eta > 2/\lambda_{\max}$），参数一步比一步大，最后溢出。反过来，学习率太小时损失曲线几乎是平的，看起来像"模型学不动"。调参的第一步：在对数刻度上扫学习率（如 1e-5、1e-4、1e-3……），先找到"刚好不发散"的那个量级，再往下退一档。

The most common cause is simply a learning rate that is too large ($\eta > 2/\lambda_{\max}$): the parameters grow every step until they overflow. Conversely, a learning rate that is too small gives an almost flat loss curve that looks like "the model can't learn". First tuning step: sweep the learning rate on a log scale (1e-5, 1e-4, 1e-3, …), find the order of magnitude that just barely does not diverge, then back off one notch.

误区 2：参数单位悬殊，却不做缩放Pitfall 2: parameters with wildly different units, left unscaled

相机标定里，焦距约 600 像素，畸变系数约 0.01；轨迹优化里，位置以米计，时间以秒计，力矩以牛·米计。单位不同，曲率就能差好几个数量级，κ 巨大，GD 会走"之"字走到天荒地老。办法是先把每个参数归一化到差不多的量级（或输入特征做标准化）；这其实就是在手动做 RMSProp / Adam 自动做的事。

In camera calibration the focal length is about 600 pixels while distortion coefficients are about 0.01; in trajectory optimization positions are in meters, times in seconds and torques in newton-meters. Different units can make curvatures differ by orders of magnitude, κ becomes enormous, and GD zig-zags forever. The fix is to rescale every parameter to a similar magnitude (or standardize input features), which is doing by hand what RMSProp / Adam do automatically.

误区 3：在高维空间里最怕局部最小值Pitfall 3: fearing local minima most in high dimensions

低维直觉会让你担心掉进局部最小值。但在上百万维的空间里，一个临界点要成为局部最小值，需要所有方向的曲率都向上，这很难；更常见的是鞍点和大片平坦区域：有些方向向上，有些向下，梯度很小，GD 在附近磨蹭很久（FIG 2 的"鞍点"地形）。动量和自适应方法逃离鞍点的速度通常快得多。

Low-dimensional intuition makes you fear local minima. But in a space with millions of dimensions, a critical point is a local minimum only if curvature points up in every direction, which is rare; far more common are saddle points and broad plateaus, curving up in some directions and down in others, with tiny gradients where GD dawdles for ages (the "saddle" terrain in FIG 2). Momentum and adaptive methods usually escape much faster.

误区 4：Adam 的默认参数是万能的Pitfall 4: Adam's defaults are magic

$\eta=10^{-3}, \beta_1=0.9, \beta_2=0.999$ 对很多神经网络是不错的起点，但不是普适答案。Adam 在固定学习率下常常在最小值附近来回抖动（FIG 2 的 RMSProp 也是），所以实际训练要配合学习率衰减；权重衰减要用 AdamW 的"解耦"写法；在一些凸问题或小问题上，调好的 SGD + 动量甚至 L-BFGS 反而更好。

$\eta=10^{-3}, \beta_1=0.9, \beta_2=0.999$ is a decent starting point for many neural networks, not a universal answer. With a constant learning rate Adam often jitters around the minimum (as RMSProp does in FIG 2), so real training pairs it with learning-rate decay; weight decay should use AdamW's decoupled form; and on some convex or small problems, well-tuned SGD with momentum, or even L-BFGS, does better.

误区 5：手推的梯度不做检查Pitfall 5: trusting a hand-derived gradient without checking

梯度写错一个符号，优化器照样会跑，只是往错误的方向跑，而且很难察觉。上线前用中心差分对比一下：

Get one sign wrong in a gradient and the optimizer still runs, just in a wrong direction, and the bug is hard to spot. Before trusting it, compare against central differences:

$$ \frac{\partial f}{\partial x_i} \approx \frac{f(\x + h\,\mathbf e_i) - f(\x - h\,\mathbf e_i)}{2h}, \qquad h \approx 10^{-5}\text{ (double precision)} $$

误差是 $O(h^2)$。相对误差在 1e-6 左右说明梯度没问题，在 1e-2 左右说明一定有 bug。本页 FIG 5 的 $J^\top\mathbf e$ 就是这样验证过的。

The error is $O(h^2)$. A relative error around 1e-6 means the gradient is fine; around 1e-2 means there is definitely a bug. The $J^\top\mathbf e$ in FIG 5 was checked exactly this way.

9小练习Check yourself

1. 对 $f(x)=\tfrac12\cdot 4x^2$ 用学习率 $\eta = 0.6$ 做梯度下降，会怎样？1. Run gradient descent on $f(x)=\tfrac12\cdot 4x^2$ with $\eta = 0.6$. What happens?

每步乘以 $1-\eta a = 1-0.6\times4 = -1.4$。负号表示跳到对面，绝对值大于 1 表示越跳越远。临界学习率是 $2/a = 0.5$。Each step multiplies by $1-\eta a = 1-0.6\times4 = -1.4$. The minus sign means jumping to the other side; magnitude above 1 means each jump is larger. The critical learning rate is $2/a = 0.5$.

2. 一个二次函数的 Hessian 特征值是 10 和 0.1。梯度下降能稳定的最大学习率约是多少？此时条件数是多少？2. A quadratic has Hessian eigenvalues 10 and 0.1. What is the largest stable GD learning rate, and what is the condition number?

稳定条件 $\eta < 2/\lambda_{\max} = 0.2$，$\kappa = 10/0.1 = 100$。用这样的步长，在 $\lambda=0.1$ 的方向上每步只缩小约 2%，GD 需要几百步；动量大约只需几十步。Stability needs $\eta < 2/\lambda_{\max} = 0.2$, and $\kappa = 10/0.1 = 100$. With such a step the $\lambda=0.1$ direction shrinks only about 2% per step, so GD needs hundreds of steps; momentum needs only dozens.

3. 在一个狭长的椭圆形山谷里，GD 为什么走"之"字？3. Why does GD zig-zag in a long, narrow elliptical valley?

椭圆越扁（κ 越大），梯度方向和"指向中心的方向"夹角越大。为了不在陡方向发散，步长又必须小于 $2/\lambda_{\max}$，于是在缓方向上进展很慢，在陡方向上来回弹。The flatter the ellipse (larger κ), the larger the angle between the gradient and the direction to the center. To avoid diverging in the steep direction the step must stay below $2/\lambda_{\max}$, so progress along the gentle direction is slow while the steep direction bounces.

4. 机械臂 IK 的代价 $C(\mathbf q)=\tfrac12\|\mathrm{FK}(\mathbf q)-\mathbf t\|^2$，它对关节角的梯度是？（$J$ 是 $\mathrm{FK}$ 的雅可比，$\mathbf e = \mathrm{FK}(\mathbf q)-\mathbf t$）4. For the IK cost $C(\mathbf q)=\tfrac12\|\mathrm{FK}(\mathbf q)-\mathbf t\|^2$, what is its gradient with respect to the joint angles? ($J$ is the Jacobian of $\mathrm{FK}$, $\mathbf e = \mathrm{FK}(\mathbf q)-\mathbf t$.)

链式法则：$dC = \mathbf e^\top d\mathbf e = \mathbf e^\top J\,d\mathbf q$，所以 $\nabla C = J^\top\mathbf e$。维度也对得上：$J$ 是 2×3，$\mathbf e$ 是 2 维，$J^\top\mathbf e$ 是 3 维，正好每个关节一个分量。$J\mathbf e$ 的维度对不上；$J$ 不是方阵，没有普通的逆。Chain rule: $dC = \mathbf e^\top d\mathbf e = \mathbf e^\top J\,d\mathbf q$, so $\nabla C = J^\top\mathbf e$. The dimensions agree: $J$ is 2×3 and $\mathbf e$ is 2-D, so $J^\top\mathbf e$ is 3-D, one entry per joint. $J\mathbf e$ does not even type-check, and a non-square $J$ has no ordinary inverse.

10延伸阅读Further reading

G. Goh, Why Momentum Really Works, Distill, 2017 — 本节 FIG 3、FIG 4 的灵感来源，对动量的特征值分析讲得极好The inspiration for FIG 3 and FIG 4; a superb eigenvalue analysis of momentum
S. Ruder, An overview of gradient descent optimization algorithms, arXiv:1609.04747 — 各种优化器的简明综述A concise survey of the optimizer zoo
S. Boyd & L. Vandenberghe, Convex Optimization, Ch. 9 — 无约束极小化：梯度法、最速下降、条件数与收敛分析Unconstrained minimization: gradient methods, steepest descent, conditioning and convergence
D. P. Kingma & J. Ba, Adam: A Method for Stochastic Optimization, ICLR 2015 — Adam 原论文，包括偏差修正的推导The original Adam paper, including the bias-correction derivation