特征值与特征向量Eigenvalues & Eigenvectors
矩阵会把大多数向量推离原来的方向,但总有几条特殊的直线:向量落在上面,只会被拉长、压短或翻到另一侧,方向不变。这些方向叫特征向量,伸缩的倍数叫特征值。找到它们,一个纠缠的矩阵就拆成几个互不干扰的一维问题:机器人会不会稳定下来、会不会来回振荡、不确定性椭圆朝向哪里,都能直接读出来。 A matrix knocks most vectors off their line. But a few special lines survive: a vector on one of them is only stretched, shrunk or flipped, never turned. Those directions are the eigenvectors and the stretch factors are the eigenvalues. Find them and a tangled matrix falls apart into independent one-dimensional problems: whether a robot settles or rings, which way an uncertainty ellipse points, all can be read straight off.
1为什么机器人需要它Why robots need this
机器人的控制器每隔几毫秒读一次状态误差(位置差多少、速度差多少),再算出下一步的指令。把这个闭环线性化,几乎总能写成
A robot's controller reads the state error every few milliseconds (how far off is the position, how far off is the velocity) and computes the next command. Linearize that closed loop and it almost always reads
误差会逐渐消失,会来回振荡,还是越滚越大?答案不藏在 \(A\) 的某一个元素里,而藏在 \(A\) 的特征值 (eigenvalues) 里。这是特征值在机器人里最重要的用途,但不是唯一的:
- 稳定性:上面这个问题。所有特征值的模都小于 1,系统就会稳定下来;有一个大于 1,就会发散(见下文第 4 和第 7 部分)。
- 不确定性椭圆:状态估计给出的协方差矩阵是对称矩阵,它的特征向量就是椭圆的长短轴方向,特征值决定轴有多长(4.4 节)。
- 旋转轴:任何三维旋转都固定了一条直线,这条轴的方向就是旋转矩阵的特征向量,特征值恰好是 1(见下文第 7 部分)。
- 振动模态:柔性机械臂的每一种“摇晃方式”是一个特征向量,对应的特征值是固有频率的平方。
- SVD 与 PCA:1.12 节的奇异值分解,以及点云的主成分分析,都建立在对称矩阵的特征分解之上。
Does the error die out, ring back and forth, or snowball? The answer is not hiding in any single entry of \(A\); it sits in the eigenvalues of \(A\). That is the most important use of eigenvalues in robotics, but not the only one:
- Stability: the question above. If every eigenvalue has magnitude below 1 the system settles; if one exceeds 1 it diverges (Sections 4 and 7 of this lesson).
- Uncertainty ellipses: the covariance matrix from a state estimator is symmetric; its eigenvectors are the axes of the ellipse and its eigenvalues set how long the axes are (Lesson 4.4).
- Rotation axes: every 3D rotation fixes one line in space. The axis direction is an eigenvector of the rotation matrix, with eigenvalue exactly 1 (Section 7).
- Vibration modes: each way a flexible arm can wobble is an eigenvector, and its eigenvalue is the square of the natural frequency.
- SVD and PCA: the singular value decomposition of Lesson 1.12, and principal component analysis of point clouds, are both built on eigen-decompositions of symmetric matrices.
2直觉:哪些方向不被“推歪”Intuition: which directions don't get knocked off their line
1.3 节里,我们把矩阵看成对整个平面的一次动作。向量 \(\mathbf v\) 经过矩阵 \(A\) 变成 \(A\mathbf v\),通常两者指向不同的方向:矩阵把它推离了原来那条过原点的直线。
不过,常常有几个特殊的方向。向量站在上面时,矩阵只会沿着这条线把它拉长、压短,或翻到另一侧,却不会让它转弯。这样的方向叫特征向量 (eigenvector),伸缩的倍数叫特征值 (eigenvalue)。“eigen” 是德语,意思是“自己的、固有的”:这是矩阵自己的方向。
In Lesson 1.3 we treated a matrix as an action on the whole plane. A vector \(\mathbf v\) becomes \(A\mathbf v\), and usually the two point in different directions: the matrix has pushed it off the line through the origin it started on.
But there are often a few special directions. A vector standing on one of them is only stretched, shrunk or flipped along that same line, never turned. Such a direction is an eigenvector and the scale factor is its eigenvalue. "Eigen" is German for "own" or "characteristic": these are the matrix's own directions.
特征值的数值直接告诉你沿这条线发生了什么:
- \(\lambda > 1\):被拉长;\(0 < \lambda < 1\):被压短;\(\lambda = 1\):原地不动。
- \(\lambda < 0\):翻到原点的另一侧(再按 \(|\lambda|\) 伸缩)。
- \(\lambda = 0\):整条线被压成原点。这正是 1.6 节的零空间:特征值 0 意味着矩阵不可逆。
拖动下面左图里的琥珀色向量 \(\mathbf v\)(它只能绕圆走)。灰色虚线是 \(\mathbf v\) 所在的直线,青色箭头是 \(A\mathbf v\)。只要 \(A\mathbf v\) 离开了这条线,\(\mathbf v\) 就不是特征向量。右图画的是“偏了多少”:从 \(\mathbf v\) 转到 \(A\mathbf v\) 的夹角 \(\delta\) 的正弦。曲线碰到 0 的地方,就是 \(A\mathbf v\) 回到这条线上的地方。靠近特征方向时,\(\mathbf v\) 会被轻轻吸住。
The eigenvalue's number tells you exactly what happens along that line:
- \(\lambda > 1\): stretched; \(0 < \lambda < 1\): shrunk; \(\lambda = 1\): untouched.
- \(\lambda < 0\): flipped to the other side of the origin (then scaled by \(|\lambda|\)).
- \(\lambda = 0\): the whole line is crushed to the origin. This is exactly the null space of Lesson 1.6: eigenvalue 0 means the matrix is not invertible.
Drag the amber vector \(\mathbf v\) in the left plot (it can only travel around the circle). The grey dashed line is the line \(\mathbf v\) lies on, and the cyan arrow is \(A\mathbf v\). Whenever \(A\mathbf v\) leaves that line, \(\mathbf v\) is not an eigenvector. The right plot graphs "how far off": the sine of the angle \(\delta\) from \(\mathbf v\) to \(A\mathbf v\). Where the curve touches 0, \(A\mathbf v\) is back on the line. Near an eigen-direction \(\mathbf v\) gets gently snapped in place.
- 选“一般矩阵”,拖动 \(\mathbf v\),让 \(A\mathbf v\) 落回 \(\mathbf v\) 所在的直线上。能找到几个这样的位置(别忘了对面还有一个)?Choose "general matrix", then drag \(\mathbf v\) until \(A\mathbf v\) lies back on the line through \(\mathbf v\). How many such positions can you find (remember the opposite side)?
特征向量是被矩阵“沿着自己的线”作用的向量:\(A\mathbf v = \lambda\mathbf v\)。在这条线上,矩阵的复杂动作退化成一个数 \(\lambda\)。如果能找到足够多条这样的线,整个矩阵就只是“沿几个方向各自乘一个数”。
An eigenvector is a vector the matrix acts on "along its own line": \(A\mathbf v = \lambda\mathbf v\). On that line the matrix's complicated action shrinks to a single number \(\lambda\). If you can find enough such lines, the whole matrix is just "multiply by a number along each of a few directions".
3动手玩:特征方向探测器Play: the eigen-direction detector
现在换到完整的实验室。先造一个矩阵:拖动红色的 Aî 和绿色的 Aĵ 手柄,或者直接改数字,或者点预设。然后用琥珀色探针 \(\cv{\mathbf v}\) 去找特征方向:读数条显示 \(\mathbf v\) 与 \(A\mathbf v\) 所在两条直线的夹角。夹角小于 4° 时探针会闪一下,并在那条线上留下一个紫色标记。“扫描一圈”会让 \(\mathbf v\) 自己转一整圈。
圆周上那一圈细线是 72 个单位向量 \(\mathbf u\) 到 \(A\mathbf u\) 的位移(一只“刺猬”)。\(\mathbf u\) 是特征向量时,\(A\mathbf u = \lambda\mathbf u\) 仍在同一条线上,这根刺就是沿半径方向的;其他的刺都斜着扎出去。打开“显示答案”,真正的特征线(紫色虚线)和沿半径的刺会被标出来。
Now the full lab. First build a matrix: drag the red Aî and green Aĵ handles, edit the numbers, or pick a preset. Then hunt for eigen-directions with the amber probe \(\cv{\mathbf v}\): the meter shows the angle between the lines through \(\mathbf v\) and \(A\mathbf v\). Below 4° the probe flashes and drops a purple mark on that line. "Sweep" turns \(\mathbf v\) through a full circle by itself.
The ring of thin lines is the displacement from 72 unit vectors \(\mathbf u\) to \(A\mathbf u\) (a "hedgehog"). When \(\mathbf u\) is an eigenvector, \(A\mathbf u = \lambda\mathbf u\) stays on the same line and that spine points along the radius; all the other spines stick out at a slant. Turn on "show answer" and the true eigen-lines (purple dashes) and the radial spines are marked.
- 选预设“对称”(\(A=\begin{bmatrix}2&1\\1&2\end{bmatrix}\)),只靠探针找到它的两个特征方向。读一下两个特征值和两个方向的夹角。Pick the preset "Symmetric" (\(A=\begin{bmatrix}2&1\\1&2\end{bmatrix}\)) and find both eigen-directions using only the probe. Read the two eigenvalues and the angle between the two directions.
- 做一个没有实特征向量的矩阵,用“扫描一圈”确认:探针永远没有对齐。提示:想想 1.3 节的哪个动作会让每个向量都转弯。Make a matrix with no real eigenvectors and confirm it with a sweep: the probe never aligns. Hint: which action from Lesson 1.3 turns every vector?
- 找一个有特征值 −1 的矩阵,并用探针把对应的特征方向标出来。这个方向上的向量发生了什么?Find a matrix with eigenvalue −1 and mark the matching eigen-direction with the probe. What happens to vectors on that line?
- 选“剪切”,再扫描一圈。为什么只有一个特征方向?(想想剪切把哪些向量推歪、推歪多少。)Select "Shear" and sweep a full turn. Why is there only one eigen-direction? (Think about which vectors the shear tilts, and by how much.)
4第二种视角:把矩阵一遍遍用下去A second view: apply the matrix again and again
控制循环每个周期都把同一个规则用一次:\(\mathbf x_{k+1} = A\mathbf x_k\)。我们真正关心的是很多步以后的 \(\mathbf x_k = A^k\mathbf x_0\)。特征向量让这件事变得很简单。把起点拆成特征向量的组合 \(\mathbf x_0 = c_1\mathbf v_1 + c_2\mathbf v_2\),由于每个 \(\mathbf v_i\) 被 \(A\) 作用时只乘自己的 \(\lambda_i\):
A control loop applies the same rule once per cycle: \(\mathbf x_{k+1} = A\mathbf x_k\). What we really care about is the state many steps later, \(\mathbf x_k = A^k\mathbf x_0\). Eigenvectors make this easy. Split the starting point into eigenvectors, \(\mathbf x_0 = c_1\mathbf v_1 + c_2\mathbf v_2\); since each \(\mathbf v_i\) is only multiplied by its own \(\lambda_i\) when \(A\) acts:
每个分量各自按 \(\lambda_i^k\) 生长或衰减,互不干扰。于是:
- \(|\lambda|\) 最大的分量最终胜出:所有轨迹都弯向主特征向量(dominant eigenvector),每步伸缩约 \(|\lambda_{\max}|\) 倍。
- 所有 \(|\lambda_i| < 1\):每个分量都在缩小,轨迹收敛到原点(稳定)。只要有一个 \(|\lambda_i| > 1\),轨迹就会发散。
- \(\lambda < 0\):这个分量每一步翻到原点另一侧,轨迹“之字形”前进。
- 复特征值 \(\lambda = re^{\pm i\varphi}\):每一步转 \(\varphi\) 角并伸缩 \(r\) 倍,轨迹是螺旋。\(r<1\) 向内收,\(r>1\) 向外散,\(r=1\) 在椭圆上转圈。
在下面的画布上点击可以丢下新的种子点(最多 40 个)。拖动步数滑块或按“播放”,看每个种子走出的轨迹。右侧小图把特征值画在复平面上,单位圆是稳定与否的分界线。
Each component grows or decays on its own as \(\lambda_i^k\), without disturbing the other. So:
- The component with the largest \(|\lambda|\) eventually wins: every orbit bends toward the dominant eigenvector, and each step scales by about \(|\lambda_{\max}|\).
- All \(|\lambda_i| < 1\): every component shrinks and orbits converge to the origin (stable). A single \(|\lambda_i| > 1\) makes orbits diverge.
- \(\lambda < 0\): that component flips to the other side of the origin every step, so the orbit zigzags.
- Complex eigenvalues \(\lambda = re^{\pm i\varphi}\): each step turns by \(\varphi\) and scales by \(r\), so the orbit is a spiral: inward for \(r<1\), outward for \(r>1\), around an ellipse for \(r=1\).
Click the canvas below to drop new seed points (up to 40). Drag the step slider or press "Play" to watch each seed's orbit unfold. The small plot on the right puts the eigenvalues in the complex plane; the unit circle is the border between stable and unstable.
- 用实特征值的矩阵(比如默认的那个),让步数 ≥ 12,所有种子排成同一条线。这条线是哪条线?With a real-eigenvalue matrix (like the default one), take at least 12 steps and watch all seeds line up along one line. Which line is it?
- 做出向内的螺旋:复特征值且 \(|\lambda|<0.95\),步数 ≥ 10。Make an inward spiral: complex eigenvalues with \(|\lambda|<0.95\), at least 10 steps.
- 再做出向外的螺旋:复特征值且 \(|\lambda|>1.05\)。这时特征值在复平面里落在单位圆的哪一侧?Now make an outward spiral: complex eigenvalues with \(|\lambda|>1.05\). On which side of the unit circle do the eigenvalues sit?
“不断乘 \(A\),再归一化”就是幂迭代。它只能找到最大的那一个特征值,但足够便宜,大型稀疏矩阵常用它。通用的特征值求解器(NumPy 的 np.linalg.eig、Eigen 的 EigenSolver,背后是 LAPACK)用的是 QR 算法,可以看成同时对所有方向做幂迭代的加强版。
"Keep multiplying by \(A\) and normalize" is power iteration. It finds only the largest eigenvalue, but it is cheap enough that large sparse problems use it. General-purpose solvers (NumPy's np.linalg.eig, Eigen's EigenSolver, both backed by LAPACK) use the QR algorithm, which can be seen as a souped-up power iteration running on all directions at once.
5第三种视角:对称矩阵是特例A third view: symmetric matrices are special
满足 \(A = A^\top\)(关于对角线对称)的矩阵,在机器人里随处可见:协方差矩阵、惯量矩阵、Hessian(3.5 节),还有 1.12 节要用的 \(A^\top A\)。它们有两条很好的性质:
- 特征值一定是实数。
- 不同特征值对应的特征向量一定互相垂直。
所以它们总能写成 \(A = Q\Lambda Q^\top\):先用 \(Q^\top\) 把特征向量转到坐标轴上,再沿坐标轴拉伸(\(\Lambda\) 是对角矩阵,对角线上是特征值),最后用 \(Q\) 转回来。这里的 \(Q\) 是 1.10 节的正交矩阵,它的列就是特征向量。单位圆变成椭圆,椭圆的两根轴恰好就是特征向量,半轴长就是 \(|\lambda_i|\)。
左图是一个对称矩阵:拖动阶段滑块,看圆被转、拉、转回的过程;拖动椭圆轴端点的圆圈可以改变特征值和方向。右图是同一个矩阵加上一点“旋转成分”:\(N = S + tJ\),其中 \(J=\begin{bmatrix}0&1\\-1&0\end{bmatrix}\)。\(N\) 不再对称,看看椭圆的轴还是不是特征向量。
Matrices with \(A = A^\top\) (mirror-symmetric across the diagonal) are everywhere in robotics: covariance matrices, inertia matrices, Hessians (Lesson 3.5), and the \(A^\top A\) that Lesson 1.12 will use. They come with two wonderful guarantees:
- The eigenvalues are always real.
- Eigenvectors for different eigenvalues are always perpendicular.
So they can always be written \(A = Q\Lambda Q^\top\): first \(Q^\top\) turns the eigenvectors onto the coordinate axes, then a stretch along the axes (\(\Lambda\) is diagonal, with the eigenvalues on its diagonal), then \(Q\) turns everything back. Here \(Q\) is an orthogonal matrix from Lesson 1.10 whose columns are the eigenvectors. The unit circle becomes an ellipse whose two axes are exactly the eigenvectors, with semi-axis lengths \(|\lambda_i|\).
The left plot is a symmetric matrix: scrub the stage slider to watch the circle rotate, stretch and rotate back; drag the rings at the ends of the ellipse axes to change the eigenvalues and direction. The right plot is the same matrix plus a bit of "rotation": \(N = S + tJ\) with \(J=\begin{bmatrix}0&1\\-1&0\end{bmatrix}\). \(N\) is no longer symmetric; see whether the ellipse's axes are still its eigenvectors.
- 把阶段滑块从 ① 拖到 ③(或按“播放”),看对称矩阵怎样“转、拉、转回”。Scrub the stage slider from ① to ③ (or press "Play") and watch a symmetric matrix rotate, stretch and rotate back.
- 增大 \(t\),让右图两个特征向量的夹角偏离 90° 至少 10°,而特征值仍是实数。Increase \(t\) until the two eigenvectors on the right deviate from 90° by at least 10° while the eigenvalues are still real.
- 继续增大 \(t\),直到右图的特征向量消失。这时 \(t\) 和特征值之差满足什么关系?Keep increasing \(t\) until the right-hand eigenvectors vanish. How do \(t\) and the eigenvalue gap compare at that moment?
对称矩阵:特征向量就是椭圆的轴,特征值就是(带符号的)半轴长。这就是为什么协方差矩阵的特征向量给出不确定性椭圆的方向(4.4 节),也是点云 PCA 的原理。非对称矩阵的椭圆轴一般不是特征向量,那该怎么办?1.12 节的办法是:造一个对称矩阵 \(A^\top A\),用它的特征分解找到椭圆的轴,这就是奇异值分解。
For a symmetric matrix the eigenvectors are the ellipse's axes and the eigenvalues are the (signed) semi-axis lengths. That is why the eigenvectors of a covariance matrix give the direction of an uncertainty ellipse (Lesson 4.4), and why PCA of a point cloud works. For a non-symmetric matrix the ellipse's axes are generally not eigenvectors. What then? Lesson 1.12's answer: build the symmetric matrix \(A^\top A\) and use its eigen-decomposition to find the ellipse's axes. That is the singular value decomposition.
6再看数学The math
提示:把鼠标悬停在公式中带底色的符号上,FIG 2 会加粗对应的箭头,悬停 \(\lambda\) 时还会临时显示真正的特征线。Tip: hover the highlighted symbols in the formulas; FIG 2 thickens the matching arrow, and hovering \(\lambda\) briefly shows the true eigen-lines.
定义Definition
设 \(A\) 是 \(n\times n\) 矩阵。如果一个非零向量 \(\mathbf v\) 和一个数 \(\lambda\) 满足下式,\(\lambda\) 就叫 \(A\) 的特征值,\(\mathbf v\) 叫对应的特征向量:
Let \(A\) be an \(n\times n\) matrix. If a nonzero vector \(\mathbf v\) and a number \(\lambda\) satisfy the equation below, \(\lambda\) is an eigenvalue of \(A\) and \(\mathbf v\) is a matching eigenvector:
要求 \(\mathbf v\neq\mathbf 0\) 是因为零向量对任何 \(\lambda\) 都成立,没有信息。注意特征向量的任何非零倍数还是特征向量,所以真正确定的是一条特征线(所有这些倍数组成的直线)。同一个特征值的所有特征向量,加上零向量,组成一个子空间,叫特征空间。
We insist on \(\mathbf v\neq\mathbf 0\) because the zero vector satisfies the equation for every \(\lambda\) and carries no information. Any nonzero multiple of an eigenvector is again an eigenvector, so what is really determined is an eigen-line (all those multiples). All eigenvectors of one eigenvalue, together with the zero vector, form a subspace called the eigenspace.
怎么找:特征多项式How to find them: the characteristic polynomial
把 \(\lambda\mathbf v\) 移到左边:\(A\mathbf v - \lambda\mathbf v = (A-\lambda I)\mathbf v = \mathbf 0\)。这是一个齐次方程组。1.6 节告诉我们,它有非零解,当且仅当 \(A-\lambda I\) 不可逆(零空间不只有零向量);1.5 节告诉我们,不可逆等价于行列式为 0:
Move \(\lambda\mathbf v\) to the left: \(A\mathbf v - \lambda\mathbf v = (A-\lambda I)\mathbf v = \mathbf 0\). That is a homogeneous system. Lesson 1.6 says it has a nonzero solution exactly when \(A-\lambda I\) is not invertible (its null space is more than the zero vector), and Lesson 1.5 says not invertible means the determinant is 0:
对 \(2\times 2\) 矩阵 \(A=\begin{bmatrix}a&b\\c&d\end{bmatrix}\),展开就是一个关于 \(\lambda\) 的二次方程,叫特征多项式 (characteristic polynomial):
For a \(2\times 2\) matrix \(A=\begin{bmatrix}a&b\\c&d\end{bmatrix}\) this expands to a quadratic in \(\lambda\), the characteristic polynomial:
由根与系数的关系,\(\lambda_1+\lambda_2 = \operatorname{tr}A\),\(\lambda_1\lambda_2 = \det A\):迹等于特征值之和,行列式等于特征值之积。后一条很直观:1.5 节说行列式是面积的缩放倍数,而特征方向上各自伸缩 \(\lambda_1,\lambda_2\) 倍,面积自然缩放 \(\lambda_1\lambda_2\) 倍。它还给出一个心算技巧:先取平均 \(m=\operatorname{tr}A/2\),再取乘积 \(p=\det A\),特征值就是 \(m\pm\sqrt{m^2-p}\)。
By Vieta's formulas \(\lambda_1+\lambda_2 = \operatorname{tr}A\) and \(\lambda_1\lambda_2 = \det A\): the trace is the sum of the eigenvalues and the determinant is their product. The second one is intuitive: Lesson 1.5 says the determinant is the area scale factor, and along the eigen-directions things scale by \(\lambda_1\) and \(\lambda_2\), so area scales by \(\lambda_1\lambda_2\). It also gives a mental trick: take the mean \(m=\operatorname{tr}A/2\) and the product \(p=\det A\); the eigenvalues are \(m\pm\sqrt{m^2-p}\).
算一个例子A worked example
取 FIG 2 里的“对称”预设 \(A=\begin{bmatrix}2&1\\1&2\end{bmatrix}\)。迹为 4,行列式为 3:
Take the "Symmetric" preset from FIG 2, \(A=\begin{bmatrix}2&1\\1&2\end{bmatrix}\). The trace is 4 and the determinant is 3:
验证:\(A\mathbf v_1 = (3,3) = 3\mathbf v_1\),\(A\mathbf v_2 = (1,-1) = 1\cdot\mathbf v_2\)。两条特征线在 45° 和 135° 上,正是你在 FIG 2 里用探针找到的位置,而且互相垂直(这是对称矩阵的特点)。
Check: \(A\mathbf v_1 = (3,3) = 3\mathbf v_1\) and \(A\mathbf v_2 = (1,-1) = 1\cdot\mathbf v_2\). The two eigen-lines sit at 45° and 135°, exactly where you found them with the probe in FIG 2, and they are perpendicular (a property of symmetric matrices).
判别式的三种情况The three cases of the discriminant
根号里的 \(\Delta=(\operatorname{tr}A/2)^2-\det A\) 决定了矩阵长什么样:
The number under the square root, \(\Delta=(\operatorname{tr}A/2)^2-\det A\), decides what the matrix looks like:
| \(\Delta\) | 特征值Eigenvalues | 特征线Eigen-lines | 例子Example |
|---|---|---|---|
| \(\Delta>0\) | 两个不同的实数two distinct real numbers | 两条two | 非均匀缩放、对称(非数量矩阵)、镜像、投影non-uniform scale, symmetric (non-scalar), reflection, projection |
| \(\Delta=0\) | 一个重根one repeated root | 两种可能:\(A=\lambda I\)(每个方向都是);或只有一条(缺陷矩阵)two possibilities: \(A=\lambda I\) (every direction), or just one (a defective matrix) | 单位矩阵;剪切identity; shear |
| \(\Delta<0\) | 一对共轭复数 \(a\pm bi\)a complex pair \(a\pm bi\) | 没有实特征线no real eigen-line | 旋转rotation |
对角化与矩阵的幂Diagonalization and powers of a matrix
如果 \(A\) 有 \(n\) 个线性无关的特征向量,把它们并排作为 \(V\) 的列,\(\Lambda\) 是特征值组成的对角矩阵。逐列写出 \(A\mathbf v_i=\lambda_i\mathbf v_i\) 就是 \(AV=V\Lambda\),而 \(V\) 的列是一组基(1.2 节),所以 \(V\) 可逆:
Suppose \(A\) has \(n\) linearly independent eigenvectors. Put them side by side as the columns of \(V\), and let \(\Lambda\) be the diagonal matrix of eigenvalues. Writing \(A\mathbf v_i=\lambda_i\mathbf v_i\) column by column is \(AV=V\Lambda\), and the columns of \(V\) form a basis (Lesson 1.2), so \(V\) is invertible:
用 1.3 节的被动视角来读:\(V^{-1}\) 把标准坐标换算成“特征坐标”,\(\Lambda\) 在每个特征坐标上各自乘一个数,\(V\) 再换算回来。连乘 \(k\) 次时,中间的 \(V^{-1}V\) 全部抵消,只剩 \(\Lambda^k\),这就是第 4 部分里 \(\lambda_i^k\) 的来历。对上面的例子:
Read it with the passive view of Lesson 1.3: \(V^{-1}\) converts standard coordinates into "eigen-coordinates", \(\Lambda\) multiplies each eigen-coordinate by its own number, and \(V\) converts back. Applying \(A\) \(k\) times, all the inner \(V^{-1}V\) cancel and only \(\Lambda^k\) remains: that is where the \(\lambda_i^k\) of Section 4 comes from. For the example above:
取 \(k=1\) 得回 \(A\) 本身;\(k\) 很大时,两行几乎相同,每个向量都被推向 \((1,1)\) 方向:幂迭代的结论。
At \(k=1\) this returns \(A\) itself; for large \(k\) the entries are nearly equal and every vector is pushed toward the \((1,1)\) direction: the conclusion of power iteration.
复特征值就是“旋转加缩放”Complex eigenvalues are "rotate and scale"
当 \(\Delta<0\),特征值是 \(\lambda=a\pm bi\),其中 \(a=\operatorname{tr}A/2\),\(b=\sqrt{-\Delta}\)。写成极坐标形式 \(\lambda=re^{\pm i\varphi}\),模长 \(r=\sqrt{a^2+b^2}=\sqrt{\det A}\),辐角 \(\varphi=\operatorname{atan2}(b,a)\)。矩阵 \(\begin{bmatrix}a&-b\\b&a\end{bmatrix}=r\,R(\varphi)\) 恰好是“旋转 \(\varphi\),再放大 \(r\) 倍”。任何特征值为 \(a\pm bi\) 的实 \(2\times2\) 矩阵,本质上都是这个动作,只是透过一个歪斜的坐标系去看。所以反复作用的轨迹是螺旋:每步转 \(\varphi\),长度乘 \(r\)。
纯旋转矩阵 \(R(\theta)\) 的特征值是 \(e^{\pm i\theta}\),\(r=1\):旋转不改变长度,只有 \(\theta=0\) 或 \(\pi\) 时才有实特征向量(\(\theta=\pi\) 时 \(\lambda=-1\),每个向量都被翻到对面)。
When \(\Delta<0\) the eigenvalues are \(\lambda=a\pm bi\) with \(a=\operatorname{tr}A/2\) and \(b=\sqrt{-\Delta}\). In polar form \(\lambda=re^{\pm i\varphi}\), with modulus \(r=\sqrt{a^2+b^2}=\sqrt{\det A}\) and angle \(\varphi=\operatorname{atan2}(b,a)\). The matrix \(\begin{bmatrix}a&-b\\b&a\end{bmatrix}=r\,R(\varphi)\) is exactly "rotate by \(\varphi\), then scale by \(r\)". Any real \(2\times2\) matrix with eigenvalues \(a\pm bi\) is essentially this action, seen through a skewed coordinate frame. So repeated application traces a spiral: each step turns by \(\varphi\) and multiplies the length by \(r\).
A pure rotation \(R(\theta)\) has eigenvalues \(e^{\pm i\theta}\) with \(r=1\): rotations preserve length, and only \(\theta=0\) or \(\pi\) leaves a real eigenvector (at \(\theta=\pi\), \(\lambda=-1\): every vector is flipped to the opposite side).
对称矩阵:谱定理Symmetric matrices: the spectral theorem
如果 \(A=A^\top\),那么特征值全是实数,并且可以选出一组标准正交的特征向量。把它们作为列组成 \(Q\),\(Q^\top Q=I\)(1.10 节),于是 \(V^{-1}=Q^\top\),不用求逆:
If \(A=A^\top\), all eigenvalues are real and we can choose orthonormal eigenvectors. Use them as the columns of \(Q\); then \(Q^\top Q=I\) (Lesson 1.10), so \(V^{-1}=Q^\top\) and no inversion is needed:
垂直性的证明只要两行:设 \(A\mathbf v_1=\lambda_1\mathbf v_1\),\(A\mathbf v_2=\lambda_2\mathbf v_2\),\(\lambda_1\neq\lambda_2\)。则 \(\lambda_1(\mathbf v_1\!\cdot\!\mathbf v_2)=(A\mathbf v_1)\!\cdot\!\mathbf v_2=\mathbf v_1\!\cdot\!(A^\top\mathbf v_2)=\mathbf v_1\!\cdot\!(A\mathbf v_2)=\lambda_2(\mathbf v_1\!\cdot\!\mathbf v_2)\),所以 \((\lambda_1-\lambda_2)(\mathbf v_1\!\cdot\!\mathbf v_2)=0\),即 \(\mathbf v_1\perp\mathbf v_2\)。(点积见 1.7 节;这里用到 \((A\mathbf x)\!\cdot\!\mathbf y=\mathbf x\!\cdot\!(A^\top\mathbf y)\)。)1.12 节会对 \(A^\top A\) 使用这个定理,来推出奇异值分解。
The proof of perpendicularity takes two lines. Let \(A\mathbf v_1=\lambda_1\mathbf v_1\), \(A\mathbf v_2=\lambda_2\mathbf v_2\) with \(\lambda_1\neq\lambda_2\). Then \(\lambda_1(\mathbf v_1\!\cdot\!\mathbf v_2)=(A\mathbf v_1)\!\cdot\!\mathbf v_2=\mathbf v_1\!\cdot\!(A^\top\mathbf v_2)=\mathbf v_1\!\cdot\!(A\mathbf v_2)=\lambda_2(\mathbf v_1\!\cdot\!\mathbf v_2)\), so \((\lambda_1-\lambda_2)(\mathbf v_1\!\cdot\!\mathbf v_2)=0\), i.e. \(\mathbf v_1\perp\mathbf v_2\). (The dot product is Lesson 1.7; we used \((A\mathbf x)\!\cdot\!\mathbf y=\mathbf x\!\cdot\!(A^\top\mathbf y)\).) Lesson 1.12 applies this theorem to \(A^\top A\) to derive the SVD.
稳定性判据The stability criterion
第 4 部分的公式 \(\mathbf x_k=\sum c_i\lambda_i^k\mathbf v_i\) 直接给出结论。对离散时间系统 \(\mathbf x_{k+1}=A\mathbf x_k\),所有分量都衰减,当且仅当每个 \(|\lambda_i|<1\)。对连续时间系统 \(\dot{\mathbf x}=A\mathbf x\),解是 \(\mathbf x(t)=\sum c_i e^{\lambda_i t}\mathbf v_i\),每个分量以 \(e^{\operatorname{Re}\lambda_i\,t}\) 的速度变化,所以条件变成 \(\operatorname{Re}\lambda_i<0\):
The formula \(\mathbf x_k=\sum c_i\lambda_i^k\mathbf v_i\) of Section 4 gives the verdict directly. For a discrete-time system \(\mathbf x_{k+1}=A\mathbf x_k\), every component decays exactly when each \(|\lambda_i|<1\). For a continuous-time system \(\dot{\mathbf x}=A\mathbf x\) the solution is \(\mathbf x(t)=\sum c_i e^{\lambda_i t}\mathbf v_i\), each component changes like \(e^{\operatorname{Re}\lambda_i\,t}\), and the condition becomes \(\operatorname{Re}\lambda_i<0\):
NumPy 里,一般矩阵用 np.linalg.eig(A)(可能返回复数),对称矩阵用 np.linalg.eigh(S):它保证特征值是实数并按升序排列,特征向量标准正交,也更快、更准。Eigen 里对应的是 EigenSolver 和 SelfAdjointEigenSolver。如果你知道矩阵是对称的(协方差、\(A^\top A\)),一定用 eigh。
In NumPy use np.linalg.eig(A) for a general matrix (it may return complex numbers) and np.linalg.eigh(S) for a symmetric one: it guarantees real eigenvalues in ascending order and orthonormal eigenvectors, and is faster and more accurate. The Eigen equivalents are EigenSolver and SelfAdjointEigenSolver. If you know the matrix is symmetric (a covariance, \(A^\top A\)), always use eigh.
7机器人中的它:控制器的稳定性与旋转轴In robotics: controller stability and the rotation axis
离散时间的 PD 控制A discrete-time PD controller
一辆单位质量的小车,位置 \(p\),速度 \(v\),用 PD 控制把它拉回原点:推力 \(u=-k_p\,p-k_d\,v\)(弹簧 \(k_p\) 加阻尼 \(k_d\))。控制器每隔 \(\Delta t\) 秒运行一次,我们用半隐式欧拉法离散化:先更新速度,再用新速度更新位置:
A unit-mass cart with position \(p\) and velocity \(v\) is pulled back to the origin by PD control: force \(u=-k_p\,p-k_d\,v\) (a spring \(k_p\) plus a damper \(k_d\)). The controller runs once every \(\Delta t\) seconds, and we discretize with semi-implicit Euler: update the velocity first, then update the position with the new velocity:
以 \(\mathbf x=(p,v)\) 为状态,这是 \(\mathbf x_{k+1}=(A-BK)\,\mathbf x_k\),其中 \(A\) 是无控制时的动力学,\(BK\) 是反馈。注意:判断稳定性要看闭环矩阵 \(A-BK\) 的特征值,而不是 \(A\) 的(这个 \(A\) 的特征值是 1 和 1,本身就不稳定):
With state \(\mathbf x=(p,v)\) this is \(\mathbf x_{k+1}=(A-BK)\,\mathbf x_k\), where \(A\) is the uncontrolled dynamics and \(BK\) is the feedback. Note: stability is read from the eigenvalues of the closed-loop matrix \(A-BK\), not of \(A\) (this \(A\) has eigenvalues 1 and 1 and is not stable on its own):
它的迹和行列式有很简洁的形式:\(\operatorname{tr}=2-\Delta t^2k_p-\Delta t\,k_d\),\(\det=1-\Delta t\,k_d\)。代入第 6 部分的公式就得到两个特征值,只要两个都落在单位圆里,小车就会稳定下来。注意:如果时间是连续的,任何正增益 \(k_p,k_d\) 都稳定;离散以后,增益和 \(\Delta t\) 一大,特征值就会冲出单位圆。拖动滑块,把特征值(右上小图)、相平面轨迹和时间响应对着看。
Its trace and determinant have a tidy closed form: \(\operatorname{tr}=2-\Delta t^2k_p-\Delta t\,k_d\) and \(\det=1-\Delta t\,k_d\). Plug them into the formula of Section 6 for the two eigenvalues; as long as both land inside the unit circle, the cart settles. Note the catch: in continuous time every positive gain \(k_p,k_d\) is stable, but after discretization large gains or a large \(\Delta t\) push an eigenvalue out of the unit circle. Drag the sliders and watch the eigenvalues (top-right plot), the phase-plane path and the time response together.
- 找到一组增益,让小车不超调地稳定下来:两个特征值都是 \((0,1)\) 内的实数。提示:阻尼要够,但 \(k_p\) 别太大。Find gains that make the cart settle without overshoot: both eigenvalues real and in \((0,1)\). Hint: enough damping, but keep \(k_p\) modest.
- 让它振荡但仍然稳定:特征值是复数,而且 \(|\lambda|<0.98\)。Make it oscillate but still settle: complex eigenvalues with \(|\lambda|<0.98\).
- 把增益调大,让系统发散(\(|\lambda|>1\))。然后保持增益不变,把 \(\Delta t\) 调小,看它能不能被救回来。Crank the gains until the system diverges (\(|\lambda|>1\)). Then keep the gains and shrink \(\Delta t\): can you rescue it?
机器人的低层关节控制环通常跑在 1 kHz 左右,原因正是这里看到的:控制周期 \(\Delta t\) 越长,同样的增益越容易让特征值越过单位圆。调增益时不要只看超调,也要看闭环特征值离单位圆还有多远:靠得太近,对模型误差和通信延迟就没有余量。连续时间的判据 \(\operatorname{Re}\lambda<0\) 与离散时间的 \(|\lambda|<1\) 通过 \(z=e^{s\Delta t}\) 相连:左半平面被映射到单位圆内部。
Low-level joint control loops on robots typically run around 1 kHz, for exactly the reason seen here: the longer the control period \(\Delta t\), the easier it is for the same gains to push an eigenvalue past the unit circle. When tuning gains, do not only look at overshoot; look at how far the closed-loop eigenvalues stay from the unit circle. Too close, and there is no margin for model error or communication delay. The continuous-time test \(\operatorname{Re}\lambda<0\) and the discrete-time test \(|\lambda|<1\) are linked by \(z=e^{s\Delta t}\): the left half-plane maps into the inside of the unit circle.
三维旋转的轴:特征值为 1 的特征向量The axis of a 3D rotation: the eigenvector with eigenvalue 1
把绕 z、y、x 轴的三个旋转连乘(1.10 节),得到一个看上去很复杂的 \(3\times3\) 旋转矩阵 \(R\)。欧拉旋转定理说:它仍然只是绕某一条固定轴转一个角度。这条轴上的向量不受影响,\(R\mathbf n=\mathbf n\),也就是说 \(\mathbf n\) 是 \(R\) 的特征向量,特征值 \(\lambda=1\)。由 \((R-I)\mathbf n=\mathbf 0\),\(\mathbf n\) 垂直于 \(R-I\) 的每一行,所以取两行的叉积(1.8 节)就能找到它。另外两个特征值是复数 \(e^{\pm i\theta}\),\(\theta\) 就是转角(\(\operatorname{tr}R=1+2\cos\theta\))。拖动三个欧拉角,看那条紫色的不动轴。
Multiply three rotations about z, y and x (Lesson 1.10) and you get a complicated-looking \(3\times3\) rotation matrix \(R\). Euler's rotation theorem says it is still just one rotation by one angle about one fixed axis. Vectors on that axis are untouched, \(R\mathbf n=\mathbf n\): \(\mathbf n\) is an eigenvector of \(R\) with eigenvalue \(\lambda=1\). Since \((R-I)\mathbf n=\mathbf 0\), \(\mathbf n\) is perpendicular to every row of \(R-I\), so the cross product (Lesson 1.8) of two rows finds it. The other two eigenvalues are the complex pair \(e^{\pm i\theta}\), where \(\theta\) is the rotation angle (\(\operatorname{tr}R=1+2\cos\theta\)). Drag the three Euler angles and watch the purple fixed axis.
- 把三个欧拉角都设成非零(至少 15°),确认合成的旋转仍然只有一条紫色轴,且读数里 \(R\mathbf n=\mathbf n\)。Set all three Euler angles nonzero (at least 15°) and confirm the composite is still one purple axis, with \(R\mathbf n=\mathbf n\) in the readout.
- 让旋转轴恰好是竖直的 z 轴(转角大于 20°)。可以把哪两个角设为 0 来做到?Make the axis exactly the vertical z-axis (rotation angle above 20°). Which two angles can you set to zero to get it?
协方差椭圆。SLAM 和卡尔曼滤波给出的位置协方差 \(\Sigma\) 是对称矩阵(4.4 节)。\(\Sigma=Q\Lambda Q^\top\):\(Q\) 的列是椭圆的轴方向,\(\sqrt{\lambda_i}\) 是 1σ 椭圆的半轴长(椭圆是单位圆在 \(Q\Lambda^{1/2}Q^\top\) 下的像)。最大特征值对应机器人最不确定的方向。FIG 4 左图就是这幅图,只是半轴长是 \(\lambda_i\) 而不是 \(\sqrt{\lambda_i}\)。
旋转轴。从旋转矩阵求出轴和转角(FIG 6 做的事),是 5.2 节“轴角表示”和 5.6 节对数映射的出发点:\(\operatorname{Log}(R)=\theta\,\mathbf n\)。
Covariance ellipses. The position covariance \(\Sigma\) from SLAM or a Kalman filter is symmetric (Lesson 4.4). \(\Sigma=Q\Lambda Q^\top\): the columns of \(Q\) are the ellipse's axis directions and \(\sqrt{\lambda_i}\) are the semi-axes of the 1σ ellipse (the ellipse is the image of the unit circle under \(Q\Lambda^{1/2}Q^\top\)). The largest eigenvalue marks the direction the robot is least sure about. The left plot of FIG 4 is this picture, except that its semi-axes are \(\lambda_i\) rather than \(\sqrt{\lambda_i}\).
Rotation axes. Extracting the axis and angle from a rotation matrix (what FIG 6 does) is the starting point of the axis–angle representation in Lesson 5.2 and the logarithm map of Lesson 5.6: \(\operatorname{Log}(R)=\theta\,\mathbf n\).
8常见误区Common pitfalls
只有特征线是确定的:\(\mathbf v\) 的任何非零倍数(包括 \(-\mathbf v\))一样是特征向量。库函数返回的通常是单位长度,但正负号是任意的,不同机器、不同版本可能不同。特征值重复时,特征空间是整个平面或子空间,任何一组基都行。所以别逐元素比较两次运行的特征向量;要比较方向,用 \(|\mathbf u\cdot\mathbf v|\) 接近 1,或者先约定一个符号规则(比如让最大分量为正)。
Only the eigen-line is determined: any nonzero multiple of \(\mathbf v\) (including \(-\mathbf v\)) is an eigenvector too. Libraries usually return unit length, but the sign is arbitrary and can differ between machines and versions. With a repeated eigenvalue the eigenspace is a whole plane or subspace and any basis of it will do. So do not compare eigenvectors entry by entry across runs; to compare directions, check that \(|\mathbf u\cdot\mathbf v|\) is close to 1, or fix a sign convention first (say, make the largest entry positive).
只有对称矩阵才保证这一点(FIG 4 右图)。一般矩阵的特征向量可以夹成任意角度,这时 \(V^{-1}\neq V^\top\),别图省事用转置。特征向量越“斜”,矩阵越“非正规”:即使所有 \(|\lambda_i|<1\),状态也可能先暂时增长很大,再衰减。
Only symmetric matrices guarantee it (FIG 4, right). The eigenvectors of a general matrix can sit at any angle, and then \(V^{-1}\neq V^\top\), so do not take the transpose to save effort. The more slanted the eigenvectors, the more "non-normal" the matrix: even with every \(|\lambda_i|<1\), the state can first grow a lot for a while before it decays.
不是。剪切 \(\begin{bmatrix}1&1\\0&1\end{bmatrix}\) 的特征值 1 是重根,却只有 x 轴这一条特征线:特征向量不够,凑不出基,叫缺陷矩阵 (defective)。为什么?\(A-I=\begin{bmatrix}0&1\\0&0\end{bmatrix}\) 的零空间只有 x 轴。这类矩阵的幂不再是 \(\lambda^k\) 那么简单:\(A^k=\begin{bmatrix}1&k\\0&1\end{bmatrix}\),虽然 \(|\lambda|=1\),状态却线性增长。在稳定性分析里,\(|\lambda|=1\) 的缺陷矩阵属于不稳定。
No. The shear \(\begin{bmatrix}1&1\\0&1\end{bmatrix}\) has the repeated eigenvalue 1 but only one eigen-line, the x-axis: too few eigenvectors to form a basis. Such a matrix is defective. Why? \(A-I=\begin{bmatrix}0&1\\0&0\end{bmatrix}\) has only the x-axis as its null space. Powers of such matrices are no longer as simple as \(\lambda^k\): \(A^k=\begin{bmatrix}1&k\\0&1\end{bmatrix}\), so although \(|\lambda|=1\) the state grows linearly. In stability analysis, a defective matrix with \(|\lambda|=1\) counts as unstable.
特征值只描述特征方向上的伸缩。剪切的特征值是 1 和 1,似乎什么都没拉伸,可它实际上能把某些向量拉到 1.62 倍(它的奇异值是 1.62 和 0.62)。想知道“最多能拉多长、最少压成多短”,要用奇异值(1.12 节)。只有对称(更一般地,正规)矩阵的奇异值才等于特征值的绝对值。
Eigenvalues describe scaling along the eigen-directions only. The shear has eigenvalues 1 and 1, apparently stretching nothing, yet it stretches some vectors by a factor of 1.62 (its singular values are 1.62 and 0.62). To ask "how much can it stretch at most, and squash at least", use singular values (Lesson 1.12). Only for symmetric (more generally, normal) matrices are the singular values equal to the absolute values of the eigenvalues.
连续时间 \(\dot{\mathbf x}=A\mathbf x\):稳定当且仅当 \(\operatorname{Re}\lambda<0\)。离散时间 \(\mathbf x_{k+1}=A\mathbf x_k\):稳定当且仅当 \(|\lambda|<1\)。把它们搞混会得到荒唐的结论:\(\lambda=0.5\) 在离散系统里稳定,在连续系统里却是指数发散;\(\lambda=-2\) 在连续系统里稳定,在离散系统里却发散。先问清楚:你手里的 \(A\) 是“每步更新”,还是“变化率”?
Continuous time \(\dot{\mathbf x}=A\mathbf x\): stable exactly when \(\operatorname{Re}\lambda<0\). Discrete time \(\mathbf x_{k+1}=A\mathbf x_k\): stable exactly when \(|\lambda|<1\). Mixing them up yields nonsense: \(\lambda=0.5\) is stable in a discrete system but diverges exponentially in a continuous one; \(\lambda=-2\) is stable in continuous time but diverges in discrete time. First ask: is the \(A\) in your hand an "update per step" or a "rate of change"?
手算 \(2\times2\) 很方便,但写进程序是个坏主意。五阶及以上的多项式没有求根公式(阿贝尔–鲁菲尼定理),多项式的根对系数的微小扰动极其敏感,数值上会损失大量精度。实际软件用 QR 迭代等直接作用在矩阵上的算法(eig、eigh),不会显式构造特征多项式。
It is handy by hand for \(2\times2\) but a bad idea in a program. Polynomials of degree five and up have no closed-form root formula (Abel–Ruffini), and polynomial roots are extremely sensitive to tiny changes in the coefficients, so a lot of accuracy is lost numerically. Real software uses algorithms such as QR iteration that act directly on the matrix (eig, eigh) and never form the characteristic polynomial.
9小练习Check yourself
10延伸阅读Further reading
- 3Blue1Brown, Essence of Linear Algebra — 第 14 章“特征向量与特征值”,以及第 15 章“计算特征值的一个小技巧”(本课用到的迹与行列式心算法)Ch. 14 "Eigenvectors and eigenvalues" and Ch. 15 "A quick trick for computing eigenvalues" (the trace-and-determinant shortcut used here)
- G. Strang, Introduction to Linear Algebra, Ch. 6 — 特征值与特征向量(对角化、对称矩阵、稳定性)Eigenvalues and eigenvectors (diagonalization, symmetric matrices, stability)
- K. J. Åström & R. M. Murray, Feedback Systems, Ch. 5–6 — 线性系统与状态反馈:特征值如何决定稳定性与响应Linear systems and state feedback: how eigenvalues determine stability and response
- J. Ström et al., Immersive Linear Algebra — 特征值与特征向量一章,带交互插图The chapter on eigenvalues and eigenvectors, with interactive illustrations
- G. Strang, MIT 18.06 Linear Algebra — 第 21–22 讲:特征值、对角化与矩阵的幂Lectures 21–22: eigenvalues, diagonalization and powers of a matrix