Part III · 第 11 章多视图几何 · 11.3Part III · Ch. 11 Multi-View Geometry · 11.3

三角化Triangulation

相机已标定，同一个点在两张（或更多）图里的像素也匹配上了，那么这个点在三维空间的哪里？这一步叫三角化，它把特征匹配变成三维路标，是双目测距、SfM 和视觉 SLAM 的共同核心。难处在于：像素有噪声，两条视线几乎永远不相交，所以"交点"必须重新定义；而答案有多准，几乎完全取决于相机之间隔了多远。本节把三种估计方法（中点、线性 DLT、最小化重投影误差）放到同一组数据上比较，再用不确定性椭球看清深度误差为什么与 Z² 成正比，为什么直行的机器人看不准前方，以及 SLAM 为什么要等"足够的视差"再初始化路标。 The cameras are calibrated and the same point has been matched in two or more images: where is it in 3D? That step is triangulation, which turns feature matches into 3D landmarks and sits at the core of stereo ranging, SfM and visual SLAM. The difficulty: pixels are noisy, so the two viewing rays almost never meet and "intersection" has to be redefined, and how accurate the answer is depends almost entirely on how far apart the cameras are. This lesson runs three estimators (midpoint, linear DLT, reprojection-optimal) on the same data, then uses the uncertainty ellipsoid to show why depth error scales as Z², why a robot driving straight cannot see what is ahead, and why SLAM waits for "enough parallax" before it initializes a landmark.

约 35 分钟~35 min 前置：1.12 SVD · 1.13 最小二乘 · 4.5 不确定性传播 · 6.9 非线性最小二乘 · 10.3 内参与外参 · 11.2 立体视觉Prereqs: 1.12 SVD · 1.13 Least Squares · 4.5 Uncertainty Propagation · 6.9 Nonlinear Least Squares · 10.3 Intrinsics & Extrinsics · 11.2 Stereo 5 个交互5 interactives

1为什么机器人需要它Why robots need this

一张图只告诉你一个三维点在哪个方向上；深度在投影时丢掉了（10.1）。要把像素变回三维，至少要从两个不同的位置看同一个点：每个视角给出一条穿过光心和像素的射线，点就在射线的交会处。这就是三角化（triangulation），机器人里到处在用它：

双目 / 多目深度：每个匹配像素的深度。11.2 的 $Z=fB/d$ 就是共面（校正过的）双目的特例。
SfM 与视觉 SLAM：把关键帧之间的特征匹配变成三维路标，供 PnP（11.4）定位、供光束法平差（11.6）联合优化。
多相机系统：动捕的反光点、AprilTag 的角点、遥操作采集数据时的人手 / 人体关键点，都是先在每台相机里检测二维点，再三角化成三维。

但真实的射线从来不相交：像素有噪声，空间中两条直线通常是异面的。于是出现三个问题：没有交点时"交点"指什么？这个答案可信到什么程度？相机该怎么放才好？本节依次回答。

An image tells you only the direction of a 3D point; depth is lost in the projection (10.1). To turn a pixel back into 3D you must look at the same point from at least two different places: each view gives a ray through the camera center and the pixel, and the point sits where the rays meet. That is triangulation, and robots use it everywhere:

Stereo / multi-camera depth: the depth of every matched pixel. Lesson 11.2's $Z=fB/d$ is the special case of a rectified pair.
SfM and visual SLAM: turning feature matches between keyframes into 3D landmarks, which PnP (11.4) localizes against and bundle adjustment (11.6) refines jointly.
Multi-camera systems: mocap markers, AprilTag corners, and the hand / body keypoints collected during teleoperation are detected in 2D in every camera and then triangulated to 3D.

But real rays never intersect: pixels are noisy and two lines in space are generically skew. Three questions follow. With no intersection, what does "the intersection" mean? How far can the answer be trusted? Where should the cameras be placed? This lesson answers them in turn.

2直觉：两条几乎相交的射线Intuition: two rays that (almost) meet

从相机 $i$ 的光心 $\mathbf o_i$ 出发，穿过观测到的像素，得到射线 $\mathbf o_i + s\,\mathbf d_i$（$s>0$，$\mathbf d_i$ 为单位方向；像素 → 方向见 10.3）。没有噪声时，两条射线恰好交于真实点 $\mathbf X$。一旦像素偏了几个像素，两条射线就擦肩而过。我们取两条射线最短的连线（公垂线）的中点作为估计：

From the center $\mathbf o_i$ of camera $i$, pass through the observed pixel to get the ray $\mathbf o_i + s\,\mathbf d_i$ ($s>0$, $\mathbf d_i$ a unit direction; pixel → direction is in 10.3). Without noise the two rays meet exactly at the true point $\mathbf X$. Once the pixels are off by a few pixels, the rays pass each other. We take the midpoint of their shortest connecting segment (the common perpendicular) as the estimate:

$$ \hat{\mathbf X} \;=\; \hl{mid}{\tfrac12\big(\mathbf o_1 + s^{*}\mathbf d_1 + \mathbf o_2 + t^{*}\mathbf d_2\big)},\qquad (s^{*},t^{*}) = \arg\min_{s,t}\ \big\|\mathbf o_1 + s\mathbf d_1 - \mathbf o_2 - t\mathbf d_2\big\|,\qquad \hl{gap}{d_{\min}} = \big\|\mathbf o_1 + s^{*}\mathbf d_1 - \mathbf o_2 - t^{*}\mathbf d_2\big\|. $$

下图左边是俯视图（可以拖动两台相机 $C_1,C_2$ 和真实点 $X$），右边是同一场景的三维视图（可以旋转，也可以切到"放大"看公垂线）。相机始终朝向场景中心。噪声被刻意放大了（默认 6 像素，真实角点检测通常在 1 像素以内），否则公垂线短得看不见。两条射线夹的角 $\alpha$ 叫三角化角（视差角）。

滑块的第二个功能是模拟一次误匹配：把相机 2 里的像素沿（或垂直于）对极线平移。猜一猜：哪一种误匹配更容易被发现？

The left panel is a top view (drag the two cameras $C_1,C_2$ and the true point $X$); the right is the same scene in 3D (orbit it, or switch to "zoom" to see the common perpendicular). The cameras always aim at the scene center. The noise is deliberately exaggerated (6 px by default; real corner detectors are usually within 1 px), otherwise the perpendicular would be too short to see. The angle $\alpha$ between the two rays is the triangulation angle (parallax angle).

The second slider simulates a wrong match: it slides the pixel in camera 2 along (or across) the epipolar line. Guess which kind of mismatch is easier to catch.

FIG 1两条几乎相交的射线Two rays that (almost) meet

俯视图 (x–y)TOP VIEW (x–y)

三维（拖动旋转）3D (drag to orbit)

青色：两台相机及其视线；紫色：真实点 $X$；琥珀色：公垂线与中点 $\hat X$。"沿对极线"移动相机 2 的像素等价于让点沿相机 1 的射线滑动，所以两条射线仍然相交，只是交在错误的位置；"垂直于对极线"移动则让射线变成异面。一致性检验：$\hat X$ 在两张图里的重投影误差合起来不超过 $3\sigma$。 Cyan: the two cameras and their rays; purple: the true point $X$; amber: the common perpendicular and the midpoint $\hat X$. Moving the camera-2 pixel "along the epipolar line" is the same as sliding the point along camera 1's ray, so the rays still intersect, just at the wrong place; moving it "across" makes them skew. Consistency test: the reprojection errors of $\hat X$ in the two images together stay within $3\sigma$.

试一试Try this

把像素噪声调到 0：公垂线缩成一个点（$d_{\min}<1$ mm），射线恰好相交。这是它们唯一相交的时候。Set the pixel noise to 0: the perpendicular shrinks to a point ($d_{\min}<1$ mm) and the rays intersect exactly. That is the only time they do.
把两台相机拖近，使 $\alpha<10^\circ$，并让估计误差 $\|\hat X-X\|$ 超过 0.5 m。同样的像素噪声，为什么误差变大了？Drag the cameras together until $\alpha<10^\circ$ and the error $\|\hat X-X\|$ exceeds 0.5 m. With the same pixel noise, why did the error grow?
选"沿对极线"，把误匹配平移至少 40 像素，使一致性检验仍然通过、而点已偏离真值 0.4 m 以上。Choose "along the line" and shift the mismatch by at least 40 px so that the consistency test still passes while the point is more than 0.4 m off.
选"垂直于对极线"，平移至少 40 像素，让检验失败。Choose "across the line" and shift by at least 40 px to make the test fail.

核心洞见Key insight

三角化 = 求一组本来不相交的射线的"交点"。所以每个算法都是对"最接近"的一种选择（什么度量？在哪个空间？），再加上一份说明：这个答案有多不确定。沿对极线的误匹配会给出一个完美自洽却错误的三维点，单靠两个视角的几何检验抓不到它，需要第三个视角或先验。

Triangulation means finding the "intersection" of rays that do not intersect. Every algorithm is therefore a choice of what "closest" means (which metric, in which space), plus an account of how uncertain the answer is. A mismatch along the epipolar line produces a perfectly consistent but wrong 3D point that two-view geometry cannot catch; you need a third view or a prior.

3动手玩：三种估计与更多视角Play: three estimators and more views

"最接近"有不止一种答案。本节在同一批带噪声的像素上运行三个真正的算法（它们都在这个页面里实时运行）：

中点：使三维空间中到各条射线的距离平方和最小（$N=2$ 时就是上一节的公垂线中点）。
线性 DLT：把每个视角写成两个线性方程，堆成 $A\tilde{\mathbf X}=0$，取 SVD 最小奇异值的右奇异向量（1.12）。
最优（重投影）：最小化 $\sum_i\|\mathbf x_i-\pi(P_i\mathbf X)\|^2$，即像素空间中的平方误差，用 DLT 作初值做 Gauss–Newton（6.6）。

左图是几何（相机沿一条路径排开，第一个最近、最后一个最远），右图把真值附近放大：小点是 150 次独立噪声实验各自的结果，大点是第 1 次实验。右侧表格给出 150 次实验的统计。

"Closest" has more than one answer. Here three real algorithms (all running live in this page) are applied to the same noisy pixels:

Midpoint: minimizes the sum of squared 3D distances to the rays (for $N=2$ it is the midpoint of the previous section).
Linear DLT: write each view as two linear equations, stack them into $A\tilde{\mathbf X}=0$, take the right singular vector of the smallest singular value (1.12).
Optimal (reprojection): minimize $\sum_i\|\mathbf x_i-\pi(P_i\mathbf X)\|^2$, the squared error in pixel space, by Gauss–Newton (6.6) started from the DLT.

The left panel shows the geometry (cameras spread along a path, the first one nearest and the last farthest); the right panel magnifies the neighborhood of the truth: small dots are the results of 150 independent noise draws, the large dots are draw 1. The table summarizes the 150 draws.

FIG 2三种估计器Three estimators

几何（俯视）GEOMETRY (top view)

真值附近放大MAGNIFIED AROUND THE TRUTH

紫色十字：真值；紫色虚线椭圆：理论协方差（Cramér–Rao 界）的 95% 椭圆（见第 5 节）。"重投影 RMS"是每个视角的像素残差的均方根；最优估计的期望值是 $\sigma\sqrt{(2N-3)/N}$（$2N$ 个测量，3 个自由度）。读数里的最后一项：把世界原点平移 3 m 后重新做一次 DLT，结果移动了多少（真正的几何方法不应该移动）。 Purple cross: the truth; purple dashed ellipse: the 95% ellipse of the theoretical covariance (the Cramér–Rao bound, Section 5). "Reproj RMS" is the root-mean-square pixel residual per view; for the optimum its expected value is $\sigma\sqrt{(2N-3)/N}$ ($2N$ measurements, 3 degrees of freedom). The last readout item: re-run the DLT after moving the world origin by 3 m and see how far the answer moves (a truly geometric method should not move).

用 $N=2$ 并把最远相机拉远（"远端距离"约 8 m 以上），使 DLT 的重投影 RMS 达到最优估计的 2 倍以上，但两者的三维误差只差几个百分点。With $N=2$ and the far camera pulled out (far distance 8 m or more), make the DLT's reprojection RMS at least 2× the optimum's, while their 3D errors differ by only a few percent.

三个观察。（1）中点和 DLT 几乎重合：DLT 每个方程都等于"点到射线的距离"乘一个常数，它其实是中点法的代数版本（第 5 节推导）。（2）最优估计的三维误差只比它们小几个到几十个百分点，但重投影误差却小得多，因为前两种方法"不在乎"像素：近处相机的 1 像素对应的米数少，远处相机的多，而它们对两者一视同仁。（3）视角越多越好，误差大致按 $1/\sqrt N$ 下降，并且最优估计逼近理论界。

Three observations. (1) The midpoint and the DLT nearly coincide: each DLT equation is a point-to-ray distance times a constant, so the DLT is an algebraic version of the midpoint (derived in Section 5). (2) The optimum's 3D error is only a few to a few tens of percent lower, but its reprojection error is much lower, because the first two do not care about pixels: one pixel at the near camera is fewer meters than one pixel at the far camera, and they treat both alike. (3) More views help, with the error falling roughly like $1/\sqrt N$, and the optimum approaches the theoretical bound.

一个问题，三种看法One idea, three views

几何的：找离所有射线最近的点（中点）。代数的：找 $A\tilde{\mathbf X}=0$ 的近似零空间向量（DLT，1.12）。统计的：假设像素噪声是高斯的，则最大似然估计就是最小化重投影误差（4.6、6.9），并且噪声传播给出答案的协方差（4.5）。三者在无噪声时给出同一个点；有噪声时它们的区别，正是"在哪个空间里度量误差"。

Geometric: find the point closest to all rays (midpoint). Algebraic: find the approximate null vector of $A\tilde{\mathbf X}=0$ (DLT, 1.12). Statistical: if pixel noise is Gaussian, the maximum-likelihood estimate minimizes the reprojection error (4.6, 6.9), and noise propagation gives the covariance of the answer (4.5). All three return the same point without noise; with noise, the difference is exactly "in which space the error is measured".

4第二个视角：不确定性椭球Second view: the uncertainty ellipsoid

只看一次估计的好坏意义不大。更有用的是问：同样的相机、同样的噪声，重复很多次，估计点会散成什么形状？答案是一个细长的椭球：它沿着观测方向很长，垂直于观测方向很短。原因很直观：一个像素误差只让射线在横向偏几毫米（$\approx Z\sigma/f$），而两条射线夹角很小时，横向的小偏移会让交点沿射线方向滑出很远。

下图用蒙特卡洛实验（500 次，种子固定）画出这个云团，并叠加由重投影雅可比算出的理论协方差 $\Sigma_X=\sigma^2(\sum_i J_i^\top J_i)^{-1}$（推导见第 5 节，用到 4.5 与 6.9）。两台相机平行朝前（+y 方向），点在前方 $Z$ 处。

Judging a single estimate is not very informative. It is more useful to ask: with the same cameras and the same noise, repeated many times, what shape do the estimates scatter into? The answer is a long thin ellipsoid, long along the viewing direction and short across it. The reason is intuitive: one pixel of error shifts a ray sideways by only a few millimeters ($\approx Z\sigma/f$), but when the two rays subtend a small angle, a small sideways shift slides their intersection far along the ray.

The figure draws that cloud by Monte Carlo (500 draws, fixed seed) and overlays the theoretical covariance $\Sigma_X=\sigma^2(\sum_i J_i^\top J_i)^{-1}$ computed from the reprojection Jacobians (derived in Section 5, using 4.5 and 6.9). Both cameras look straight ahead (the +y direction), and the point is $Z$ in front.

FIG 3不确定性椭球实验室Uncertainty-ellipsoid lab

真实比例的场景（俯视）SCENE TO SCALE (top view)

估计点云（放大，等比例）ESTIMATE CLOUD (magnified, equal scale)

琥珀色小点：500 次噪声实验的三角化结果（x–y 投影，每次用 DLT 初始化再做 Gauss–Newton）；青色实线椭圆：理论协方差的 x–y 边缘 95% 椭圆；蓝色虚线：样本协方差的同一椭圆。左图虚线方框是右图的取景范围。基线沿"运动方向"放置：横向时两台相机左右并排，向前时一台在另一台正后方。 Small amber dots: triangulation results of 500 noise draws (x–y projection; each from the DLT followed by Gauss–Newton). Solid cyan ellipse: the 95% x–y marginal ellipse of the theoretical covariance; dashed blue: the same ellipse from the sample covariance. The dashed box in the left panel is the right panel's field of view. The baseline lies along the "motion direction": side by side for lateral, one camera directly behind the other for forward.

点"设为参考"，然后把基线 $B$ 加倍（其他不变）：沿视线方向的误差 $\sigma_1$ 变成原来的多少？Press "Set reference", then double the baseline $B$ (everything else unchanged): what fraction of the old range error $\sigma_1$ is left?
重新设参考，把深度 $Z$ 加倍：$\sigma_1$ 变成原来的多少倍？Set a new reference and double the depth $Z$: by what factor does $\sigma_1$ grow?
把运动方向换成"向前"：同样的 $B$，$\sigma_1$ 比横向时大 5 倍以上。这就是直行的机器人看不准正前方的原因。Switch the motion direction to "forward": for the same $B$, $\sigma_1$ is more than 5× that of lateral motion. This is why a robot driving straight cannot see well straight ahead.

注意读数里两处对照：理论值与蒙特卡洛值在横向运动时吻合到百分之几（样本只有 500 个，本身就有约 3% 的抽样误差）；而在"向前"或很小基线时，云团变成弯曲的、有长尾的"香蕉"，线性化的协方差低估了真实的散布，而且估计还带有偏差。这提醒我们：协方差只是一阶近似。

Compare the two columns in the readout: for lateral motion the theoretical and Monte-Carlo values agree to a few percent (500 samples carry about 3% sampling error themselves); for "forward" motion or a tiny baseline the cloud turns into a curved, long-tailed "banana", the linearized covariance underestimates the true scatter, and the estimates are biased. The covariance is only a first-order approximation.

同一个椭球，三句话One ellipsoid, three sentences

（1）深度精度：$\Delta Z\approx Z^2\Delta d/(fB)$，基线加倍误差减半，深度加倍误差变四倍（11.2 的结果，这里是它的一般形式）。（2）椭球的长宽比约为 $2/\alpha$（$\alpha$ 为三角化角，弧度）：夹角 6.8° 时约 17:1。（3）向前运动：$\alpha\to0$，椭球沿视线无限拉长；点正好在光轴上时深度完全不可观测。

(1) Depth precision: $\Delta Z\approx Z^2\Delta d/(fB)$: double the baseline and the error halves, double the depth and it quadruples (the result of 11.2, here in its general form). (2) The aspect ratio of the ellipsoid is about $2/\alpha$ ($\alpha$ the triangulation angle in radians): about 17:1 at 6.8°. (3) Forward motion: $\alpha\to0$ and the ellipsoid stretches without bound along the line of sight; a point exactly on the optical axis has no observable depth at all.

5再看数学The math

提示：把鼠标悬停在公式中带底色的符号上，FIG 1、3、4、5 中对应的元素会被高亮。Tip: hover the highlighted symbols in the formulas; the matching element in FIGs 1, 3, 4 and 5 lights up.

每个视角给出什么What each view gives you

相机 $i$ 的外参 ${}^{c}T_{w}=(R_i,\mathbf t_i)$（世界 → 相机，OpenCV 的 $[R|t]$），投影矩阵 $P_i=K_i[R_i\,|\,\mathbf t_i]$（10.3）。点 $\tilde{\mathbf X}=(\mathbf X,1)$ 的像素满足 $\lambda_i\tilde{\mathbf x}_i=P_i\tilde{\mathbf X}$，其中 $\lambda_i$ 恰是点在相机 $i$ 里的深度 $z_i$。反过来，光心是 $\mathbf o_i=-R_i^\top\mathbf t_i$（相机位姿 ${}^{w}T_{c}$ 的平移），像素 $\mathbf x_i$ 的射线方向是 $\mathbf d_i\propto R_i^\top K_i^{-1}\tilde{\mathbf x}_i$。

Camera $i$ has extrinsics ${}^{c}T_{w}=(R_i,\mathbf t_i)$ (world → camera, OpenCV's $[R|t]$) and projection matrix $P_i=K_i[R_i\,|\,\mathbf t_i]$ (10.3). The pixel of a point $\tilde{\mathbf X}=(\mathbf X,1)$ satisfies $\lambda_i\tilde{\mathbf x}_i=P_i\tilde{\mathbf X}$, where $\lambda_i$ is exactly the depth $z_i$ of the point in camera $i$. Conversely, the optical center is $\mathbf o_i=-R_i^\top\mathbf t_i$ (the translation of the camera pose ${}^{w}T_{c}$) and the ray direction of pixel $\mathbf x_i$ is $\mathbf d_i\propto R_i^\top K_i^{-1}\tilde{\mathbf x}_i$.

中点：二次函数的极小The midpoint: minimizing a quadratic

令 $\mathbf w=\mathbf o_1-\mathbf o_2$，$b=\mathbf d_1\!\cdot\!\mathbf d_2=\cos\hl{alpha}{\alpha}$，$c_1=\mathbf d_1\!\cdot\!\mathbf w$，$c_2=\mathbf d_2\!\cdot\!\mathbf w$。对 $\|\mathbf w+s\mathbf d_1-t\mathbf d_2\|^2$ 分别对 $s,t$ 求导并令其为零，得到 $s-bt=-c_1$，$-bs+t=c_2$，因此

Let $\mathbf w=\mathbf o_1-\mathbf o_2$, $b=\mathbf d_1\!\cdot\!\mathbf d_2=\cos\hl{alpha}{\alpha}$, $c_1=\mathbf d_1\!\cdot\!\mathbf w$, $c_2=\mathbf d_2\!\cdot\!\mathbf w$. Setting the derivatives of $\|\mathbf w+s\mathbf d_1-t\mathbf d_2\|^2$ in $s$ and $t$ to zero gives $s-bt=-c_1$, $-bs+t=c_2$, hence

$$ s^{*}=\frac{b\,c_2-c_1}{1-b^{2}},\qquad t^{*}=\frac{c_2-b\,c_1}{1-b^{2}},\qquad 1-b^2=\sin^2\alpha,\qquad d_{\min}=\frac{|\mathbf w\cdot(\mathbf d_1\times\mathbf d_2)|}{\|\mathbf d_1\times\mathbf d_2\|}. $$

分母 $\sin^2\alpha$：射线平行（$\alpha\to0$）时公式失效，这是第一次遇到"小基线灾难"。$N$ 条射线的推广是 $\hat{\mathbf X}=\big(\sum_i(I-\mathbf d_i\mathbf d_i^\top)\big)^{-1}\sum_i(I-\mathbf d_i\mathbf d_i^\top)\mathbf o_i$，其中 $I-\mathbf d_i\mathbf d_i^\top$ 把向量投影到与射线垂直的平面；它是线性最小二乘（1.13）。注意它最小化的是米，不是像素。

The denominator is $\sin^2\alpha$: the formula breaks down for parallel rays ($\alpha\to0$), our first encounter with the "small-baseline disaster". For $N$ rays the generalization is $\hat{\mathbf X}=\big(\sum_i(I-\mathbf d_i\mathbf d_i^\top)\big)^{-1}\sum_i(I-\mathbf d_i\mathbf d_i^\top)\mathbf o_i$, where $I-\mathbf d_i\mathbf d_i^\top$ projects onto the plane perpendicular to the ray; it is linear least squares (1.13). Note that it minimizes meters, not pixels.

线性 DLT 与代数误差Linear DLT and the algebraic error

$\lambda\tilde{\mathbf x}=P\tilde{\mathbf X}$ 里未知的 $\lambda$ 可以用叉积消去：$\tilde{\mathbf x}\times(P\tilde{\mathbf X})=\mathbf 0$。记 $\mathbf p_k^{\top}$ 为 $P$ 的第 $k$ 行，其中两个独立方程是 $(u\,\mathbf p_3^\top-\mathbf p_1^\top)\tilde{\mathbf X}=0$ 和 $(v\,\mathbf p_3^\top-\mathbf p_2^\top)\tilde{\mathbf X}=0$（第三个是它们的线性组合）。$N$ 个视角堆成 $2N\times4$ 的矩阵：

The unknown $\lambda$ in $\lambda\tilde{\mathbf x}=P\tilde{\mathbf X}$ is eliminated with a cross product: $\tilde{\mathbf x}\times(P\tilde{\mathbf X})=\mathbf 0$. With $\mathbf p_k^{\top}$ the $k$-th row of $P$, two independent equations are $(u\,\mathbf p_3^\top-\mathbf p_1^\top)\tilde{\mathbf X}=0$ and $(v\,\mathbf p_3^\top-\mathbf p_2^\top)\tilde{\mathbf X}=0$ (the third is a combination of them). $N$ views stack into a $2N\times4$ matrix:

$$ A\tilde{\mathbf X}=\begin{bmatrix} u_1\,\mathbf p_3^{(1)\top}-\mathbf p_1^{(1)\top}\\ v_1\,\mathbf p_3^{(1)\top}-\mathbf p_2^{(1)\top}\\ \vdots\\ u_N\,\mathbf p_3^{(N)\top}-\mathbf p_1^{(N)\top}\\ v_N\,\mathbf p_3^{(N)\top}-\mathbf p_2^{(N)\top}\end{bmatrix}\tilde{\mathbf X}=\mathbf 0,\qquad \tilde{\mathbf X}^{*}=\arg\min_{\|\tilde{\mathbf X}\|=1}\|A\tilde{\mathbf X}\|=\mathbf v_{4},\qquad \mathbf X=\tfrac{\tilde X_{1:3}}{\tilde X_4}. $$

$\mathbf v_4$ 是 $A$ 的 SVD（1.12）中最小奇异值对应的右奇异向量。看看它在最小化什么：每一行等于 $(\mathbf p_3^\top\tilde{\mathbf X})(u-\hat u)=z_i\,(u_i-\hat u_i)$，即深度乘像素残差。它与点到射线的距离也有关系：这两行恰是叉积 $\mathbf X_c\times\mathbf m$ 的两个分量（再乘以焦距；$\mathbf X_c$ 是点在相机坐标系里的坐标，$\mathbf m=K^{-1}\tilde{\mathbf x}$ 是射线方向），而这个叉积的模等于 $\|\mathbf m\|$ 乘以点到射线的距离，所以 DLT 是中点法的代数版本（FIG 2 里两者几乎重合）。因此 DLT 最小化的不是像素误差，它让远处的相机占更大的权重；并且条件 $\|\tilde{\mathbf X}\|=1$ 依赖于坐标系的选择，把世界原点平移一下答案就会变（FIG 2 的最后一项）。实现时应先对像素做归一化（用 $K^{-1}$ 变成归一化坐标）并把世界坐标平移缩放到原点附近。

$\mathbf v_4$ is the right singular vector of the smallest singular value in the SVD (1.12) of $A$. Look at what it minimizes: each row equals $(\mathbf p_3^\top\tilde{\mathbf X})(u-\hat u)=z_i\,(u_i-\hat u_i)$, the depth times the pixel residual. It is also tied to the point-to-ray distance: the two rows are two components of the cross product $\mathbf X_c\times\mathbf m$, times the focal length ($\mathbf X_c$ the point in camera coordinates, $\mathbf m=K^{-1}\tilde{\mathbf x}$ the ray direction), and that cross product has norm $\|\mathbf m\|$ times the point-to-ray distance, so the DLT is an algebraic version of the midpoint method (the two nearly coincide in FIG 2). So the DLT does not minimize pixel error; it weights far cameras more heavily; and the constraint $\|\tilde{\mathbf X}\|=1$ depends on the choice of coordinates, so moving the world origin changes the answer (the last item of FIG 2). In practice, normalize the pixels first (to normalized coordinates with $K^{-1}$) and translate and scale the world coordinates to be near the origin.

最优：最小化重投影误差The optimum: minimizing the reprojection error

$$ \hat{\mathbf X}=\arg\min_{\mathbf X}\sum_{i=1}^{N}\big\|\mathbf x_i-\pi(P_i\tilde{\mathbf X})\big\|^2,\qquad \Delta\mathbf X=\Big(\sum_i J_i^\top J_i\Big)^{-1}\sum_i J_i^\top\mathbf r_i,\qquad J_i=\frac{\partial\pi}{\partial\mathbf X}=\begin{bmatrix}\tfrac{f_x}{z} & 0 & -\tfrac{f_x x_c}{z^2}\\ 0 & \tfrac{f_y}{z} & -\tfrac{f_y y_c}{z^2}\end{bmatrix}R_i. $$

其中 $(x_c,y_c,z)=R_i\mathbf X+\mathbf t_i$ 是点在相机 $i$ 坐标系里的坐标，$\mathbf r_i=\mathbf x_i-\pi(\cdot)$ 是残差（3.3 的雅可比，6.6 的 Gauss–Newton）。像素噪声是独立同方差的高斯时，这就是最大似然估计（4.6）。问题只有 3 个未知数，用 DLT 作初值，3 到 5 次迭代就收敛；页面里的实现还加了步长折半以保证代价不上升。两个视角时还有闭式的全局最优解：Hartley–Sturm 把问题化为一个 6 次多项式求根；Lindstrom 给出更快的近似。

Here $(x_c,y_c,z)=R_i\mathbf X+\mathbf t_i$ is the point in camera $i$'s frame and $\mathbf r_i=\mathbf x_i-\pi(\cdot)$ the residual (the Jacobian of 3.3, Gauss–Newton of 6.6). With independent, equal-variance Gaussian pixel noise this is the maximum-likelihood estimate (4.6). The problem has only 3 unknowns; started from the DLT it converges in 3 to 5 iterations (the page's implementation also halves the step to keep the cost from rising). For two views there is even a closed-form global optimum: Hartley–Sturm reduces it to finding the roots of a degree-6 polynomial, and Lindstrom gives a faster approximation.

协方差与椭球的形状Covariance and the shape of the ellipsoid

把像素噪声 $\mathrm{cov}(\mathbf x_i)=\sigma^2I$ 通过 Gauss–Newton 的解传播（4.5）：线性化后 $\delta\mathbf X=(\sum J_i^\top J_i)^{-1}\sum J_i^\top\delta\mathbf x_i$，所以

Propagating the pixel noise $\mathrm{cov}(\mathbf x_i)=\sigma^2I$ through the Gauss–Newton solution (4.5): after linearization $\delta\mathbf X=(\sum J_i^\top J_i)^{-1}\sum J_i^\top\delta\mathbf x_i$, so

$$ \hl{cov}{\Sigma_{X}}=\sigma^2\Big(\sum_{i=1}^{N}J_i^\top J_i\Big)^{-1}. $$

这是 6.9 里"高斯–牛顿的 Hessian 的逆"，也是 Cramér–Rao 界，信息 $\sum J_i^\top J_i$ 在视角之间相加。每个 $J_i^\top J_i\approx(f/r_i)^2(I-\mathbf u_i\mathbf u_i^\top)$（$\mathbf u_i$ 是单位视线，$r_i$ 是点到相机的距离）：一个视角只在垂直于视线的两个方向上提供信息，沿视线的方向一点也没有。取两个等距（$r_1=r_2=r$）的相机，三角化角为 $\alpha$，三个主轴上的标准差为

This is the "inverse of the Gauss–Newton Hessian" of 6.9 and the Cramér–Rao bound; the information $\sum J_i^\top J_i$ adds across views. Each $J_i^\top J_i\approx(f/r_i)^2(I-\mathbf u_i\mathbf u_i^\top)$ ($\mathbf u_i$ the unit line of sight, $r_i$ the distance to the camera): a view informs only the two directions perpendicular to its line of sight and says nothing along it. For two equidistant cameras ($r_1=r_2=r$) with triangulation angle $\alpha$, the standard deviations on the three principal axes are

$$ \sigma_{\text{range}}\approx\frac{r\sigma}{\sqrt2\,f\,\sin(\alpha/2)},\qquad \sigma_{\perp}\approx\frac{r\sigma}{\sqrt2\,f\,\cos(\alpha/2)},\qquad \sigma_{\text{vert}}\approx\frac{r\sigma}{\sqrt2\,f},\qquad \frac{\sigma_{\text{range}}}{\sigma_\perp}=\cot\frac\alpha2\approx\frac2\alpha. $$

范围方向（沿角平分线）的信息只有 $2\sin^2(\alpha/2)\,(f/r)^2$，随 $\alpha$ 平方消失。对校正过的双目，$\sin(\alpha/2)\approx B/(2Z)$，代入得 $\sigma_Z=\sqrt2\,\sigma Z^2/(fB)$，恰是 11.2 里 $\Delta Z=Z^2\Delta d/(fB)$ 配上视差噪声 $\sigma_d=\sqrt2\sigma$（两张图各有独立噪声）。一般地，$\alpha\approx B\sin\gamma/r$，其中 $\gamma$ 是基线与视线的夹角：基线与视线平行（$\gamma=0$，向前运动）时没有视差。

The information along the range direction (the bisector) is only $2\sin^2(\alpha/2)\,(f/r)^2$, vanishing like $\alpha^2$. For a rectified pair $\sin(\alpha/2)\approx B/(2Z)$, which gives $\sigma_Z=\sqrt2\,\sigma Z^2/(fB)$: exactly Lesson 11.2's $\Delta Z=Z^2\Delta d/(fB)$ with disparity noise $\sigma_d=\sqrt2\sigma$ (independent noise in the two images). In general $\alpha\approx B\sin\gamma/r$ with $\gamma$ the angle between the baseline and the line of sight: when the baseline is parallel to the line of sight ($\gamma=0$, forward motion) there is no parallax.

逆深度Inverse depth

校正过的双目里 $Z=fB/d$，所以 $\rho=1/Z=d/(fB)$ 是视差的线性函数。视差噪声是高斯的，$\rho$ 就是高斯的，且 $\sigma_\rho=\sqrt2\sigma/(fB)$ 与深度无关；而 $Z=1/\rho$ 的分布是歪的、有长尾，当 $\rho$ 的误差棒跨过 0 时，$Z$ 甚至会变成负的或无穷大（FIG 5）。一般的单目情形把路标写成"锚点相机光心 $\mathbf c$ + 方向 $\mathbf m(\theta,\phi)$ + 逆深度 $\rho$"：$\mathbf X=\mathbf c+\mathbf m/\rho$。另一台相机 $j$ 看到它的像素是 $\pi\big(R_j(\rho(\mathbf c-\mathbf o_j)+\mathbf m)\big)$（投影对整体缩放不变，乘以 $\rho$），$\rho=0$ 时依然有定义：无穷远的点只约束相机的旋转（Civera–Davison–Montiel）。

For a rectified pair $Z=fB/d$, so $\rho=1/Z=d/(fB)$ is a linear function of disparity. Gaussian disparity noise makes $\rho$ Gaussian, with $\sigma_\rho=\sqrt2\sigma/(fB)$ independent of depth, whereas $Z=1/\rho$ is skewed and long-tailed, and when the error bar of $\rho$ crosses 0, $Z$ can even become negative or infinite (FIG 5). The general monocular case writes a landmark as "anchor camera center $\mathbf c$ + direction $\mathbf m(\theta,\phi)$ + inverse depth $\rho$": $\mathbf X=\mathbf c+\mathbf m/\rho$. Camera $j$ sees it at $\pi\big(R_j(\rho(\mathbf c-\mathbf o_j)+\mathbf m)\big)$ (projection is invariant to overall scale, so multiply by $\rho$), which remains well defined at $\rho=0$: a point at infinity still constrains the camera's rotation (Civera–Davison–Montiel).

6机器人中的它：SLAM 里的视差门槛In robotics: the parallax threshold in SLAM

单目 SLAM 的机器人在路上走，每个新特征第一次被看到时只有一条射线，深度未知。随着机器人移动，同一特征在后续帧里被再次观测，才有可能三角化。这时就要回答一个很实际的问题：什么时候可以把它当作路标放进地图？太早（视差不足），深度几乎是猜的，坏路标会拖垮位姿估计；太晚，又浪费了观测。

FIG 4 把这件事做成实验。机器人沿 x 轴直线行驶，距离首帧 $b$；路标在 8 m 远，相对行驶方向的方位角 $\gamma$ 可选："侧视"（90°）、"斜前方"（30°）、"正前方"（8°）。右图是路标沿视线方向的相对不确定性 $\sigma_r/r$（用首帧与当前帧的两视角协方差计算，对数坐标）与行驶距离的关系。常见的两种准入规则分别画成阈值：不确定性小于 $\varepsilon$，或者三角化角大于 $\alpha_{\min}$。

A monocular-SLAM robot driving along sees a new feature for the first time through a single ray with unknown depth. Only after it has moved and re-observed the feature can it be triangulated, and a very practical question follows: when may it enter the map as a landmark? Too early (too little parallax) and the depth is nearly a guess, and bad landmarks drag down the pose estimate; too late and observations are wasted.

FIG 4 turns that into an experiment. The robot drives straight along the x axis, a distance $b$ from the first frame; the landmark is 8 m away at a bearing $\gamma$ from the direction of travel: "side-looking" (90°), "oblique" (30°) or "straight ahead" (8°). The right panel plots the landmark's relative uncertainty along the line of sight, $\sigma_r/r$ (from the two-view covariance of the first and current frames, log scale) against the distance driven. The two usual admission rules appear as thresholds: uncertainty below $\varepsilon$, or triangulation angle above $\alpha_{\min}$.

FIG 4SLAM 里的视差门槛The parallax threshold in SLAM

场景（俯视，真实比例）SCENE (top view, to scale)

σᵣ/r（对数）随行驶距离σᵣ/r (log) vs. DISTANCE DRIVEN

左图：青色为首帧相机，琥珀色为当前相机，紫点是路标真值，琥珀色椭圆是两视角估计的 95% 椭圆（x–y 边缘，真实比例）。右图：三条曲线分别对应三个方位角（当前选中的最粗），横向虚线是 $\varepsilon$，竖向实线是当前的 $b$；琥珀 / 紫色三角分别标出满足"不确定性"和"视差角"规则所需的最小行驶距离。 Left: cyan is the first-frame camera, amber the current camera, the purple dot the true landmark, and the amber ellipse the 95% ellipse of the two-view estimate (x–y marginal, to scale). Right: the three curves are the three bearings (the selected one is thickest), the horizontal dashed line is $\varepsilon$ and the vertical solid line the current $b$; amber / purple triangles mark the minimum distance needed to satisfy the "uncertainty" and "parallax angle" rules.

先在"侧视"下找到满足不确定性规则（$\sigma_r/r<\varepsilon$）的最小 $b$；再选"正前方"，把 $b$ 拖到同样满足规则。需要多行驶几倍？Under "side-looking" find the smallest $b$ that satisfies the uncertainty rule ($\sigma_r/r<\varepsilon$); then choose "straight ahead" and drag $b$ until it satisfies the rule too. How many times farther must the robot drive?

读图的要点：$\sigma_r/r\approx\sqrt2\,(\sigma/f)\,r/(b\sin\gamma)$，与 $b\sin\gamma$（垂直于视线的位移，即有效基线）成反比。侧视时 $b$ 就是有效基线；正前方时 $\sin\gamma$ 很小，必须多走约七倍（$1/\sin8^\circ$）才能达到同样的精度。这正是视觉 SLAM 在"直行靠近一面墙"时尺度和深度容易漂的原因。

下面的 FIG 5 再看另一件事：就算已经过了门槛，深度的分布也不是高斯的。

How to read it: $\sigma_r/r\approx\sqrt2\,(\sigma/f)\,r/(b\sin\gamma)$, inversely proportional to $b\sin\gamma$, the displacement perpendicular to the line of sight (the effective baseline). Side-looking, $b$ itself is the effective baseline; straight ahead $\sin\gamma$ is tiny and the robot must drive about seven times farther ($1/\sin8^\circ$) to reach the same precision. This is why visual SLAM tends to drift in scale and depth when driving straight toward a wall.

FIG 5 below looks at something else: even past the threshold, the depth distribution is not Gaussian.

FIG 5深度 Z 与逆深度 ρ 的分布The distribution of depth Z and inverse depth ρ

深度 Z（米）的直方图HISTOGRAM OF DEPTH Z (m)

逆深度 ρ = 1/Z（1/米）的直方图HISTOGRAM OF INVERSE DEPTH ρ = 1/Z (1/m)

1200 次独立噪声实验，每次对一对平行双目相机做真正的 DLT 三角化。青色曲线是高斯分布：左图是"在 Z 里线性化"的高斯 $\mathcal N(Z,\ \sigma_\rho Z^2)$，右图是 $\mathcal N(\rho,\ \sigma_\rho)$；虚线标出真值。直方图是对全部样本归一化的密度；左图只画 $0$ 到 $2.5Z$ 的范围，范围外的样本不显示，所以它的面积小于 1，缺的部分就是长尾。 1200 independent noise draws, each a real DLT triangulation of a parallel stereo pair. The cyan curves are Gaussians: on the left the "linearized in Z" Gaussian $\mathcal N(Z,\ \sigma_\rho Z^2)$, on the right $\mathcal N(\rho,\ \sigma_\rho)$; dashed lines mark the truth. Histograms are densities normalized over all samples; the left one shows only $0$ to $2.5Z$, so samples outside that range are not drawn and its area falls short of 1 by exactly the long tail.

拖动深度或基线，让 $Z$ 的偏度超过 2，而 $\rho$ 的偏度仍在 $\pm0.3$ 以内。Move the depth or the baseline until the skewness of $Z$ exceeds 2 while that of $\rho$ stays within $\pm0.3$.

真实系统里In real systems

ORB-SLAM 系列：新地图点由共视关键帧之间的匹配三角化，并且拒绝视差角过小（量级约 1°）、重投影误差过大或深度为负的点；VINS-Mono 等视觉惯性系统在滑窗里用逆深度表示特征，并按平均视差挑选关键帧；DSO 等直接法同样使用逆深度。COLMAP（SfM）在增量重建时对每个轨迹做多视图三角化，有最小三角化角（默认约 1.5°）和最大重投影误差的过滤，并用 RANSAC 抵抗误匹配。双目 / RGB-D 与立体相机：校正后每个像素就是一个共面三角化，即 $Z=fB/d$。动捕系统（如 Vicon、OptiTrack）把标定好的多台相机看到的反光球三角化成三维点；多相机 AprilTag 与人体 / 手部关键点（遥操作数据采集）则是在每台相机里检测二维点，再加权三角化。（具体阈值与默认值随版本不同，使用前请查阅文档。）

The ORB-SLAM family: new map points are triangulated from matches between co-visible keyframes, rejecting points with too small a parallax angle (on the order of 1°), a large reprojection error or negative depth. VINS-Mono and similar visual-inertial systems represent features by inverse depth in a sliding window and pick keyframes by average parallax; direct methods such as DSO also use inverse depth. COLMAP (SfM) triangulates every track from multiple views during incremental reconstruction, filtering by a minimum triangulation angle (about 1.5° by default) and a maximum reprojection error, and uses RANSAC against mismatches. Stereo / RGB-D cameras: after rectification each pixel is a rectified triangulation, $Z=fB/d$. Motion-capture systems (e.g. Vicon, OptiTrack) triangulate retro-reflective markers seen by several calibrated cameras; multi-camera AprilTag setups and body / hand keypoints for teleoperation data collection detect 2D points in every camera and then triangulate them with weights. (Exact thresholds and defaults vary between versions; check the documentation before relying on them.)

7常见误区Common pitfalls

误区 1：基线太小也照样三角化Pitfall 1: triangulating with a tiny baseline anyway

代码不会报错：DLT 总会给你一个点，只不过它的深度误差可能是几十米。机器人原地转动（纯旋转）时基线为 0，根本没有深度信息，此时两帧之间的关系是单应（10.5），不是对极几何。工程上要检查三角化角（或基线 / 深度比），并把"深度未定"的点保留为只有方向的特征。FIG 3 的"向前"挡直接展示了这一点。

The code will not complain: the DLT always returns a point, just with a depth error that may be tens of meters. When the robot rotates in place (pure rotation) the baseline is zero and there is no depth information at all; the relation between the two frames is a homography (10.5), not epipolar geometry. In practice, check the triangulation angle (or the baseline-to-depth ratio) and keep "depth-unknown" points as direction-only features. The "forward" setting of FIG 3 shows exactly this.

误区 2：把远处的深度误差当作对称高斯Pitfall 2: treating long-range depth error as a symmetric Gaussian

"Z = 20 ± 5 m"听起来对称，真实分布却是歪的：偏近的一侧很窄，偏远的一侧拖着长尾，甚至包含无穷远（FIG 5）。把 $Z$ 当作卡尔曼滤波的状态，线性化会在视差小的时候失效；用逆深度，噪声的分布就接近高斯，滤波和优化都稳得多。

"Z = 20 ± 5 m" sounds symmetric, but the true distribution is skewed: narrow on the near side, with a long tail on the far side that even includes infinity (FIG 5). If you put $Z$ in a Kalman filter's state, the linearization fails at small parallax; with inverse depth the noise is close to Gaussian and both filtering and optimization behave far better.

误区 3：以为"几何自洽"就是"对了"Pitfall 3: believing "geometrically consistent" means "correct"

好的三角化管线要做三个检验：（a）手性（cheirality）：点在每台相机前面（$z_i>0$，并且射线参数 $s,t>0$），否则射线是发散的；（b）重投影误差在每张图里都小于阈值；（c）视差角足够大。但 FIG 1 说明，沿对极线的误匹配通过 (a)(b)，因为它恰好对应射线 1 上的另一个点；只有第三个视角、描述子距离或深度先验才能抓住它。因此 SfM 和 SLAM 才依赖多视图轨迹和 RANSAC，而不是两两检验。

A good triangulation pipeline runs three tests: (a) cheirality: the point is in front of every camera ($z_i>0$, and the ray parameters $s,t>0$), otherwise the rays diverge; (b) the reprojection error is below a threshold in every image; (c) the parallax angle is large enough. But FIG 1 shows that a mismatch along the epipolar line passes (a) and (b), because it corresponds to another point on ray 1; only a third view, the descriptor distance, or a depth prior can catch it. That is why SfM and SLAM rely on multi-view tracks and RANSAC, not on pairwise tests.

误区 4：用 DLT 的结果直接当"最优解"Pitfall 4: treating the DLT result as "the optimum"

DLT 最小化的是代数误差 $\|A\tilde{\mathbf X}\|$（深度 × 像素残差），既不是像素误差，也不具有射影不变性。三维误差也许只差几个百分点，但它的重投影残差明显更大，会误导后面的外点检验。DLT 只适合作为初值，之后做几步 Gauss–Newton。如果不归一化（像素除以 $K$，世界坐标靠近原点），数值条件也会变差。

The DLT minimizes the algebraic error $\|A\tilde{\mathbf X}\|$ (depth × pixel residual), which is neither the pixel error nor projectively invariant. Its 3D error may be only a few percent worse, but its reprojection residuals are noticeably larger and will mislead later outlier tests. Use the DLT only as an initial guess and follow it with a few Gauss–Newton steps. Without normalization (pixels through $K^{-1}$, world coordinates near the origin) the numerical conditioning also suffers.

误区 5：相机模型、单位或位姿方向搞错Pitfall 5: wrong camera model, units, or pose direction

最常见的 bug：把相机位姿 ${}^{w}T_{c}$ 当作外参 ${}^{c}T_{w}$ 塞进 $P=K[R|t]$（二者互逆）；用带畸变的像素直接三角化而忘了先去畸变（10.3）；基线用厘米、焦距用像素、深度用米混着算；左右图的顺序反了。每一种都会得到"看起来合理"的点云，只是位置或尺度错了。最可靠的自检：把得到的三维点用原相机重投影，看误差是否在噪声量级。

The most common bugs: feeding the camera pose ${}^{w}T_{c}$ as the extrinsics ${}^{c}T_{w}$ into $P=K[R|t]$ (they are inverses); triangulating distorted pixels without undistorting them first (10.3); mixing centimeters of baseline with pixels of focal length and meters of depth; swapping the left and right images. Each yields a "plausible" point cloud with the wrong position or scale. The most reliable self-check: reproject the 3D points with the original cameras and see whether the error is at the noise level.

8小练习Check yourself

1. 相机的投影矩阵第 $k$ 行为 $\mathbf p_k^\top$，观测像素是 $(u,v)$。它给 DLT 矩阵 $A$ 贡献的两行是？1. A camera's projection matrix has rows $\mathbf p_k^\top$ and the observed pixel is $(u,v)$. Which two rows does it contribute to the DLT matrix $A$?

$\lambda(u,v,1)^\top=P\tilde{\mathbf X}$ 给出 $u=\mathbf p_1^\top\tilde{\mathbf X}/\mathbf p_3^\top\tilde{\mathbf X}$ 与 $v=\mathbf p_2^\top\tilde{\mathbf X}/\mathbf p_3^\top\tilde{\mathbf X}$，交叉相乘得 $(u\mathbf p_3^\top-\mathbf p_1^\top)\tilde{\mathbf X}=0$ 和 $(v\mathbf p_3^\top-\mathbf p_2^\top)\tilde{\mathbf X}=0$。选项 B 把 $\mathbf p_3$ 与 $\mathbf p_1$ 的位置弄反了；C 混淆了齐次与非齐次；D 的那一行线性相关于前两行，一个视角实际上提供 2 个独立方程，3 个未知数要求至少两个视角。$\lambda(u,v,1)^\top=P\tilde{\mathbf X}$ gives $u=\mathbf p_1^\top\tilde{\mathbf X}/\mathbf p_3^\top\tilde{\mathbf X}$ and $v=\mathbf p_2^\top\tilde{\mathbf X}/\mathbf p_3^\top\tilde{\mathbf X}$; cross-multiplying gives $(u\mathbf p_3^\top-\mathbf p_1^\top)\tilde{\mathbf X}=0$ and $(v\mathbf p_3^\top-\mathbf p_2^\top)\tilde{\mathbf X}=0$. Option B swaps the roles of $\mathbf p_3$ and $\mathbf p_1$; C mixes homogeneous and inhomogeneous quantities; the row in D is linearly dependent on the first two, so a view really supplies 2 independent equations and three unknowns need at least two views.

2. 校正后的双目：$f=600$ 像素，基线 $B=0.12$ m，目标距离 $Z=3$ m，视差噪声 $0.5$ 像素。深度误差约为？2. A rectified stereo pair: $f=600$ px, baseline $B=0.12$ m, target at $Z=3$ m, disparity noise $0.5$ px. The depth error is about

$\Delta Z\approx Z^2\Delta d/(fB)=9\times0.5/(600\times0.12)=4.5/72=0.0625$ m。2.1 cm 是漏掉一个 $Z$ 的结果（$Z\Delta d/(fB)$）；0.7 cm 是忘了乘 $Z^2$；25 cm 则相当于基线只有 3 cm（小了四倍）。深度误差与 $Z^2$ 成正比，这就是 3 m 时厘米级、30 m 时就是几米的原因。$\Delta Z\approx Z^2\Delta d/(fB)=9\times0.5/(600\times0.12)=4.5/72=0.0625$ m. 2.1 cm results from dropping one factor of $Z$ ($Z\Delta d/(fB)$); 0.7 cm from forgetting $Z^2$ altogether; 25 cm would correspond to a baseline of only 3 cm (four times too small). Depth error grows like $Z^2$, which is why it is centimeters at 3 m and meters at 30 m.

3. DLT 三角化的"代数误差"$(u\,\mathbf p_3^\top-\mathbf p_1^\top)\tilde{\mathbf X}$ 等于？3. The DLT's "algebraic error" $(u\,\mathbf p_3^\top-\mathbf p_1^\top)\tilde{\mathbf X}$ equals

$u\,\mathbf p_3^\top\tilde{\mathbf X}-\mathbf p_1^\top\tilde{\mathbf X}=(\mathbf p_3^\top\tilde{\mathbf X})(u-\hat u)$，而 $\mathbf p_3^\top\tilde{\mathbf X}=z$。所以远处的相机残差被放大，DLT 与"所有像素等权"的最大似然不同。它与点到射线的距离只差一个近似常数因子（焦距与视场相关），所以选项 C 不精确；有噪声时 $\tilde{\mathbf X}$ 只在近似零空间里，残差不为零。$u\,\mathbf p_3^\top\tilde{\mathbf X}-\mathbf p_1^\top\tilde{\mathbf X}=(\mathbf p_3^\top\tilde{\mathbf X})(u-\hat u)$ and $\mathbf p_3^\top\tilde{\mathbf X}=z$. So far cameras' residuals are amplified, and the DLT differs from the maximum-likelihood estimate that weights all pixels equally. It is proportional to the point-to-ray distance only up to a roughly constant factor (related to focal length and field of view), so option C is not exact; with noise $\tilde{\mathbf X}$ lies only in the approximate null space and the residual is nonzero.

4. 机器人沿光轴直线前进，用前后两帧三角化正前方光轴上的一个点。结果是？4. A robot drives straight ahead along its optical axis and triangulates a point exactly on the axis from two successive frames. The result is

基线与视线平行，$\sin\gamma=0$，三角化角 $\alpha=0$：两帧里该点都落在主点，像素完全相同，没有视差，信息矩阵沿视线方向为零，深度的方差无穷大。基线长不等于视差大，有效基线是垂直于视线的那一部分 $b\sin\gamma$。稍微偏离光轴的点会有一点点视差，但很小（FIG 3、FIG 4）。The baseline is parallel to the line of sight, $\sin\gamma=0$ and the triangulation angle is $\alpha=0$: the point lands on the principal point in both frames, the pixels are identical, there is no parallax, the information is zero along the line of sight and the depth variance is infinite. A long baseline does not mean large parallax; the effective baseline is the component perpendicular to the line of sight, $b\sin\gamma$. A point slightly off the axis gets a little parallax, but very little (FIG 3, FIG 4).

5. 校正双目、视差噪声为高斯。对一个很远的点，哪个量的估计分布最接近高斯？5. Rectified stereo, Gaussian disparity noise. For a very distant point, which quantity has the most nearly Gaussian estimate?

$\rho=d/(fB)$ 是视差 $d$ 的线性函数，高斯噪声经过线性映射仍是高斯；$Z=fB/d$ 是 $d$ 的倒数，当 $d$ 的标准差与 $d$ 本身可比时，$Z$ 的分布严重偏斜并有长尾，甚至 $d\le0$ 时 $Z$ 为负或无穷。$\log Z=-\log\rho+\text{常数}$ 也不是高斯，尽管比 $Z$ 好一些。$\rho=d/(fB)$ is a linear function of the disparity $d$, and Gaussian noise stays Gaussian under a linear map. $Z=fB/d$ is the reciprocal of $d$; when the standard deviation of $d$ is comparable to $d$, $Z$ is heavily skewed with a long tail, and for $d\le0$ it is negative or infinite. $\log Z=-\log\rho+\text{const}$ is not Gaussian either, though better than $Z$.

9延伸阅读Further reading

R. Hartley & A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed., Cambridge University Press, 2004, Ch. 12 — 结构计算：线性与几何（重投影）三角化，最优解，以及估计点的概率分布Structure computation: linear and geometric (reprojection) triangulation, the optimal solution, and the probability distribution of the estimated point
R. I. Hartley & P. Sturm, Triangulation, Computer Vision and Image Understanding 68(2), 1997, 146–157 — 两视图的最优三角化：化为 6 次多项式Optimal two-view triangulation via a degree-6 polynomial
P. Lindstrom, Triangulation Made Easy, CVPR 2010 — 快速、非迭代、接近最优的两视图三角化Fast, non-iterative, near-optimal two-view triangulation
R. Szeliski, Computer Vision: Algorithms and Applications (1st ed., 2010, §7.1; in the 2nd ed., 2022, the chapter on structure from motion and SLAM) — 三角化与 SfM 的整体流程Triangulation within the whole SfM pipeline
J. Civera, A. J. Davison & J. M. M. Montiel, Inverse Depth Parametrization for Monocular SLAM, IEEE Transactions on Robotics 24(5), 2008 — 逆深度：为什么它能让无穷远的路标进入滤波器Inverse depth: how it lets landmarks at infinity enter the filter
J. L. Schönberger & J.-M. Frahm, Structure-from-Motion Revisited, CVPR 2016 — COLMAP 的增量重建，包含多视图三角化与过滤的工程细节COLMAP's incremental reconstruction, including the engineering details of multi-view triangulation and filtering