Part IV · 第 13 章 反馈控制 · 13.3Part IV · Ch. 13 Feedback Control · 13.3

LQR：最优状态反馈LQR: Optimal State Feedback

6.12 节用动态规划解出了有限时域的 LQR。这一节把时域拉到无穷远：Riccati 递推收敛到代数 Riccati 方程，给出一个常数增益 \(K\)，一行 \(\mathbf u=-K\mathbf x\) 就能把倒立摆小车立住。我们会在增益平面上看"最优"长什么样，用 Bryson 法则按物理单位调 \(Q\)、\(R\)，看线性控制器在非线性、有饱和的真实系统上能管多大的范围，用时变 LQR 和 iLQR 把单摆从下方摆起来，最后看 LQR 自带的鲁棒裕度，以及加上卡尔曼滤波器后的 LQG 与分离原理。 Lesson 6.12 solved the finite-horizon LQR with dynamic programming. Here the horizon goes to infinity: the Riccati recursion converges to the algebraic Riccati equation, which gives one constant gain \(K\), and the single line \(\mathbf u=-K\mathbf x\) balances a cart-pole. We look at what "optimal" means on a plane of gains, tune \(Q\) and \(R\) in physical units with Bryson's rule, measure how much of the real, nonlinear and saturated system a linear controller can handle, swing a pendulum up with time-varying LQR and iLQR, and finish with the robustness margins LQR guarantees and with LQG, a Kalman filter plus LQR, and its separation principle.

约 45 分钟~45 min 前置：6.12 动态规划 · 13.2 状态空间与稳定性 · 4.8 卡尔曼滤波 · 8.1 拉格朗日动力学Prereqs: 6.12 Dynamic Programming · 13.2 State Space & Stability · 4.8 Kalman Filter · 8.1 Lagrangian Dynamics 6 个交互6 interactives

    1为什么机器人需要它Why robots need this

    13.1 节的 PID 一次只管一个回路：一个误差，一个力矩。可是很多机器人只有一个执行器，却要同时照顾好几个量。8.1 节的倒立摆小车只能推小车，却要同时让摆杆竖直、让小车回到中间；平衡车、球形机器人、四旋翼悬停、人形机器人的质心平衡都是这样。状态有四个分量 \(\mathbf x=(x,\dot x,\theta,\dot\theta)\)，反馈律 \(F=-K\mathbf x\) 有四个耦合在一起的增益，一个一个手调几乎不可能，而且它们的符号还常常"反直觉"。

    13.2 节的极点配置能把闭环极点放到任何指定的位置，却不告诉你该放在哪里：极点越靠左，响应越快，力也越大，会撞上电机的极限。LQR（线性二次调节器，linear-quadratic regulator）换一个问法：你只需写下你在乎什么，

    The PID of Lesson 13.1 handles one loop at a time: one error, one torque. But many robots have a single actuator and several quantities to look after at once. The cart-pole of Lesson 8.1 can only push its cart, yet must keep the pole upright and bring the cart back to the middle; self-balancing two-wheelers, ballbots, a hovering quadrotor and a humanoid keeping its centre of mass over its feet are all like this. The state has four components \(\mathbf x=(x,\dot x,\theta,\dot\theta)\), the feedback law \(F=-K\mathbf x\) has four coupled gains, tuning them one by one is hopeless, and their signs are often "counter-intuitive".

    The pole placement of Lesson 13.2 can put the closed-loop poles anywhere you like, but it does not tell you where to put them: the further left, the faster the response and the larger the force, until the motor's limit is hit. LQR (the linear-quadratic regulator) asks a different question: you only write down what you care about,

    $$ J=\sum_{k=0}^{\infty}\Big(\mathbf x_k^{\top}Q\,\mathbf x_k+\mathbf u_k^{\top}R\,\mathbf u_k\Big),\qquad \mathbf x_{k+1}=A\mathbf x_k+B\mathbf u_k, $$

    "偏离多少要付多少代价"（\(Q\)）和"用多大力要付多少代价"（\(R\)），LQR 就给出让 \(J\) 最小的反馈律。答案出奇地简单：它是一个常数矩阵 \(K\)，\(\mathbf u=-K\mathbf x\)，离线解一个矩阵方程就能得到，每个控制周期只需做一次矩阵乘向量。6.12 节已经推出了有限时域 \(N\) 步的版本（Riccati 递推和"价值函数是椭圆"），并看到时域足够长时增益趋于常数。这一节接着往下走：

    • 无限时域的增益从哪里来（代数 Riccati 方程），为什么它一定让线性系统稳定；
    • \(Q\)、\(R\) 怎么定（Bryson 法则），"最优"在增益空间里长什么样；
    • 在真实的非线性、有力的上限的系统上，它能管多大的范围（吸引域）；
    • 不止平衡一个点，还要跟踪一条轨迹：时变 LQR，以及用它当求解器的 iLQR；
    • 传感器只测到部分状态、还带噪声时怎么办（LQG），以及 LQR 有多"皮实"（鲁棒裕度）。

    what deviation costs (\(Q\)) and what effort costs (\(R\)), and LQR returns the feedback law that minimizes \(J\). The answer is surprisingly simple: a constant matrix \(K\) with \(\mathbf u=-K\mathbf x\), computed offline by solving one matrix equation, after which each control cycle costs a single matrix–vector product. Lesson 6.12 already derived the finite-horizon version over \(N\) steps (the Riccati recursion, and "the value function is an ellipse") and saw the gain settle to a constant for long horizons. This lesson picks up from there:

    • where the infinite-horizon gain comes from (the algebraic Riccati equation), and why it always stabilizes the linear system;
    • how to choose \(Q\) and \(R\) (Bryson's rule), and what "optimal" looks like in the space of gains;
    • how much of a real system, nonlinear and with a force limit, it can handle (the region of attraction);
    • tracking a trajectory instead of holding one point: time-varying LQR, and iLQR, which uses it as a solver;
    • what to do when sensors measure only part of the state, with noise (LQG), and how tough LQR is (robustness margins).

    2直觉：每个增益都有一个价钱Intuition: every gain has a price

    先看最简单的不稳定系统：一个电机直接驱动的倒立单摆（质量 \(m=1\) kg 集中在 \(l=0.5\) m 处，\(\theta\) 从竖直向上量起）。线性化后 \(ml^2\ddot\theta=mgl\,\theta+u\)，控制律 \(u=-k_\theta\theta-k_\omega\dot\theta\) 只有两个增益，于是所有可能的控制器构成一个平面。平面上每一点是一个控制器，它从初始倾角 \(\theta_0=0.2\) rad 出发会产生一条响应，这条响应有一个总代价

    Start with the simplest unstable system: an inverted pendulum driven directly by a motor (mass \(m=1\) kg concentrated at \(l=0.5\) m, \(\theta\) measured from straight up). Linearized, \(ml^2\ddot\theta=mgl\,\theta+u\), and the law \(u=-k_\theta\theta-k_\omega\dot\theta\) has only two gains, so all possible controllers form a plane. Each point of the plane is one controller; starting from the tilt \(\theta_0=0.2\) rad it produces a response, and that response has a total cost

    $$ \hl{J}{J(K)}=\sum_{k}\big(\theta_k^2+0.1\,\dot\theta_k^2+r\,u_k^2\big)\,\Delta t ,\qquad \hl{Kopt}{K^\star}=\arg\min_K J(K). $$

    （\(\Delta t=10\) ms，控制器以 100 Hz 运行，求和乘 \(\Delta t\) 只是为了让数值像积分。）下图左边把每个 \(K\) 的代价画成颜色，越青代价越低；红色区域是闭环不稳定的增益，代价为无穷。拖动琥珀色的点亲手调增益，绿色菱形是 LQR 算出的 \(K^\star\)。右边是两者的响应。

    (\(\Delta t=10\) ms, the controller runs at 100 Hz; the factor \(\Delta t\) only makes the sum read like an integral.) On the left the figure colours every \(K\) by its cost, more cyan = lower cost; the red region holds gains whose closed loop is unstable, with infinite cost. Drag the amber dot to tune the gains by hand; the green diamond is the \(K^\star\) that LQR computes. On the right are the two responses.

    FIG 1增益平面上的代价地形The cost landscape over the gain plane
    左：横轴 \(k_\theta\)（N·m/rad），纵轴 \(k_\omega\)（N·m·s/rad）。颜色是 \(\log_{10}(J/J^\star)\)，青色最浓处是最低点，从最优往外每深一档代价大一个台阶（细青线是 \(J/J^\star=1.01,\,1.1,\,1.5,\,3\) 的等值线）。红色虚线 \(k_\theta=mgl\approx4.9\) N·m/rad 左边，电机的"弹簧"抵不过重力，摆一定倒下。紫色虚线是 \(r\) 从 \(10^{-4}\) 变到 \(10^{4}\) 时 \(K^\star\) 走过的路。右：上面是 \(\theta(t)\)，下面是力矩 \(u(t)\)；琥珀色是你的增益，绿色虚线是 \(K^\star\)。响应用的是线性模型，与 \(J\) 的计算一致。 Left: horizontal \(k_\theta\) (N·m/rad), vertical \(k_\omega\) (N·m·s/rad). The colour is \(\log_{10}(J/J^\star)\); the deepest cyan is the minimum and each step outward is a step up in cost (thin cyan lines are the contours \(J/J^\star=1.01,\,1.1,\,1.5,\,3\)). Left of the red dashed line \(k_\theta=mgl\approx4.9\) N·m/rad the motor's "spring" cannot beat gravity and the pole must fall. The purple dashed curve is the path of \(K^\star\) as \(r\) goes from \(10^{-4}\) to \(10^{4}\). Right: \(\theta(t)\) on top, torque \(u(t)\) below; amber is your gain, dashed green is \(K^\star\). The responses use the linear model, consistent with how \(J\) is computed.
    试一试Try this
    • 保持 \(r=0.01\)，拖动琥珀色的点，让代价比最优只高不到 1%（\(J/J^\star<1.01\)）。你停下的位置离绿色菱形有多远？代价地形在最优点附近有多平？Keep \(r=0.01\) and drag the amber dot until the cost is less than 1% above optimal (\(J/J^\star<1.01\)). How far from the green diamond did you stop? How flat is the landscape near the optimum?
    • 把 \(r\) 调到 100 以上（力矩极贵）。\(K^\star\) 会不会一路退到 0，或者退到红线上？它停在哪里？Raise \(r\) above 100 (torque is very expensive). Does \(K^\star\) retreat all the way to 0, or to the red line? Where does it stop?
    核心洞见Key insight

    LQR 不是"另一种调参技巧"，而是把调参换成了说清楚目标：一旦写下 \(Q\)、\(R\)，最优增益就唯一确定了，而且它对所有初始状态同时最优（换一个 \(\theta_0\)，地形变了，最低点不变）。最低点附近的地形很平：把 \(K^\star\) 整体放大 1.3 倍，代价只多 1.9%；缩小到 0.7 倍，多 5%。所以真正要花心思的不是把增益算到小数点后几位，而是选对 \(Q\)、\(R\)。另一面是 \(r\to\infty\) 时 \(K^\star\) 不会退到零：对不稳定的系统，"不花力气"不是一个选项，LQR 会停在用最小代价把不稳定极点镜像到左半平面的增益上（这里 \(k_\theta\to9.6\)，大约是 \(2mgl\)）。

    LQR is not "another tuning trick"; it replaces tuning with stating the goal. Once \(Q\) and \(R\) are written down the optimal gain is unique, and it is optimal for every initial state at once (change \(\theta_0\) and the landscape changes, but not its minimum). Near the minimum the landscape is flat: scaling \(K^\star\) up by 1.3 costs only 1.9% more, scaling it down to 0.7 costs 5% more. So the effort belongs in choosing \(Q\) and \(R\), not in computing gains to many decimals. The flip side: as \(r\to\infty\), \(K^\star\) does not retreat to zero. For an unstable system "spend nothing" is not an option, and LQR settles on the cheapest gain that mirrors the unstable pole into the left half-plane (here \(k_\theta\to9.6\), about \(2mgl\)).

    3动手玩：用 LQR 立住倒立摆小车Play: balancing a cart-pole with LQR

    现在换成 8.1 节的倒立摆小车：小车 \(M_c=1\) kg，摆端质量 \(m=0.3\) kg，杆长 \(l=0.8\) m，只能推小车。在竖直位置线性化（13.2 节的做法），状态 \(\mathbf x=(x,\dot x,\theta,\dot\theta)\)：

    Now take the cart-pole of Lesson 8.1: cart \(M_c=1\) kg, bob \(m=0.3\) kg, rod \(l=0.8\) m, and only the cart can be pushed. Linearize at the upright position (as in Lesson 13.2), with state \(\mathbf x=(x,\dot x,\theta,\dot\theta)\):

    $$ \dot{\mathbf x}=\underbrace{\begin{bmatrix}0&1&0&0\\0&0&-\frac{mg}{M_c}&0\\0&0&0&1\\0&0&\frac{(M_c+m)g}{M_cl}&0\end{bmatrix}}_{A_c}\mathbf x+\underbrace{\begin{bmatrix}0\\\frac{1}{M_c}\\0\\-\frac{1}{M_cl}\end{bmatrix}}_{B_c}F . $$

    \(A_c\) 有一个特征值 \(+\sqrt{(M_c+m)g/(M_cl)}\approx+3.99\) s⁻¹：不加控制，倾角的误差每 0.25 s 放大 \(e\) 倍。控制器以 100 Hz 运行，所以先按 13.2 节的零阶保持把 \((A_c,B_c)\) 离散化成 \((A,B)\)，再解离散 Riccati 方程得到 \(K\)。

    怎么选 \(Q\) 和 \(R\)？它们的数值本身没有意义（同时乘 10，\(K\) 不变），有意义的是比例。Bryson 法则给出一个按物理单位思考的起点：

    \(A_c\) has an eigenvalue \(+\sqrt{(M_c+m)g/(M_cl)}\approx+3.99\) s⁻¹: uncontrolled, a tilt error grows by a factor \(e\) every 0.25 s. The controller runs at 100 Hz, so we first discretize \((A_c,B_c)\) to \((A,B)\) with the zero-order hold of Lesson 13.2, then solve the discrete Riccati equation for \(K\).

    How to choose \(Q\) and \(R\)? Their values mean nothing by themselves (multiply both by 10 and \(K\) does not change); only their ratios do. Bryson's rule gives a starting point in physical units:

    $$ Q_{ii}=\frac{1}{(\text{largest acceptable }x_i)^2},\qquad R_{jj}=\frac{1}{(\text{largest acceptable }u_j)^2}. $$

    每一项代价都变成"相对于可接受值的比例的平方"：小车偏离 \(x_{\max}\)、摆杆倾斜 \(\theta_{\max}\)、用力 \(F_{\max}\)，各算 1 份代价，单位就消掉了。下图的三个滑块就是 \(x_{\max}\)、\(\theta_{\max}\)、\(F_{\max}\)（速度的权重设为 0），右上角的公式实时显示由它们算出的 \(Q\)、\(R\)、\(K\)。把绿色的目标标记拖到导轨上的新位置，松手后小车开到那里；按"踢一下"给摆杆一个 1.5 rad/s 的角速度冲量；也可以直接拖动摆端的小球把摆杆扳歪，松手看它恢复。

    Every cost term becomes "the squared fraction of what is acceptable": a cart offset of \(x_{\max}\), a tilt of \(\theta_{\max}\) and a force of \(F_{\max}\) each cost one unit, and the units cancel. The three sliders below are \(x_{\max}\), \(\theta_{\max}\) and \(F_{\max}\) (the velocity weights are zero); the formula at the top right shows the \(Q\), \(R\) and \(K\) they produce, live. Drag the green target marker to a new spot on the rail and release it, and the cart drives there; "Kick" gives the pole an angular-velocity impulse of 1.5 rad/s; or drag the bob to tilt the pole by hand and release it to watch the recovery.

    FIG 2LQR 倒立摆小车（非线性仿真，100 Hz 控制）LQR cart-pole (nonlinear simulation, 100 Hz control)
    小车蓝色，摆杆琥珀色，紫色箭头是控制力 \(F\)（1 cm ↔ 1 N），绿色三角是目标位置 \(x_{\mathrm{ref}}\)（控制器用的是 \(x-x_{\mathrm{ref}}\)）。电机最多出 30 N，导轨两端在 ±2 m 处有挡板。右侧滚动显示最近 8 s 的 \(\theta\)（琥珀，度）、\(x\)（蓝，m）和 \(F\)（紫，N）。仿真用完整的非线性方程（8.1 节），RK4 步长 2 ms，控制力每 10 ms 更新一次并保持不变。摆杆倾角超过 75° 判为倒下，按"复位"重来。 The cart is blue, the pole amber, the purple arrow is the control force \(F\) (1 cm ↔ 1 N), and the green triangle is the target \(x_{\mathrm{ref}}\) (the controller uses \(x-x_{\mathrm{ref}}\)). The motor delivers at most 30 N, and the rail has end stops at ±2 m. On the right the last 8 s of \(\theta\) (amber, degrees), \(x\) (blue, m) and \(F\) (purple, N) scroll by. The simulation uses the full nonlinear equations (Lesson 8.1), RK4 with a 2 ms step, and the control force is updated every 10 ms and held in between. A tilt beyond 75° counts as fallen; press "Reset" to start again.
    试一试Try this
    • 用默认权重，把目标挪开至少 0.8 m（拖动绿色三角，或者点"目标 +1 m / −1 m"）。小车出发时先往反方向退了几厘米（至少 3 cm）。为什么要先退？With the default weights move the target at least 0.8 m away (drag the green triangle or press "target +1 m / −1 m"). The cart first backs off at least 3 cm in the opposite direction. Why must it back off first?
    • 调成"温柔"的控制器：完成一次至少 0.8 m 的移动并停稳（误差 5 cm 内），整个过程中力的峰值小于 2 N，中途不要踢或拖摆杆。提示：用 Bryson 法则的语言想一想，你愿意让小车偏多远、愿意出多大的力？Make the controller "gentle": complete a move of at least 0.8 m and settle (within 5 cm) with a peak force below 2 N throughout, without kicking or dragging the pole during the move. Hint: in the language of Bryson's rule, how far are you willing to let the cart stray, and how much force are you willing to spend?
    • 反过来，让控制器对小车位置"贪心"：\(x_{\max}\le0.1\) m，\(F_{\max}\ge30\) N（\(\theta_{\max}\) 取 2° 以上），然后踢一下。线性闭环的极点全在左半平面，摆杆却倒了。是什么出了问题？（第 4 节的 FIG 3 会回答。）Now make the controller "greedy" about the cart position: \(x_{\max}\le0.1\) m, \(F_{\max}\ge30\) N (with \(\theta_{\max}\) at 2° or more), then kick. All poles of the linear closed loop are in the left half-plane, yet the pole falls. What went wrong? (FIG 3 in Section 4 answers.)
    核心洞见Key insight

    LQR 自己"发现"了倒立摆的诀窍。要把小车向右开，必须先让摆杆向右倾，否则一加速摆杆就向左倒；而要让摆杆向右倾，小车得先向左退一下。默认权重下，移动 1 m 时小车先倒退约 12 cm，力的峰值 18.4 N，2.4 s 内停到目标 5 cm 以内。这就是增益 \(K\) 里小车位置那一项"符号看上去是反的"的原因（8.1 节提到过）：没有人告诉 LQR 这个策略，它只是 Riccati 方程的解。手调四个耦合的增益很难碰对这种结构。

    LQR "discovers" the trick of the inverted pendulum by itself. To drive the cart right the pole must first lean right, otherwise accelerating would topple it to the left; and to make the pole lean right the cart must first back off to the left. With the default weights a 1 m move starts with the cart backing off about 12 cm, peaks at 18.4 N, and settles within 5 cm of the target in 2.4 s. This is why the cart-position entry of \(K\) "looks like it has the wrong sign" (as Lesson 8.1 noted): nobody told LQR this strategy; it is simply the solution of the Riccati equation. Tuning four coupled gains by hand rarely hits on such structure.

    4另外两个视角：吸引域与轨迹跟踪Two more views: region of attraction and trajectory tracking

    线性的增益，非线性的系统A linear gain on a nonlinear system

    LQR 的"最优"和"一定稳定"都是对线性化模型说的。真实系统里有 \(\sin\theta\)、有离心项 \(ml\dot\theta^2\sin\theta\)，电机还有上限。2.4 节说过：线性化只在工作点附近可信。那么"附近"到底多大？下图回答这个问题：把初始倾角 \(\theta_0\) 和初始角速度 \(\dot\theta_0\) 铺成一张网格（小车静止在原点），每一格都用非线性模型、带力限幅地仿真 6 s，能立回来的格子涂绿色，倒下的涂红色。绿色区域就是这个控制器的吸引域（region of attraction）在这张切片上的样子。

    LQR's "optimal" and "always stable" are statements about the linearized model. The real system has \(\sin\theta\), the centrifugal term \(ml\dot\theta^2\sin\theta\), and a motor with a limit. As Lesson 2.4 said, a linearization is trustworthy only near the operating point. How near is "near"? The figure answers this: it lays the initial tilt \(\theta_0\) and initial rate \(\dot\theta_0\) out as a grid (cart at rest at the origin), simulates each cell for 6 s with the nonlinear model and the force limit, and paints the cell green if the pole is caught and red if it falls. The green region is this slice of the controller's region of attraction.

    FIG 3吸引域：力的上限与增益的"贪心"程度Region of attraction: the force limit and how greedy the gains are
    左：横轴 \(\theta_0\)（−90° 到 90°），纵轴 \(\dot\theta_0\)（−8 到 8 rad/s），61 × 41 格。绿色 = 6 s 后摆杆回到竖直附近（\(|\theta|<0.05\) rad，\(|\dot\theta|<0.3\) rad/s），红色 = 倒下（\(|\theta|\) 超过 100°）或没回来。琥珀色椭圆是二次型 \(\mathbf x^\top P\mathbf x\le c\)（\(P\) 是 Riccati 方程的解）在 \(x=\dot x=0\) 上的切片，\(c\) 取在 4000 个随机方向上都检查到"\(V\) 每个控制周期都在下降"的最大值：一个抽样估计的、偏保守的"保证区"。三套权重用 Bryson 法则给出：温柔 \((1\text{ m},20^\circ,2\text{ N})\)，默认 \((0.5\text{ m},10^\circ,10\text{ N})\)，贪心 \((0.1\text{ m},2^\circ,50\text{ N})\)。点击任意一格，右边回放从该状态出发的运动（导轨不设挡板，地面刻度每 0.5 m 一格），左边用蓝线画出它在 \((\theta,\dot\theta)\) 上的轨迹。 Left: horizontal \(\theta_0\) (−90° to 90°), vertical \(\dot\theta_0\) (−8 to 8 rad/s), 61 × 41 cells. Green = after 6 s the pole is back near upright (\(|\theta|<0.05\) rad, \(|\dot\theta|<0.3\) rad/s); red = it fell (\(|\theta|\) beyond 100°) or did not come back. The amber ellipse is the slice at \(x=\dot x=0\) of the quadratic form \(\mathbf x^\top P\mathbf x\le c\) (\(P\) solves the Riccati equation), with \(c\) the largest level at which "\(V\) decreases every control period" was confirmed along 4000 random directions: a sampled, conservative "guaranteed zone". The three weight sets come from Bryson's rule: gentle \((1\text{ m},20^\circ,2\text{ N})\), default \((0.5\text{ m},10^\circ,10\text{ N})\), greedy \((0.1\text{ m},2^\circ,50\text{ N})\). Click any cell to replay the motion from that state on the right (no end stops here; ground ticks every 0.5 m) and to draw its path in \((\theta,\dot\theta)\) as a blue line on the left.
    试一试Try this
    • 用默认权重，先把力的上限调到 100 N（绿色约占网格的 35%），再降到 10 N 或更低：绿色缩到 15% 以下。线性模型对这件事一无所知：它的闭环极点一点没变。With the default weights first set the force limit to 100 N (about 35% of the grid is green), then lower it to 10 N or less: the green region shrinks below 15%. The linear model knows nothing of this: its closed-loop poles have not moved at all.
    • 在同一个力上限下，分别看"温柔"和"贪心"两套权重（两者都要算完）。哪一个能接住更多的初始状态？这和 FIG 2 里贪心控制器被踢倒是同一回事吗？At the same force limit look at both the "gentle" and the "greedy" weights (let both finish computing). Which one catches more initial states? Is this the same thing that made the greedy controller fall after a kick in FIG 2?
    核心洞见Key insight

    在 30 N 的上限下，温柔的权重接住了网格的约 54%，默认的约 28%，贪心的只有约 6%。贪心的增益在线性模型里"更好"（极点更靠左），可它为一点点偏差就要求巨大的力，力一饱和，反馈律就不再是 \(-K\mathbf x\)，最优性和稳定性的证明都不再成立。实践中的结论是：先用 Bryson 法则把 \(F_{\max}\) 设得比电机的真实上限小，让控制器在正常扰动下根本碰不到饱和；必须贴着极限工作时，就要换成能显式处理约束的方法（13.4 节的 MPC）。琥珀色椭圆比绿色区域小得多：Lyapunov 函数给出的保证是保守的，但它是一个证书，而网格仿真只是一些例子。把这种椭圆沿轨迹连成一棵树，就是 LQR-Trees（Tedrake 等，2010）。

    At a 30 N limit the gentle weights catch about 54% of the grid, the default about 28%, the greedy ones only about 6%. The greedy gains are "better" in the linear model (poles further left), but they demand huge forces for small deviations; once the force saturates the law is no longer \(-K\mathbf x\), and the proofs of optimality and stability no longer apply. The practical lesson: use Bryson's rule to set \(F_{\max}\) below the motor's real limit, so that normal disturbances never reach saturation; when you must work at the limit, switch to a method that handles constraints explicitly (MPC, Lesson 13.4). The amber ellipse is much smaller than the green region: the guarantee from a Lyapunov function is conservative, but it is a certificate, while the grid simulations are only examples. Chain such ellipses along trajectories into a tree and you get LQR-Trees (Tedrake et al., 2010).

    跟踪一条轨迹：时变 LQR 与 iLQRTracking a trajectory: time-varying LQR and iLQR

    平衡只是"守住一个点"。很多任务要沿一条轨迹运动：把单摆从下垂位置甩到竖直，四旋翼翻个跟头，机械臂快速搬运。思路是一样的：在一条名义轨迹 \((\mathbf x^\star_k,\mathbf u^\star_k)\) 附近线性化，偏差 \(\delta\mathbf x=\mathbf x-\mathbf x^\star_k\) 近似满足 \(\delta\mathbf x_{k+1}\approx A_k\delta\mathbf x_k+B_k\delta\mathbf u_k\)，只是 \(A_k,B_k\) 随时间变。这时 6.12 节的有限时域 Riccati 递推正好派上用场，得到时变增益 \(K_k\)，控制律是 \(\mathbf u_k=\mathbf u^\star_k-K_k(\mathbf x_k-\mathbf x^\star_k)\)。这就是时变 LQR（TVLQR）。

    名义轨迹本身从哪里来？iLQR 用同一个工具来求它：从一串猜测的控制量出发，(1) 前向仿真得到轨迹；(2) 沿轨迹线性化动力学、把代价展开到二次；(3) 用 Riccati 递推反向求出"该怎么改"（一个前馈修正 \(\mathbf k_k\) 和一个反馈增益 \(K_k\)）；(4) 带线搜索地前向执行修正，重复直到收敛。6.13 节把它称为打靶法的一个高效求解器。下图让一个单摆（\(m=1\) kg，\(l=0.5\) m）在 4 s 内从下垂摆到竖直。仿真里没有硬性的力矩上限，但力矩的代价 \(R\) 让最优解舍不得用大力矩，峰值远小于 \(mgl\)，只好来回荡几下积攒能量：先运行 iLQR，再把得到的力矩序列用到一个质量有偏差的"真实"单摆上，比较直接回放（开环）和 TVLQR 跟踪。

    Balancing is "holding one point". Many tasks move along a trajectory: fling a pendulum from hanging to upright, flip a quadrotor, carry a load quickly with an arm. The idea is the same: linearize about a nominal trajectory \((\mathbf x^\star_k,\mathbf u^\star_k)\); the deviation \(\delta\mathbf x=\mathbf x-\mathbf x^\star_k\) approximately obeys \(\delta\mathbf x_{k+1}\approx A_k\delta\mathbf x_k+B_k\delta\mathbf u_k\), only now \(A_k,B_k\) change with time. This is exactly where the finite-horizon Riccati recursion of Lesson 6.12 fits: it yields time-varying gains \(K_k\) and the law \(\mathbf u_k=\mathbf u^\star_k-K_k(\mathbf x_k-\mathbf x^\star_k)\). This is time-varying LQR (TVLQR).

    Where does the nominal trajectory come from? iLQR computes it with the same tool: start from a guessed control sequence, (1) simulate forward to get a trajectory; (2) linearize the dynamics and expand the cost to second order along it; (3) run the Riccati recursion backward to find "how to change it" (a feedforward correction \(\mathbf k_k\) and a feedback gain \(K_k\)); (4) apply the correction forward with a line search, and repeat until convergence. Lesson 6.13 called it an efficient solver for shooting. In the figure a pendulum (\(m=1\) kg, \(l=0.5\) m) must go from hanging to upright in 4 s. There is no hard torque limit in the simulation, but the torque cost \(R\) makes the optimum reluctant to use large torques, so its peak stays far below \(mgl\) and it has to pump energy by swinging: first run iLQR, then apply the resulting torque sequence to a "real" pendulum whose mass is off, and compare plain playback (open loop) with TVLQR tracking.

    FIG 4iLQR 摆起 + 时变 LQR 跟踪iLQR swing-up + time-varying LQR tracking
    左：相平面，横轴 \(\theta\)（rad，0 = 竖直向上，π = 下垂），纵轴 \(\dot\theta\)（rad/s）。青色是 iLQR 当前迭代的轨迹（运行时逐次更新，淡青色是之前的迭代），绿点是目标。按"播放"后：紫色是在质量有偏差的单摆上直接回放 \(\mathbf u^\star\)（开环，4 s 后力矩为 0），琥珀色是 TVLQR（4 s 后切换到竖直点的常数 LQR 增益）。右上：两个单摆的动画（同色）。右下：TVLQR 增益 \(K_k=(k_\theta,k_\omega)\) 随时间的变化（红 = \(k_\theta\)，蓝 = \(k_\omega\)），虚线是竖直点的常数增益。代价权重：\(Q=\mathrm{diag}(0.05,0.005)\)，\(R=0.2\)，终端 \(\mathrm{diag}(300,30)\)，步长 20 ms；TVLQR 的跟踪权重 \(Q=\mathrm{diag}(10,1)\)，\(R=0.1\)。 Left: the phase plane, horizontal \(\theta\) (rad, 0 = upright, π = hanging), vertical \(\dot\theta\) (rad/s). Cyan is the trajectory of the current iLQR iterate (updated live while it runs; faint cyan are earlier iterates); the green dot is the goal. After "Play": purple replays \(\mathbf u^\star\) on the pendulum whose mass is off (open loop; zero torque after 4 s), amber is TVLQR (switching to the constant LQR gain of the upright point after 4 s). Top right: animation of both pendulums (same colours). Bottom right: the TVLQR gains \(K_k=(k_\theta,k_\omega)\) over time (red = \(k_\theta\), blue = \(k_\omega\)); dashed lines are the constant gains of the upright point. Cost weights: \(Q=\mathrm{diag}(0.05,0.005)\), \(R=0.2\), terminal \(\mathrm{diag}(300,30)\), step 20 ms; TVLQR tracking weights \(Q=\mathrm{diag}(10,1)\), \(R=0.1\).
    试一试Try this
    • 按"运行 iLQR"直到收敛（相对代价下降小于 \(10^{-6}\)）。最优轨迹在相平面上先往哪边走？为什么它要先往回荡一下，而不是直接往上推？（看读数里的力矩峰值和 \(mgl\)。）Press "Run iLQR" until it converges (relative cost decrease below \(10^{-6}\)). Which way does the optimal path head first in the phase plane? Why does it swing back once instead of pushing straight up? (Compare the peak torque in the readout with \(mgl\).)
    • iLQR 收敛后，把质量误差调到 ±10% 或更大，按"播放"：开环回放最后离竖直超过 30°，TVLQR 却停在 1° 以内。After iLQR has converged, set the mass error to ±10% or more and press "Play": the open-loop playback ends more than 30° from upright, while TVLQR ends within 1°.
    一个递推，三种用法One recursion, three uses

    稳态 LQR：把 6.12 节的 Riccati 递推跑到收敛，得到常数 \(K\)，守住一个平衡点（FIG 2）。时变 LQR：沿一条已知轨迹的 \(A_k,B_k\) 跑一遍有限时域递推，得到 \(K_k\)，守住一条轨迹（FIG 4 的琥珀色）。iLQR / DDP：轨迹本身未知，就"递推一次、改一次轨迹"反复迭代，每次迭代就是一个局部的时变 LQR 问题；它最后一次反向传递给出的 \(K_k\) 本身就是一个现成的跟踪控制器。三者的计算核心都是同一个矩阵递推。

    Steady-state LQR: run the Riccati recursion of Lesson 6.12 to convergence, get a constant \(K\), hold an equilibrium (FIG 2). Time-varying LQR: run the finite-horizon recursion once along the \(A_k,B_k\) of a known trajectory, get \(K_k\), hold a trajectory (amber in FIG 4). iLQR / DDP: when the trajectory itself is unknown, alternate "one recursion, one trajectory update"; each iteration is a local time-varying LQR problem, and the \(K_k\) of its last backward pass is a ready-made tracking controller. All three run the same matrix recursion at their core.

    5再看数学The math

    提示：把鼠标悬停在公式中带底色的符号上，对应的图元素会被高亮：\(J(K)\) 点亮 FIG 1 的等值线，\(K^\star\) 点亮 FIG 1 的绿色菱形，\(\mathbf u=-K\mathbf x\) 点亮 FIG 2 的力箭头，\(P\) 点亮 FIG 3 的椭圆，\(K_k\) 点亮 FIG 4 的增益曲线，回差不等式点亮 FIG 5 的禁区圆盘，卡尔曼增益 \(L\) 点亮 FIG 6 的估计器极点。Tip: hover the highlighted symbols in the formulas and the matching figure element lights up: \(J(K)\) the contours of FIG 1, \(K^\star\) the green diamond of FIG 1, \(\mathbf u=-K\mathbf x\) the force arrow of FIG 2, \(P\) the ellipse of FIG 3, \(K_k\) the gain curves of FIG 4, the return-difference inequality the forbidden disk of FIG 5, and the Kalman gain \(L\) the estimator poles of FIG 6.

    从有限时域到代数 Riccati 方程From the finite horizon to the algebraic Riccati equation

    6.12 节的递推是 \(P_k=Q+A^\top P_{k+1}A-A^\top P_{k+1}B\,(R+B^\top P_{k+1}B)^{-1}B^\top P_{k+1}A\)（与 6.12 节写成 \(Q+A^\top P_{k+1}(A-BK_k)\) 的形式等价）。时域 \(N\to\infty\) 时，若 \(P_k\) 收敛到某个 \(P\)，它必须是递推的不动点，也就是离散代数 Riccati 方程（DARE）的解：

    The recursion of Lesson 6.12 is \(P_k=Q+A^\top P_{k+1}A-A^\top P_{k+1}B\,(R+B^\top P_{k+1}B)^{-1}B^\top P_{k+1}A\) (equivalent to the form \(Q+A^\top P_{k+1}(A-BK_k)\) used there). As the horizon \(N\to\infty\), if \(P_k\) converges to some \(P\), that \(P\) must be a fixed point of the recursion, i.e. a solution of the discrete algebraic Riccati equation (DARE):

    $$ \hl{P}{P}=Q+A^{\top}PA-A^{\top}PB\,\big(R+B^{\top}PB\big)^{-1}B^{\top}PA,\qquad K=\big(R+B^{\top}PB\big)^{-1}B^{\top}PA,\qquad \hl{u}{\mathbf u=-K\mathbf x}. $$

    什么时候解存在且唯一？若 \((A,B)\) 可镇定（不稳定的模态都能被 \(\mathbf u\) 影响到，13.2 节的可控性条件的弱化版）且 \((A,Q^{1/2})\) 可检测（每个不稳定的模态都会在代价里"被看见"），则 DARE 有唯一的半正定解使 \(A-BK\) 的所有特征值都在单位圆内。第二个条件常被忽略：若 \(Q\) 对某个不稳定模态是零，"让它发散"不花任何代价，LQR 就不会去管它。FIG 2 里速度权重为零却没有问题，因为位置和角度被罚了，而速度会体现在它们的变化里。

    When does a solution exist, and is it unique? If \((A,B)\) is stabilizable (every unstable mode can be influenced by \(\mathbf u\); a weaker form of the controllability of Lesson 13.2) and \((A,Q^{1/2})\) is detectable (every unstable mode "shows up" in the cost), the DARE has a unique positive semi-definite solution, and it makes all eigenvalues of \(A-BK\) lie inside the unit circle. The second condition is often forgotten: if \(Q\) is zero on some unstable mode, letting it diverge costs nothing and LQR will not bother with it. Zero velocity weights in FIG 2 cause no trouble because position and angle are penalized, and velocities show up in how those change.

    怎么解？最直接的是把 6.12 节的递推一直跑下去。对 8.1 节那组权重（\(Q=\mathrm{diag}(1,1,10,1)\)，\(R=0.1\)）在 100 Hz 的模型上，要跑 536 步，增益相邻两步的相对变化才小于 \(10^{-6}\)。实用的求解器更快：Schur 方法（Laub，1979）对哈密顿矩阵做一次特征分解直接得到 \(P\)，MATLAB、SciPy 的 solve_discrete_are 就是这一类；本页的代码用"倍增"算法，每一步把递推的时域翻倍，同样的问题 12 步就收敛。

    How to solve it? The most direct way is to keep running the recursion of Lesson 6.12. For the weights of Lesson 8.1 (\(Q=\mathrm{diag}(1,1,10,1)\), \(R=0.1\)) on the 100 Hz model it takes 536 steps before the gain changes by less than \(10^{-6}\) (relative) from one step to the next. Practical solvers are faster: the Schur method (Laub, 1979) gets \(P\) from one eigen-decomposition of a Hamiltonian matrix (MATLAB and SciPy's solve_discrete_are are of this kind); the code on this page uses a "doubling" algorithm, which doubles the horizon of the recursion at every step and converges on the same problem in 12 steps.

    为什么闭环一定稳定：\(P\) 本身就是 Lyapunov 函数Why the closed loop is stable: \(P\) itself is a Lyapunov function

    贝尔曼方程说，最优剩余代价 \(V(\mathbf x)=\mathbf x^\top P\mathbf x\) 等于"这一步的代价 + 下一步的剩余代价"。沿最优闭环 \(\mathbf x_{k+1}=(A-BK)\mathbf x_k\)：

    The Bellman equation says that the optimal cost-to-go \(V(\mathbf x)=\mathbf x^\top P\mathbf x\) equals "the cost of this step + the cost-to-go from the next". Along the optimal closed loop \(\mathbf x_{k+1}=(A-BK)\mathbf x_k\):

    $$ V(\mathbf x_{k+1})-V(\mathbf x_k)=-\big(\mathbf x_k^{\top}Q\mathbf x_k+\mathbf u_k^{\top}R\mathbf u_k\big)\le0 . $$

    \(V\) 正定、每一步都不增，正是 13.2 节的 Lyapunov 函数，所以 LQR 的稳定性是"白送"的：不需要另外去找证书，Riccati 方程的解就是证书。FIG 3 的椭圆就是 \(V\) 的一个等值面：在非线性系统上，只要在某个水平集 \(\{V\le c\}\) 内 \(V\) 仍然下降，这个集合里的初始状态就都能被接住。

    \(V\) is positive definite and never increases: it is exactly a Lyapunov function in the sense of Lesson 13.2, so LQR's stability comes "for free": no separate certificate is needed, because the solution of the Riccati equation is the certificate. The ellipse of FIG 3 is a level set of \(V\): on the nonlinear system, if \(V\) still decreases throughout some level set \(\{V\le c\}\), every initial state inside it is caught.

    连续时间版本The continuous-time version

    对 \(\dot{\mathbf x}=A_c\mathbf x+B_c\mathbf u\)、\(J=\int_0^\infty(\mathbf x^\top Q\mathbf x+\mathbf u^\top R\mathbf u)\,dt\)，同样的推导（6.12 节的贝尔曼方程换成哈密顿–雅可比–贝尔曼方程）给出连续代数 Riccati 方程（CARE）：

    For \(\dot{\mathbf x}=A_c\mathbf x+B_c\mathbf u\) and \(J=\int_0^\infty(\mathbf x^\top Q\mathbf x+\mathbf u^\top R\mathbf u)\,dt\), the same derivation (with the Hamilton–Jacobi–Bellman equation in place of the Bellman equation of Lesson 6.12) gives the continuous algebraic Riccati equation (CARE):

    $$ A_c^{\top}P+PA_c-PB_cR^{-1}B_c^{\top}P+Q=0,\qquad K=R^{-1}B_c^{\top}P . $$

    8.1 节"自动平衡"用的就是它：\(Q=\mathrm{diag}(1,1,10,1)\)，\(R=0.1\) 给出 \(K=(-3.16,-6.10,-55.0,-14.9)\)。用 100 Hz 的离散模型解 DARE（权重同比例），小车位置的增益是 \(-2.97\)，小了约 6%；步长缩到 1 ms，变成 \(-3.14\)，趋向连续解。控制器以多快的频率运行，就应该用那个频率的离散模型来解。

    The "auto-balance" of Lesson 8.1 is this: \(Q=\mathrm{diag}(1,1,10,1)\), \(R=0.1\) gives \(K=(-3.16,-6.10,-55.0,-14.9)\). Solving the DARE on the 100 Hz discrete model (same weight ratios) gives a cart-position gain of \(-2.97\), about 6% smaller; at a 1 ms step it becomes \(-3.14\), approaching the continuous solution. Solve on the discrete model at the rate the controller actually runs.

    回差不等式与鲁棒裕度The return-difference inequality and robustness margins

    在执行器处把回路断开（单输入、全状态反馈、连续时间），环路传递函数是 \(L(s)=K(sI-A_c)^{-1}B_c\)。从 CARE 可以推出（Kalman，1964）：

    Break the loop at the actuator (single input, full-state feedback, continuous time); the loop transfer function is \(L(s)=K(sI-A_c)^{-1}B_c\). From the CARE one can show (Kalman, 1964):

    $$ \hl{ret}{\big|1+L(j\omega)\big|\ge1}\quad\text{for all }\omega . $$

    几何上：奈奎斯特曲线 \(L(j\omega)\) 永远不进入以 \(-1\) 为圆心、半径为 1 的圆盘。于是它与负实轴的交点要么在 \(-2\) 左边、要么在 0 右边，把执行器增益乘上任何 \(\beta>1/2\) 都不会让曲线穿过 \(-1\)：增益裕度至少是 \([1/2,\infty)\)。单位圆与这个圆盘交在与 \(-1\) 相隔 60° 的两点，所以曲线穿过单位圆时离 \(-1\) 至少 60°：相位裕度至少 60°。这是 LQR 白送的第二件礼物，FIG 5 会把它画出来，并说明它的边界。

    Geometrically: the Nyquist curve \(L(j\omega)\) never enters the disk of radius 1 centred at \(-1\). So it crosses the negative real axis either left of \(-2\) or right of 0, and multiplying the actuator gain by any \(\beta>1/2\) cannot push the curve across \(-1\): the gain margin is at least \([1/2,\infty)\). The unit circle meets this disk at two points 60° away from \(-1\), so where the curve crosses the unit circle it is at least 60° from \(-1\): the phase margin is at least 60°. This is LQR's second free gift; FIG 5 draws it and shows its limits.

    时变 LQR 与 iLQR 的递推The recursions of time-varying LQR and iLQR

    TVLQR 就是 6.12 节的递推，把 \(A,B\) 换成沿名义轨迹的雅可比 \(A_k=\partial\mathbf f/\partial\mathbf x\)、\(B_k=\partial\mathbf f/\partial\mathbf u\)（在 \((\mathbf x^\star_k,\mathbf u^\star_k)\) 处取值），终端 \(P_N\) 取目标点的稳态 LQR 解：

    TVLQR is the recursion of Lesson 6.12 with \(A,B\) replaced by the Jacobians along the nominal trajectory, \(A_k=\partial\mathbf f/\partial\mathbf x\), \(B_k=\partial\mathbf f/\partial\mathbf u\) (evaluated at \((\mathbf x^\star_k,\mathbf u^\star_k)\)), and the terminal \(P_N\) set to the steady-state LQR solution at the goal:

    $$ \begin{gathered} \hl{Kt}{K_k}=\big(R+B_k^{\top}P_{k+1}B_k\big)^{-1}B_k^{\top}P_{k+1}A_k,\qquad P_k=Q+A_k^{\top}P_{k+1}\big(A_k-B_kK_k\big),\\ \mathbf u_k=\mathbf u^\star_k-K_k\big(\mathbf x_k-\mathbf x^\star_k\big). \end{gathered} $$

    iLQR 的反向传递多带一个一次项：把代价在当前轨迹附近展开成 \(\ell_k\approx\ell+\ell_{\mathbf x}^\top\delta\mathbf x+\ell_{\mathbf u}^\top\delta\mathbf u+\tfrac12(\cdots)\)，价值函数也就多了线性项 \(\mathbf v_k^\top\delta\mathbf x\)。记 \(Q_{\mathbf u}\)、\(Q_{\mathbf u\mathbf u}\)、\(Q_{\mathbf u\mathbf x}\) 为局部"Q 函数"（这一步的代价加下一步的价值函数）对 \(\delta\mathbf u\)、\(\delta\mathbf x\) 的导数（它们不是权重矩阵 \(Q\)），反向递推给出前馈修正 \(\mathbf k_k=-Q_{\mathbf u\mathbf u}^{-1}Q_{\mathbf u}\) 和与 TVLQR 同形的增益 \(K_k=Q_{\mathbf u\mathbf u}^{-1}Q_{\mathbf u\mathbf x}\)，前向传递用 \(\mathbf u_k=\bar{\mathbf u}_k+\alpha\mathbf k_k-K_k(\mathbf x_k-\bar{\mathbf x}_k)\)，步长 \(\alpha\) 从 1 开始减半，直到代价下降。iLQR 线性化动力学时只保留一阶项，相当于对整条轨迹做 6.6 节的高斯–牛顿；保留动力学的二阶导数就是 DDP（Jacobson 与 Mayne，1970），收敛更快，每步更贵。FIG 4 里 \(Q_{\mathbf u\mathbf u}\) 上加了一个很小的正则项 \(\mu\)，线搜索失败时把 \(\mu\) 放大，这是 6.6 节 LM 阻尼的同一个想法。

    The backward pass of iLQR carries an extra linear term: expanding the cost about the current trajectory as \(\ell_k\approx\ell+\ell_{\mathbf x}^\top\delta\mathbf x+\ell_{\mathbf u}^\top\delta\mathbf u+\tfrac12(\cdots)\) gives the value function a linear term \(\mathbf v_k^\top\delta\mathbf x\) too. Writing \(Q_{\mathbf u}\), \(Q_{\mathbf u\mathbf u}\), \(Q_{\mathbf u\mathbf x}\) for the derivatives of the local "Q-function" (this step's cost plus the next value function) with respect to \(\delta\mathbf u\) and \(\delta\mathbf x\) (they are not the weight matrix \(Q\)), the backward recursion yields the feedforward correction \(\mathbf k_k=-Q_{\mathbf u\mathbf u}^{-1}Q_{\mathbf u}\) and a gain \(K_k=Q_{\mathbf u\mathbf u}^{-1}Q_{\mathbf u\mathbf x}\) of the same form as TVLQR's, and the forward pass uses \(\mathbf u_k=\bar{\mathbf u}_k+\alpha\mathbf k_k-K_k(\mathbf x_k-\bar{\mathbf x}_k)\), halving the step \(\alpha\) from 1 until the cost decreases. iLQR keeps only first-order terms when linearizing the dynamics, which amounts to Gauss–Newton (Lesson 6.6) over the whole trajectory; keeping the second derivatives of the dynamics gives DDP (Jacobson and Mayne, 1970), which converges faster at a higher cost per step. FIG 4 adds a small regularizer \(\mu\) to \(Q_{\mathbf u\mathbf u}\) and enlarges it when the line search fails: the same idea as the LM damping of Lesson 6.6.

    LQG 与分离原理LQG and the separation principle

    LQR 假设整个状态 \(\mathbf x\) 都量得到。实际上 FIG 2 的小车只有两个编码器：小车位置 \(x\) 和摆角 \(\theta\)，而且带噪声，即 \(\mathbf y=C\mathbf x+\mathbf v\)，\(C\) 取出 \(x\) 和 \(\theta\) 两行。4.8 节的卡尔曼滤波器给出最优估计 \(\hat{\mathbf x}\)（那里卡尔曼增益记作 \(K\)，这里为了不和 LQR 增益混淆改记为 \(L\)）。它的稳态增益来自对偶的一个 Riccati 方程：把 DARE 里的 \((A,B,Q,R)\) 换成 \((A^\top,C^\top,W,V)\)，\(W\)、\(V\) 是过程噪声和测量噪声的协方差。线性二次高斯（LQG）控制器就是把两者串起来：

    LQR assumes the whole state \(\mathbf x\) is measured. In fact the cart of FIG 2 has just two encoders, cart position \(x\) and pole angle \(\theta\), with noise: \(\mathbf y=C\mathbf x+\mathbf v\), where \(C\) picks out the rows of \(x\) and \(\theta\). The Kalman filter of Lesson 4.8 gives the optimal estimate \(\hat{\mathbf x}\) (there the Kalman gain was called \(K\); here it is \(L\), to avoid a clash with the LQR gain). Its steady-state gain comes from the dual Riccati equation: replace \((A,B,Q,R)\) in the DARE with \((A^\top,C^\top,W,V)\), where \(W\) and \(V\) are the process and measurement noise covariances. The linear-quadratic-Gaussian (LQG) controller chains the two:

    $$ \hat{\mathbf x}_k=\hat{\mathbf x}_k^-+\hl{Lk}{L}\big(\mathbf y_k-C\hat{\mathbf x}_k^-\big),\qquad \mathbf u_k=-K\hat{\mathbf x}_k,\qquad \hat{\mathbf x}_{k+1}^-=A\hat{\mathbf x}_k+B\mathbf u_k . $$

    分离原理：写成真实状态 \(\mathbf x\) 与估计误差 \(\mathbf e=\mathbf x-\hat{\mathbf x}\) 的组合，闭环矩阵是块三角的，它的特征值恰好是 \(A-BK\)（控制器）和 \((I-LC)A\)（估计器）的特征值的并集。所以可以分开设计：LQR 只管 \(Q\)、\(R\)，卡尔曼滤波器只管 \(W\)、\(V\)，合起来既稳定，在线性高斯假设下也是最优的。FIG 6 把 8 × 8 闭环矩阵的特征值直接算出来，与两组 4 个极点逐一重合。

    Separation principle: in terms of the true state \(\mathbf x\) and the estimation error \(\mathbf e=\mathbf x-\hat{\mathbf x}\), the closed-loop matrix is block triangular, so its eigenvalues are exactly the union of those of \(A-BK\) (the controller) and \((I-LC)A\) (the estimator). The two can therefore be designed separately: LQR looks only at \(Q\), \(R\), the Kalman filter only at \(W\), \(V\), and together they are stable and, under the linear-Gaussian assumptions, optimal. FIG 6 computes the eigenvalues of the 8 × 8 closed-loop matrix directly, and they land on the two sets of 4 poles one by one.

    6机器人中的它：鲁棒性与只量得到一部分状态In robotics: robustness, and measuring only part of the state

    真实的控制器不在完美的模型上运行：电机的力矩常数有误差（相当于执行器增益 \(\beta\neq1\)），通信、滤波和计算会带来延迟 \(\tau\)。下图在连续时间里为倒立摆小车解 CARE，画出 \(L(j\omega)\) 在 \(-1\) 附近的奈奎斯特图，再让你乘上增益 \(\beta\)、加上纯延迟 \(e^{-j\omega\tau}\)。右边是线性化闭环从 3° 初始倾角出发的响应，直接告诉你稳不稳。

    A real controller does not run on a perfect model: the motor's torque constant is off (an actuator gain \(\beta\neq1\)), and communication, filtering and computation add a delay \(\tau\). The figure solves the CARE for the cart-pole in continuous time and draws the Nyquist plot of \(L(j\omega)\) near \(-1\); then it lets you multiply by a gain \(\beta\) and add a pure delay \(e^{-j\omega\tau}\). On the right, the response of the linearized closed loop from a 3° initial tilt tells you directly whether it is stable.

    FIG 5LQR 的鲁棒裕度：增益与延迟LQR robustness margins: gain and delay
    左：复平面上 \(-1\) 附近的区域（横轴实部 −3.2 到 1.2）。红色圆盘是回差不等式划出的禁区（圆心 \(-1\)，半径 1），青色虚线是单位圆，琥珀色是名义环路 \(L(j\omega)\)，\(\omega>0\)（\(\omega<0\) 的镜像画成细虚线），紫色是乘上 \(\beta e^{-j\omega\tau}\) 后的曲线。小车的两个积分器使 \(|L|\) 在 \(\omega\to0\) 时趋于无穷，所以曲线从画面外进来。右：线性化闭环（带增益 \(\beta\) 与延迟 \(\tau\)）的 \(\theta(t)\)（紫色），从 3° 出发，12 s；比较 4–6 s 与 10–12 s 的包络判断稳定与否。三套权重同 FIG 3。 Left: the region of the complex plane around \(-1\) (real part from −3.2 to 1.2). The red disk is the zone the return-difference inequality forbids (centre \(-1\), radius 1), the dashed cyan line is the unit circle, amber is the nominal loop \(L(j\omega)\) for \(\omega>0\) (the mirror image for \(\omega<0\) is the thin dashed line), and purple is the curve after multiplying by \(\beta e^{-j\omega\tau}\). The cart's two integrators make \(|L|\to\infty\) as \(\omega\to0\), so the curve enters from outside the view. Right: \(\theta(t)\) (purple) of the linearized closed loop (with gain \(\beta\) and delay \(\tau\)) from a 3° tilt over 12 s; stability is judged by comparing the envelope over 4–6 s with that over 10–12 s. The three weight sets are those of FIG 3.
    试一试Try this
    • 选"贪心"权重，\(\beta=1\)，慢慢加大延迟，找到闭环失稳的那个 \(\tau\)（应在 25 ms 以内）。再换成"默认"：同样的延迟还稳吗？60° 的相位裕度为什么挡不住延迟？Pick the "greedy" weights with \(\beta=1\) and slowly increase the delay until the closed loop goes unstable (it should happen within 25 ms). Switch to "default": is it still stable at the same delay? Why does a 60° phase margin not protect against delay?

    三套权重的相位裕度都在 60° 到 65° 之间，名不虚传；但能承受的延迟相差很大：温柔约 116 ms，默认约 66 ms，贪心只有约 16 ms。延迟裕度等于相位裕度除以穿越频率 \(\omega_c\)，而贪心的控制器 \(\omega_c\approx70\) rad/s，是默认的 4 倍多。增益方向上，保证的是 \(\beta>1/2\) 都稳定，实际的下限是：温柔 0.47，默认 0.27，贪心 0.05。100 Hz 的数字控制器单是零阶保持就相当于约 5 ms 的延迟，再加上传感器滤波和通信，几十毫秒的延迟在真实机器人上并不少见，所以"带宽越高越好"并不成立。

    LQR 的保证还有一个前提：全部状态都被精确地测到。只要中间插进一个观测器，这些保证就没有了。Doyle（1978）的论文标题就是结论："Guaranteed margins for LQG regulators"，摘要只有一句："There are none."。下图在 FIG 2 的小车上只测量带噪声的 \(x\) 和 \(\theta\)，比较三种做法：理想的全状态 LQR、用相邻两次测量的差分来估计速度、卡尔曼滤波器（LQG）。

    All three weight sets have phase margins between 60° and 65°, as promised; but the delays they tolerate differ widely: gentle about 116 ms, default about 66 ms, greedy only about 16 ms. The delay margin is the phase margin divided by the crossover frequency \(\omega_c\), and the greedy controller has \(\omega_c\approx70\) rad/s, more than four times the default. In gain, the guarantee is stability for every \(\beta>1/2\); the actual lower limits are 0.47 (gentle), 0.27 (default) and 0.05 (greedy). A 100 Hz digital controller's zero-order hold alone acts like a delay of about 5 ms, and with sensor filtering and communication, delays of tens of milliseconds are common on real robots; "the higher the bandwidth the better" does not hold.

    LQR's guarantees carry one more assumption: the whole state is measured exactly. Insert an observer and they are gone. The title of Doyle's 1978 paper is the conclusion, "Guaranteed margins for LQG regulators", and its abstract is one sentence: "There are none." The figure measures only noisy \(x\) and \(\theta\) on the cart of FIG 2 and compares three approaches: ideal full-state LQR, velocities estimated by differencing consecutive measurements, and a Kalman filter (LQG).

    FIG 6LQG：卡尔曼滤波 + LQR 与分离原理LQG: Kalman filter + LQR and the separation principle
    左：10 s 的仿真（非线性小车，默认权重，30 N 限幅，从 \(\theta_0\approx4.6^\circ\) 出发），上面是 \(\theta(t)\)（度），下面是控制力 \(F(t)\)（N）。青色 = 理想全状态 LQR，紫色 = 差分估速度，琥珀色 = LQG。三者用同一串噪声：小车位置噪声 \(\sigma_x=2\) mm，角度噪声 \(\sigma_\theta\) 可调，另有每 10 ms 一个、标准差 1 N 的随机扰动力。"滤波器假设的过程噪声"是卡尔曼滤波器以为的扰动大小与真实值之比：小了就过分相信模型，大了就过分相信测量。右：连续时间等效极点 \(s=\ln z/\Delta t\) 的位置（横轴实部，纵轴虚部，单位 s⁻¹）。琥珀色叉 = \(A-BK\) 的 4 个控制器极点，青色圈 = \((I-LC)A\) 的 4 个估计器极点，紫色大圆环 = 直接对 8 × 8 闭环矩阵求出的 8 个特征值。表中 RMS 取 1–10 s；摆杆倒下则显示"倒下"。 Left: a 10 s simulation (nonlinear cart, default weights, 30 N limit, starting from \(\theta_0\approx4.6^\circ\)): \(\theta(t)\) in degrees on top, control force \(F(t)\) in N below. Cyan = ideal full-state LQR, purple = velocities by differencing, amber = LQG. All three share the same noise: cart-position noise \(\sigma_x=2\) mm, an adjustable angle noise \(\sigma_\theta\), and a random disturbance force with a standard deviation of 1 N every 10 ms. "Filter's assumed process noise" is the ratio of the disturbance the Kalman filter believes in to the true one: too small and it trusts the model too much, too large and it trusts the measurements too much. Right: positions of the equivalent continuous-time poles \(s=\ln z/\Delta t\) (horizontal real part, vertical imaginary part, in s⁻¹). Amber crosses = the 4 controller poles of \(A-BK\), cyan circles = the 4 estimator poles of \((I-LC)A\), large purple rings = the 8 eigenvalues computed directly from the 8 × 8 closed-loop matrix. RMS values in the table are over 1–10 s; a fallen pole shows "fell".
    试一试Try this
    • 把角度噪声调到 1° 左右（1.0° 到 1.3°）：差分估速度的控制器把摆杆弄倒了，LQG 的 \(\theta\) 的 RMS 却仍在 1° 以内。噪声为什么会被差分放大？（每 10 ms 差一次，1° 的噪声变成多大的速度噪声？）Raise the angle noise to about 1° (1.0° to 1.3°): the differencing controller drops the pole, while LQG keeps the RMS of \(\theta\) within 1°. Why does differencing amplify noise? (Differencing every 10 ms, what velocity noise does 1° of angle noise become?)
    • 保持 \(\sigma_\theta=0.5^\circ\)，把"滤波器假设的过程噪声"拉到 0.01 倍（过分相信模型），LQG 的 \(\theta\) RMS 涨到原来的 3 倍以上；再拉到 100 倍（过分相信测量），这时变差的是哪一项？Keep \(\sigma_\theta=0.5^\circ\) and pull "filter's assumed process noise" down to 0.01× (trusting the model too much): LQG's RMS \(\theta\) more than triples. Then push it to 100× (trusting the measurements too much): which number gets worse now?

    默认设置（\(\sigma_\theta=0.5^\circ\)）下：理想 LQR 的力 RMS 是 0.37 N，LQG 是 0.68 N，差分估速度是 23.8 N。差分的力几乎一直在 30 N 的上限之间来回打，真实的电机会发热、齿轮会磨损，摆杆的 RMS 是 2.8°，LQG 是 0.49°（理想 LQR 是 0.18°）。卡尔曼滤波器用模型把各次测量"平均"起来：它知道摆杆不会在 10 ms 里突然改变 1 rad/s 的角速度。

    At the default setting (\(\sigma_\theta=0.5^\circ\)) the RMS force is 0.37 N for ideal LQR, 0.68 N for LQG and 23.8 N for differencing. The differenced force slams back and forth between the ±30 N limits almost all the time, which on a real motor means heat and gear wear, and the pole's RMS is 2.8°, against 0.49° for LQG (0.18° for ideal LQR). The Kalman filter "averages" the measurements through the model: it knows the pole cannot change its angular velocity by 1 rad/s within 10 ms.

    真实系统里In real systems

    平衡类机器人：两轮自平衡车、球形机器人（ballbot）、反作用轮倒立摆，常见的做法正是本节的流程：在竖直点线性化，离线解 Riccati 方程，在 100 Hz 到 1 kHz 的回路里做一次 \(\mathbf u=-K\hat{\mathbf x}\)，状态由 IMU 和编码器经滤波器估计。人形机器人：MIT 的 Atlas 控制器（Kuindersma 等，2016）把质心/ZMP 简化模型上时变 LQR 的最优剩余代价（cost-to-go）当作目标函数，放进每个控制周期求解、包含整机动力学与接触约束的二次规划里（13.7 节）。轨迹优化：iLQR/DDP 是很多运动生成工具的核心，例如面向足式机器人多接触最优控制的 Crocoddyl（FDDP 求解器），以及 MuJoCo MPC 里的 iLQG 规划器；它们反向传递得到的 \(K_k\) 常被直接用作跟踪反馈。软件：MATLAB 的 lqr/dlqr/lqi、python-control 的 lqr/dlqr、SciPy 的 solve_continuous_are/solve_discrete_are、Drake 的 LinearQuadraticRegulator 与 FiniteHorizonLinearQuadraticRegulator。边界：一旦约束（力矩、关节限位、摩擦锥）经常起作用，LQR 就让位给 13.4 节的 MPC；而 MPC 的终端代价通常就是这里的 \(\mathbf x^\top P\mathbf x\)。

    Balancing robots: two-wheeled self-balancers, ballbots and reaction-wheel pendulums commonly follow exactly this lesson's recipe: linearize at upright, solve the Riccati equation offline, and compute one \(\mathbf u=-K\hat{\mathbf x}\) in a 100 Hz to 1 kHz loop, with the state estimated from IMU and encoders by a filter. Humanoids: MIT's Atlas controller (Kuindersma et al., 2016) uses the optimal cost-to-go of a time-varying LQR on a simplified centre-of-mass/ZMP model as the objective of a quadratic program, solved every control cycle with the full robot's dynamics and contact constraints (Lesson 13.7). Trajectory optimization: iLQR/DDP sits at the core of many motion-generation tools, such as Crocoddyl (its FDDP solver) for multi-contact optimal control of legged robots, and the iLQG planner of MuJoCo MPC; the \(K_k\) from their backward pass is often used directly as tracking feedback. Software: MATLAB's lqr/dlqr/lqi, python-control's lqr/dlqr, SciPy's solve_continuous_are/solve_discrete_are, Drake's LinearQuadraticRegulator and FiniteHorizonLinearQuadraticRegulator. Limits: once constraints (torque, joint limits, friction cones) are active often, LQR gives way to the MPC of Lesson 13.4, whose terminal cost is usually this lesson's \(\mathbf x^\top P\mathbf x\).

    7常见误区Common pitfalls

    误区 1：把权重当成有绝对意义的数Pitfall 1: treating the weights as absolute numbers

    \(Q\) 和 \(R\) 同时乘以任何正数，\(K\) 不变（只有 \(P\) 跟着缩放）；把角度从弧度换成度，\(Q\) 里对应的项就差了 \((180/\pi)^2\approx3283\) 倍。"\(Q=I,\ R=1\)"不是一个中性的默认值，它隐含着"1 m 的偏差、1 rad 的倾斜和 1 N 的力一样重要"。先用 Bryson 法则按物理单位写下可接受的范围，再根据仿真结果调整。

    Multiply \(Q\) and \(R\) by the same positive number and \(K\) does not change (only \(P\) scales); switch an angle from radians to degrees and its entry in \(Q\) changes by \((180/\pi)^2\approx3283\). "\(Q=I,\ R=1\)" is not a neutral default; it silently says "1 m of offset, 1 rad of tilt and 1 N of force matter equally". Write down acceptable ranges in physical units with Bryson's rule first, then adjust from simulation.

    误区 2：以为"最优"就是"好"Pitfall 2: assuming "optimal" means "good"

    LQR 只对你写下的代价和线性模型最优。它不知道力有上限（FIG 3：同样 30 N 的上限，贪心权重的吸引域只有温柔权重的约八分之一），不知道 \(\sin\theta\ne\theta\)，也不知道模型参数不准。闭环极点"更靠左"只说明线性模型里收敛更快，不说明真实系统更稳。上线前要在非线性、带饱和、带噪声和延迟的仿真里检验。

    LQR is optimal only for the cost you wrote and the linear model. It does not know the force is limited (FIG 3: at the same 30 N limit the greedy weights have about one eighth of the gentle weights' region of attraction), that \(\sin\theta\ne\theta\), or that the model parameters are off. Poles "further left" mean faster convergence in the linear model, not a more stable real system. Validate in simulations with nonlinearity, saturation, noise and delay before deploying.

    误区 3：没有积分作用，却期待零稳态误差Pitfall 3: expecting zero steady-state error without integral action

    LQR 是纯比例的状态反馈。对 FIG 2 的默认控制器，小车上持续施加 1 N 的推力，小车最后会停在离目标约 5.4 cm 的地方，而且偏向推力的反方向（因为要靠小车位置那一项产生 −1 N 的力来抵消推力，而那一项的符号是"反的"）。和 13.1 节的 I 项一样，补救办法是把误差的积分 \(\int(x-x_{\mathrm{ref}})\,dt\) 加进状态里一起做 LQR（MATLAB 的 lqi），或者估计出扰动再前馈抵消。

    LQR is purely proportional state feedback. For the default controller of FIG 2, a steady 1 N push on the cart leaves the cart about 5.4 cm from the target, and on the side opposite to the push (the cart-position term has to produce −1 N to cancel the push, and that term has the "reversed" sign). As with the I term of Lesson 13.1, the fix is to append the integral of the error \(\int(x-x_{\mathrm{ref}})\,dt\) to the state and do LQR on the augmented system (MATLAB's lqi), or to estimate the disturbance and cancel it by feedforward.

    误区 4：在离散控制器里用连续时间的增益，或忽略延迟Pitfall 4: using a continuous-time gain in a discrete controller, or ignoring delay

    CARE 的增益假设控制力连续变化。控制器以 100 Hz 运行时，零阶保持本身就带来约半个周期的延迟，第 5 节里同一组权重的离散增益已经比连续增益小了约 6%；频率更低、增益更"贪心"时差别更大，FIG 5 里贪心控制器 16 ms 的延迟就足以失稳。用实际的采样周期离散化后解 DARE，把已知的计算和通信延迟也放进模型（例如给状态加上一拍延迟），再看 FIG 5 那样的裕度。

    The CARE gain assumes the force varies continuously. At 100 Hz the zero-order hold alone adds about half a period of delay, and in Section 5 the discrete gain for the same weights is already about 6% smaller than the continuous one; at lower rates and with greedier gains the gap grows, and in FIG 5 a 16 ms delay is enough to destabilize the greedy controller. Discretize at the actual sampling period and solve the DARE, put known computation and communication delays into the model (for example, a one-step delay state), and then check margins as in FIG 5.

    误区 5：以为 LQG 继承了 LQR 的裕度Pitfall 5: assuming LQG inherits LQR's margins

    分离原理保证 LQG 的标称闭环稳定，但 \([1/2,\infty)\) 的增益裕度和 60° 的相位裕度只对全状态 LQR 成立，加上观测器后可能任意小（Doyle，1978）。估计器越"相信测量"（FIG 6 里过程噪声假设得越大），就越接近差分，噪声越多地进到力里。要找回鲁棒性，可以用回路传递恢复（LTR）有意调整滤波器，或者直接用 \(H_\infty\)、\(\mu\) 综合这类鲁棒控制方法，至少要在带参数误差的仿真里检验 LQG。

    The separation principle guarantees that LQG's nominal closed loop is stable, but the \([1/2,\infty)\) gain margin and the 60° phase margin hold only for full-state LQR; with an observer they can be arbitrarily small (Doyle, 1978). The more the estimator "trusts the measurements" (the larger the assumed process noise in FIG 6), the closer it gets to differencing and the more noise reaches the force. To recover robustness, tune the filter on purpose with loop transfer recovery (LTR), or use robust-control methods such as \(H_\infty\) or \(\mu\)-synthesis; at the very least test LQG in simulations with parameter errors.

    8小练习Check yourself

    1. 把一个 LQR 问题的 \(Q\) 和 \(R\) 都乘以 10，会发生什么？1. In an LQR problem, \(Q\) and \(R\) are both multiplied by 10. What happens?
    代价 \(J\) 整体乘以 10，最小化它的 \(\mathbf u\) 不变，所以 \(K\) 不变；最优代价 \(\mathbf x^\top P\mathbf x\) 也乘以 10，即 \(P\to10P\)。代入 DARE 可以直接验证：\(K=(R+B^\top PB)^{-1}B^\top PA\) 中分子分母各多一个 10 倍。闭环极点由 \(K\) 决定，也不变。只有 \(Q\) 与 \(R\) 的相对大小才影响增益。The whole cost \(J\) is multiplied by 10, so the minimizing \(\mathbf u\) does not change and neither does \(K\); the optimal cost \(\mathbf x^\top P\mathbf x\) is also multiplied by 10, i.e. \(P\to10P\). Substituting into the DARE confirms it: in \(K=(R+B^\top PB)^{-1}B^\top PA\) the factor 10 appears in both the inverse and the product and cancels. The closed-loop poles depend on \(K\), so they stay put too. Only the relative size of \(Q\) and \(R\) affects the gain.
    2. FIG 1 的倒立单摆（\(mgl\approx4.9\) N·m/rad），控制权重 \(r\to\infty\) 时，最优增益 \(K^\star\) 会怎样？2. For the inverted pendulum of FIG 1 (\(mgl\approx4.9\) N·m/rad), what happens to the optimal gain \(K^\star\) as the control weight \(r\to\infty\)?
    不稳定系统不控制，代价是无穷大，所以 \(K^\star\) 不可能趋于零；停在稳定边界上，响应永远不衰减，代价也是无穷大。"最省力"的稳定化把开环极点 \(+\sqrt{g/l}\) 翻到 \(-\sqrt{g/l}\)：闭环特征多项式 \((s+\sqrt{g/l})^2\) 要求 \(k_\theta=2mgl\approx9.8\) N·m/rad（连续时间），100 Hz 的离散版本是 9.6。FIG 1 的紫色虚线就停在那里。Leaving an unstable system uncontrolled has infinite cost, so \(K^\star\) cannot tend to zero; sitting on the stability boundary, the response never decays and the cost is infinite too. The "cheapest" stabilization flips the open-loop pole \(+\sqrt{g/l}\) to \(-\sqrt{g/l}\): the closed-loop characteristic polynomial \((s+\sqrt{g/l})^2\) requires \(k_\theta=2mgl\approx9.8\) N·m/rad (continuous time), or 9.6 for the 100 Hz discrete version. The purple dashed curve of FIG 1 ends there.
    3. 对单输入、全状态反馈的连续时间 LQR，下列哪一条是有保证的？3. For single-input, full-state-feedback continuous-time LQR, which of these is guaranteed?
    回差不等式 \(|1+L(j\omega)|\ge1\) 给出 \([1/2,\infty)\) 的增益裕度和 60° 的相位裕度。延迟裕度是相位裕度除以穿越频率，并没有统一的下界：FIG 5 的贪心控制器 16 ms 就失稳了。增益降到 1/2 以下不一定失稳，但不再有保证（FIG 5 的温柔权重在 0.47 处就失稳了）。加上观测器后保证全部消失（Doyle，1978）。The return-difference inequality \(|1+L(j\omega)|\ge1\) gives a gain margin of \([1/2,\infty)\) and a 60° phase margin. The delay margin is the phase margin divided by the crossover frequency and has no universal lower bound: FIG 5's greedy controller goes unstable at 16 ms. A gain below 1/2 is not necessarily unstable, but it is no longer guaranteed (FIG 5's gentle weights go unstable at 0.47). With an observer, all guarantees disappear (Doyle, 1978).
    4. 用 LQR 增益 \(K\) 和稳态卡尔曼增益 \(L\) 组成 LQG 控制器，闭环的极点是什么？4. An LQG controller is built from the LQR gain \(K\) and the steady-state Kalman gain \(L\). What are the closed-loop poles?
    闭环状态是 \((\mathbf x,\hat{\mathbf x})\)，共 8 维。换成 \((\mathbf x,\mathbf e)\) 坐标后矩阵是块三角的，误差 \(\mathbf e\) 的演化与 \(\mathbf x\) 无关，所以特征值是两个对角块的并集。这就是分离原理，FIG 6 右图的 8 个紫色圆环与 8 个小符号逐一重合。极点由矩阵决定，与噪声的具体取值无关。The closed-loop state is \((\mathbf x,\hat{\mathbf x})\), 8-dimensional. In \((\mathbf x,\mathbf e)\) coordinates the matrix is block triangular, because the error \(\mathbf e\) evolves independently of \(\mathbf x\), so its eigenvalues are the union of those of the two diagonal blocks. This is the separation principle; in the right panel of FIG 6 the 8 purple rings sit on the 8 small markers one by one. The poles are properties of the matrices, not of the noise realization.
    5. 按 Bryson 法则，倒立摆小车可接受的范围是：小车偏离 0.5 m，摆杆倾斜 10°，力 10 N（速度不加权）。\(Q\) 的位置项、角度项（角度用弧度）和 \(R\) 分别约是多少？5. With Bryson's rule, the acceptable ranges for the cart-pole are a 0.5 m cart offset, a 10° tilt and a 10 N force (no velocity weights). What are, approximately, the position and angle entries of \(Q\) (angle in radians) and \(R\)?
    \(1/0.5^2=4\)；10° = 0.1745 rad，\(1/0.1745^2\approx32.8\)；\(1/10^2=0.01\)。这正是 FIG 2 的默认设置，右上角公式里能看到同样的数。选项 A 忘了平方，选项 C 把倒数倒了过来，选项 D 把角度项和 \(R\) 弄反了。\(1/0.5^2=4\); 10° = 0.1745 rad and \(1/0.1745^2\approx32.8\); \(1/10^2=0.01\). This is FIG 2's default setting, and the formula at its top right shows the same numbers. Option A forgets the square, option C inverts the reciprocals, and option D swaps the angle entry and \(R\).

    9延伸阅读Further reading

    • R. E. Kalman, Contributions to the Theory of Optimal Control, Boletín de la Sociedad Matemática Mexicana 5, 1960 — LQR 与 Riccati 方程的奠基论文The founding paper of LQR and the Riccati equation
    • R. E. Kalman, When Is a Linear Control System Optimal?, J. Basic Engineering 86(1), 1964 — 回差不等式与"逆最优"问题The return-difference inequality and the inverse optimal problem
    • B. D. O. Anderson & J. B. Moore, Optimal Control: Linear Quadratic Methods, Prentice Hall, 1990 — LQR、LQG、鲁棒性与回路传递恢复的标准教材The standard text on LQR, LQG, robustness and loop transfer recovery
    • A. E. Bryson & Y.-C. Ho, Applied Optimal Control, Hemisphere, 1975 — 最优控制的经典教材，按可接受范围选权重的思路常归于 BrysonA classic optimal-control text; choosing weights from acceptable ranges is usually credited to Bryson
    • J. C. Doyle, Guaranteed Margins for LQG Regulators, IEEE Trans. Automatic Control 23(4), 1978 — "There are none.""There are none."
    • M. G. Safonov & M. Athans, Gain and Phase Margin for Multiloop LQG Regulators, IEEE Trans. Automatic Control 22(2), 1977 — 多输入 LQR 的增益与相位裕度Gain and phase margins of multi-input LQR
    • A. J. Laub, A Schur Method for Solving Algebraic Riccati Equations, IEEE Trans. Automatic Control 24(6), 1979 — 今天的 Riccati 求解器的基础The basis of today's Riccati solvers
    • D. H. Jacobson & D. Q. Mayne, Differential Dynamic Programming, Elsevier, 1970 — DDP 的原始专著The original monograph on DDP
    • W. Li & E. Todorov, Iterative Linear Quadratic Regulator Design for Nonlinear Biological Movement Systems, ICINCO 2004 — iLQRiLQR
    • Y. Tassa, T. Erez & E. Todorov, Synthesis and Stabilization of Complex Behaviors through Online Trajectory Optimization, IROS 2012 — 实时 iLQG 模型预测控制Real-time iLQG model-predictive control
    • R. Tedrake, I. R. Manchester, M. Tobenkin & J. W. Roberts, LQR-Trees: Feedback Motion Planning via Sums-of-Squares Verification, Int. J. Robotics Research 29(8), 2010 — 用 Lyapunov 椭圆拼出的反馈运动规划Feedback motion planning from chained Lyapunov ellipses
    • R. Tedrake, Underactuated Robotics, MIT course notes — LQR、时变 LQR、吸引域与倒立摆小车，附 Drake 代码LQR, time-varying LQR, regions of attraction and the cart-pole, with Drake code
    • S. Kuindersma, R. Deits, M. Fallon, A. Valenzuela, H. Dai, F. Permenter, T. Koolen, P. Marion & R. Tedrake, Optimization-based Locomotion Planning, Estimation, and Control Design for the Atlas Humanoid Robot, Autonomous Robots 40(3), 2016 — 人形机器人上的时变 LQR 与 QP 控制Time-varying LQR and QP control on a humanoid