经典工作时间线（可交互）An Interactive Timeline of Landmark Works

前面的 30 多节课是按概念组织的：先学回归和反向传播，再学生成模型和强化学习，最后学模仿、sim-to-real 和 model-based 方法。但这些方法当初不是按这个顺序出现的，每一个都是为了解决某个具体的失败而提出的。DAgger 是因为行为克隆一偏离示范就越错越远；Diffusion Policy 是因为 MSE 回归会把"从左绕"和"从右绕"平均成"直直撞上去"；域随机化是因为仿真里训练好的策略到了真机上就失灵。知道一个方法在修什么，才真正理解它。

历史还会告诉你两件课本里看不出来的事。第一，很多想法会回来：1989 年的 ALVINN 和 2016 年的 PilotNet 做的是同一件事，区别在于网络、数据和算力；1997 年 Atkeson 和 Schaal 从示范中学模型、再用模型规划的思路，在今天的世界模型里还能看到。第二，机器人学习大量借用机器学习：Transformer（2017）六年后变成了 ACT 的骨干，DDPM（2020）三年后变成了 Diffusion Policy。读新论文时，能把它放进这张谱系图里，你就能更快判断它到底新在哪里。

The 30-odd lessons before this one are organized by concept: regression and backpropagation first, then generative models and RL, then imitation, sim-to-real and model-based methods. The methods did not appear in that order, though, and each was proposed to fix a specific failure. DAgger exists because behaviour cloning drifts further and further once it leaves the demonstrations; Diffusion Policy because MSE regression averages "go around on the left" and "go around on the right" into "drive straight into it"; domain randomization because policies trained in simulation broke on the real robot. Knowing what a method repairs is how you really understand it.

History also shows two things a textbook hides. First, ideas come back: ALVINN in 1989 and PilotNet in 2016 did the same thing, and what changed was the network, the data and the compute; the idea of Atkeson and Schaal in 1997, learning a model from a demonstration and planning with it, is still visible in today's world models. Second, robot learning borrows heavily from machine learning: the Transformer (2017) became the backbone of ACT six years later, and DDPM (2020) became Diffusion Policy three years later. When you read a new paper and can place it on this family tree, you can judge much faster what is actually new about it.

收录了什么。83 篇 1989–2024 年的工作，外加 7 篇标为"Part VII 从这里开始"的基础模型时代工作（空心标记）。一篇工作入选，要么是它提出了一个被后来的工作反复使用的想法，要么是它第一次在真实机器人上让某种方法站住了脚；纯机器学习的工作只在本课程讲过的机器人学习方法直接建立在它之上时才收录。这是一份精选，不是统计：漏掉的重要工作一定比收录的多。

每张卡片的约定。年份是被引用的那个版本出现的年份：会议论文用开会的年份，只在 arXiv 上发表的用 arXiv 年份；当会议年份、出版年份和 arXiv 年份不一致时，卡片上会写明（例如 ALVINN 在 1988 年的 NIPS 会议上报告、论文集 1989 年出版，通常引作 1989）。作者只写第一作者，多人合著加 "et al."；按字母顺序署名或以团队署名的论文，卡片会说明。每一条的标题、第一作者、年份和发表处都在 2026 年 10 月对照出版方、会议论文集或 arXiv 的记录核对过。

八条泳道。每篇工作有一个主领域，决定它画在哪条泳道上；如果它同时属于别的领域，那些泳道上会出现一个同色的小空心圈。八个领域是：模仿学习（琥珀色）、强化学习（蓝色）、model-based（紫色）、表示、架构与训练（青色）、生成模型（红色）、数据与硬件（绿色）、sim-to-real（与正文同色的菱形）和基础模型（与正文同色的方块）。把鼠标悬停在这些名字上，FIG 1 里对应的泳道会加上底色。约 30 篇标为核心（画得更大，带一圈外环）：如果只读 30 篇，先读这些。

What is in it. 83 works from 1989–2024, plus 7 foundation-model-era works marked "where Part VII begins" (hollow markers). A work is included if it introduced an idea that later work keeps reusing, or if it was the first to make a kind of method stand up on a real robot; pure machine-learning works are included only when a robot-learning method taught in this course builds directly on them. It is a selection, not a census: the important works left out certainly outnumber those included.

Conventions on every card. The year is the year the cited version appeared: the meeting year for conference papers, the arXiv year for papers that only appeared on arXiv; when the conference, publication and arXiv years differ, the card says so (ALVINN, for example, was presented at NIPS in 1988 and its proceedings were published in 1989; it is usually cited as 1989). Only the first author is given, with "et al." when there are more; papers with alphabetical or team author lists are flagged on the card. The title, first author, year and venue of every entry were checked against the publisher's, the proceedings' or arXiv's records in October 2026.

Eight lanes. Each work has one primary area, which decides its lane; if it also belongs to other areas, a small hollow ring of the same colour appears in those lanes. The eight areas are imitation learning (amber), reinforcement learning (blue), model-based (purple), representation, architecture & training (cyan), generative models (red), data & hardware (green), sim-to-real (diamonds in the text colour) and foundation models (squares in the text colour). Hover these names and the matching lane in FIG 1 gets a tinted background. About 30 entries are marked core (drawn larger, with an outer ring): if you read only 30 of them, read these first.

横轴是年份，每条泳道是一个领域。拖动画布左右平移；在触控板上捏合、按住 Ctrl 滚动滚轮，或用"放大 / 缩小"按钮缩放；也可以点三个时代按钮直接跳过去。点一个圆点，右边的卡片会显示它的年份、标题、作者、发表处、它贡献的那一个想法、它建立在哪些工作之上、哪些工作建立在它之上，以及讲它的那一课（少数几篇课程里没有专门讲，卡片会链接到讲这类方法的一课，并注明）。图头的筛选器可以按领域、年份范围、"只看核心"和关键词（标题、作者、发表处和简介，中英文都可以）缩小范围；不符合条件的工作会缩成小灰点。

The horizontal axis is the year and each lane is an area. Drag the canvas to pan; pinch on a trackpad, scroll with Ctrl held, or use the "zoom in / out" buttons to zoom; or jump with the three era buttons. Click a dot and the card on the right shows its year, title, authors and venue, the one idea it contributed, which works it builds on, which works build on it, and the lesson that teaches it (for the few works the course does not discuss by name, the card links to the lesson that teaches the method and says so). The filters narrow things down by area, year range, "core only" and keyword (title, authors, venue and summary, in either language); works that do not match shrink to small grey dots.

打开"只看核心"、从 2015 年开始时，剩下 21 篇（不含 Part VII）。缩放到整个范围再看一遍：1989–2011 年间每条泳道都很稀疏，而且 1989–2008 年的每一篇工作都在模仿学习、强化学习和表示这三条泳道里；除了两个小空心圈（ALVINN 和 Kohl 与 Stone 的次要领域），数据与硬件、sim-to-real 两条泳道直到 2012 年的 MuJoCo 才开始有东西，到 2016–2017 年才密起来。这不是说早年没人做仿真或数据，而是说它们在那之前还不是推动方法前进的主要力量。第 6 节会回到这一点。

With "core only" on and the start year at 2015, 21 works remain (Part VII excluded). Zoom back out to the full range: every lane is sparse from 1989 to 2011, and every work from 1989–2008 sits in the imitation, RL or representation lane; apart from two small rings (secondary areas of ALVINN and of Kohl and Stone), the data & hardware and sim-to-real lanes stay empty until MuJoCo in 2012 and only fill up from 2016–2017. That does not mean nobody did simulation or data collection earlier; it means they were not yet the main force driving the methods. Section 6 returns to this.

时间线告诉你"什么时候"，影响关系图告诉你"从哪里来"。图里的一条箭头 A → B 表示 B 直接建立在 A 之上：B 的方法里有 A 的核心部件，或者 B 明确是 A 的改进版，而且这一点在领域里是公认的（例如 DAgger → HG-DAgger、DDPM → Diffusion Policy、DQN → Double DQN、Isaac Gym → 几分钟学会走路）。它不是引用关系：B 引用 A 的原因可能只是比较或背景，那样的不画。所以这张图故意画得稀疏，只有 55 条边，没有边也不代表没有影响。

为了让 2015 年之后密集的部分展得开，这里的横轴是年份的序号（每个有工作的年份占一样宽），不是线性的年份。点一个节点：在"祖先"模式下，所有能沿箭头倒着走到的工作都会亮起来，实线箭头标出这条谱系；"后代"模式用虚线箭头标出所有顺着箭头能走到的工作；其余节点变淡。

The timeline tells you when; the influence graph tells you where from. An arrow A → B means B builds directly on A: B's method contains a core component of A, or B is explicitly an improvement of A, and this is well established in the field (DAgger → HG-DAgger, DDPM → Diffusion Policy, DQN → Double DQN, Isaac Gym → learning to walk in minutes). It is not a citation link: B may cite A just for comparison or background, and such links are not drawn. The graph is therefore deliberately sparse, with only 55 edges, and a missing edge does not mean there was no influence.

To spread out the crowded years after 2015, the horizontal axis here is the rank of the year (every year that has works gets the same width), not linear time. Click a node: in "ancestors" mode every work reachable by walking the arrows backwards lights up and solid arrows mark the lineage; "descendants" mode marks with dashed arrows every work reachable forwards; all other nodes fade.

"几分钟学会走路"有 5 个祖先：Isaac Gym 和 PPO，PPO 往回是 TRPO、策略梯度定理和 REINFORCE。一篇 2021 年的腿式机器人工作，方法上的根在 1992 年的一篇强化学习论文里。直接后代最多的是 Transformer（5 篇：BERT、GPT-3、Decision Transformer、ACT 和 RT-1）：一个为机器翻译设计的架构，成了语言模型、离线 RL、模仿学习和机器人基础模型共同的骨干。

"Walk in minutes" has 5 ancestors: Isaac Gym and PPO, and behind PPO TRPO, the policy gradient theorem and REINFORCE. A 2021 legged-robot paper has its methodological roots in a 1992 reinforcement-learning paper. The most direct descendants belong to the Transformer (5: BERT, GPT-3, Decision Transformer, ACT and RT-1): an architecture designed for machine translation became the shared backbone of language models, offline RL, imitation learning and robot foundation models.

下面 4 条导览各沿一条主线走 10–11 步，每一步说明这篇工作解决了上一步留下的什么问题。时间线会平移到当前这一步，导览路线用一条与正文同色的粗折线连起来。用"上一步 / 下一步"或点编号跳转；每一站都看过，就算完成这条导览。

Each of the four tours below follows one storyline for 10–11 steps, and each step says which problem left by the previous step this work solved. The timeline pans to the current step and the tour route is joined by a thick polyline in the text colour. Use "previous / next" or click a step number; visiting every stop completes the tour.

把 83 篇（不含 Part VII）按五年分组，按主领域堆叠起来，就是下面的 FIG 4。它画的是这份精选，不是整个领域的论文数量（第 8 节的第一个误区），但领域组成的变化还是很说明问题。

Group the 83 works (Part VII excluded) into five-year bins and stack them by primary area, and you get FIG 4 below. It shows this selection, not the number of papers in the field (the first pitfall in Section 8), but the change in the mix of areas is still telling.

那个区间是 2020–2024（28%）；在 2015–2019 年它是 19%，2010 年之前是 0。下面是这张图背后的四条规律。

1. 数据规模涨了好几个数量级，而且越来越多来自真实世界。ALVINN 主要用模拟生成的道路图像训练。2016 年，Pinto 和 Gupta 在真实机器人上自己收集了 5 万次抓取尝试（700 个机器人小时），这在当时就是"大规模"；QT-Opt 用多台真实机器人收集的抓取数据做 off-policy 训练。到了 2024 年，OpenVLA 在 97 万条真实机器人示范上训练，而这些示范来自 Open X-Embodiment 这样由许多实验室汇集的数据集。采数据本身变成了研究课题：ALOHA 的低成本遥操作、UMI 的手持夹爪（22.3 节）。

2. 算力和仿真让"试错"变便宜了。2004 年 Kohl 和 Stone 在真实的 Aibo 上优化步态，每次评估都要真的走一遍。MuJoCo（2012）让连续控制的仿真变得又快又能处理接触；Isaac Gym（2021）把物理仿真和网络训练都放到 GPU 上，报告比"CPU 仿真 + GPU 训练"的常规做法快两到三个数量级；在它上面，一个平地行走策略在单块 GPU 上不到 4 分钟就能训练好（23.1 节）。代价是现实差距，于是有了域随机化、执行器网络、教师–学生蒸馏和 RMA 这一整条泳道（23.2 节）。

3. 想法从机器学习流向机器人，有几年的延迟。在 FIG 2 里，从机器学习泳道（表示与架构、生成模型、基础模型）指向机器人方法泳道（模仿、强化学习、model-based、数据、sim-to-real）的边有 8 条（不含 Part VII），年份差的中位数是 4 年，最短 2 年（GAN → GAIL、DDPM → Diffuser），其中 6 条在 6 年以内；例外是两个更老的部件被重新用起来：VAE → ACT 的 CVAE（9 年）和 LSTM → World Models（21 年）。这只是这份精选上的一个粗略数字，但它说明了一件事：关注机器学习里刚刚成熟的生成模型和架构，往往能提前看到机器人学习下一步会用什么。

4. 旧问题换个尺度会回来。行为克隆的协变量偏移在 1989 年就存在，2011 年才被 DAgger 说清楚，到今天的大规模模仿学习里仍然是核心问题；"从数据学模型再规划"的思路从 Atkeson（1997）、PILCO（2011）一路走到 Dreamer 和 TD-MPC2。每一代新方法往往不是推翻旧问题，而是用更多数据、更好的模型类或更便宜的仿真重新回答它。

That bin is 2020–2024 (28%); in 2015–2019 it is 19%, and before 2010 it is 0. Here are four patterns behind the chart.

1. Data grew by orders of magnitude, and more and more of it comes from the real world. ALVINN trained mostly on simulated road images. In 2016 Pinto and Gupta collected 50,000 grasp attempts (700 robot hours) on a real robot themselves, which counted as "large-scale" at the time; QT-Opt trained off-policy on grasps collected by several real robots. By 2024 OpenVLA trained on 970,000 real robot demonstrations, drawn from datasets pooled by many labs such as Open X-Embodiment. Collecting data became a research topic of its own: ALOHA's low-cost teleoperation and UMI's hand-held gripper (Lesson 22.3).

2. Compute and simulation made trial and error cheap. In 2004 Kohl and Stone optimized a gait on a real Aibo, and every evaluation meant really walking. MuJoCo (2012) made continuous-control simulation fast and able to handle contact; Isaac Gym (2021) put both the physics and the network training on the GPU and reported a speed-up of two to three orders of magnitude over the usual "CPU simulator + GPU training" setup; on top of it, a flat-terrain walking policy trains in under 4 minutes on a single GPU (Lesson 23.1). The price is the reality gap, hence a whole lane of domain randomization, actuator networks, teacher–student distillation and RMA (Lesson 23.2).

3. Ideas flow from machine learning to robotics with a lag of a few years. In FIG 2 there are 8 edges from the machine-learning lanes (representation & architecture, generative, foundation) into the robot-method lanes (imitation, RL, model-based, data, sim-to-real), Part VII excluded; the median gap is 4 years, the shortest 2 (GAN → GAIL, DDPM → Diffuser), and 6 of the 8 are within 6 years; the exceptions are two older parts put back to use: VAE → ACT's CVAE (9 years) and LSTM → World Models (21 years). This is a rough number on a curated list, but it makes a point: watching which generative models and architectures have just matured in machine learning often tells you what robot learning will use next.

4. Old problems come back at a new scale. The covariate shift of behaviour cloning was already there in 1989, was made precise by DAgger in 2011, and is still a core problem in today's large-scale imitation learning; "learn a model from data, then plan" runs from Atkeson (1997) and PILCO (2011) all the way to Dreamer and TD-MPC2. A new generation of methods usually does not dissolve an old problem; it answers it again with more data, a better model class or cheaper simulation.

时间线最右边的 7 个空心标记属于下一部分。它们和前面的工作有一个共同的不同之处：先在网络规模的数据上预训练，再用到机器人上。SayCan（CoRL 2022）用大语言模型提出候选技能，用技能的价值函数判断"现在做不做得到"；RT-1（RSS 2023）在大量真实示范上训练一个 Transformer，把图像和语言指令映射成动作 token；RT-2（CoRL 2023）把动作写成文本 token，和网络上的视觉–语言数据一起微调视觉语言模型，这就是视觉–语言–动作 (VLA) 模型；Open X-Embodiment（ICRA 2024）汇集多家机构、多种机器人的数据；OpenVLA（CoRL 2024）是开源的 70 亿参数 VLA；π0（RSS 2025）在视觉语言模型上接一个用 flow matching 生成连续动作的"动作专家"；Genie（ICML 2024）则从无动作标注的网络视频学出一个可以逐帧交互的世界模型。

在 FIG 2 里你能看到它们和本课的连接：Transformer → RT-1 → RT-2 / Open X-Embodiment → OpenVLA，flow matching → π0，VQ-VAE → Genie。Part VII 会从这些工作讲起：大模型做规划、VLA、跨本体数据、世界模型和人形机器人的基础模型。

The 7 hollow markers at the right end of the timeline belong to the next part. They share one difference from everything before them: they are pre-trained on web-scale data first and then applied to robots. SayCan (CoRL 2022) has a large language model propose candidate skills and the skills' value functions judge whether they are feasible right now; RT-1 (RSS 2023) trains a Transformer on many real demonstrations to map images and language instructions to action tokens; RT-2 (CoRL 2023) writes actions as text tokens and fine-tunes a vision-language model on them together with web vision-language data, which is a vision-language-action (VLA) model; Open X-Embodiment (ICRA 2024) pools data from many institutions and many robots; OpenVLA (CoRL 2024) is an open-source 7-billion-parameter VLA; π0 (RSS 2025) attaches to a vision-language model an "action expert" that generates continuous actions with flow matching; and Genie (ICML 2024) learns, from web videos without action labels, a world model one can interact with frame by frame.

FIG 2 shows how they connect to this course: Transformer → RT-1 → RT-2 / Open X-Embodiment → OpenVLA, flow matching → π0, VQ-VAE → Genie. Part VII starts from these works: LLMs as planners, VLAs, cross-embodiment data, world models and foundation models for humanoids.