Part V · 第 17 章 网络架构 · 17.5Part V · Ch. 17 Network Architectures · 17.5

Tokenization（含动作 token 化）Tokenization (incl. Action Tokens)

Transformer 只会读写一串离散的符号（token）。文字、图像和机器人动作都得先变成 token 序列：文字用 BPE 切成子词，图像切成小块，连续的动作则可以按维度均匀分箱（RT-1、RT-2、OpenVLA 的做法）、先做 DCT 再用 BPE 压缩（FAST），或者查一本学出来的码本（VQ）。本节在浏览器里把一条机器人轨迹 token 化再还原回来，看清分辨率、序列长度和平滑性三者之间的取舍，并亲手训练一个“对 token 做分类”的动作头，看它为什么能表示多峰的示范。 A transformer can only read and write a string of discrete symbols (tokens). Text, images and robot actions must all become token sequences first: text is cut into subwords by BPE, images into patches, and continuous actions can be binned uniformly per dimension (as in RT-1, RT-2 and OpenVLA), compressed by a DCT followed by BPE (FAST), or looked up in a learned codebook (VQ). In this lesson you tokenize a robot trajectory in the browser and reconstruct it, see the trade-off between resolution, sequence length and smoothness, and train an action head that classifies tokens, to see why it can represent multimodal demonstrations.

约 40 分钟~40 min 前置：17.3 Transformer 与 Attention · 16.1 回归与分类 · 4.11 信息论入门 · 1.10 正交矩阵Prereqs: 17.3 Transformers & Attention · 16.1 Regression & Classification · 4.11 Information theory · 1.10 Orthogonal matrices 6 个交互6 interactives

    1为什么机器人需要它Why robots need this

    17.3 节的 Transformer 处理的是一串向量，而这串向量通常来自一张查找表：每个位置先是一个整数 \(k\in\{0,\dots,V-1\}\)，再查出它的嵌入向量。输出端也一样：语言模型在每个位置输出一个对 \(V\) 个整数的 softmax 分布，训练用交叉熵（16.1 节），生成时一个一个地采样（19.4 节）。这个整数就是 token，把原始数据变成 token 序列、并能从 token 序列还原回来的那一对函数，就是 tokenizer。

    对文字来说，tokenizer 早已是标准件。机器人的麻烦在于它要同时处理三种完全不同的东西：一句指令（“把红色方块放进碗里”），几路相机图像，以及一串连续的动作，比如每秒 50 次的末端位移和夹爪开合。如果想直接复用一个在互联网文本和图片上预训练好的大模型（这就是 RT-2、OpenVLA 这类“视觉-语言-动作模型”VLA 的思路），最省事的办法是把动作也变成 token，让模型像“说话”一样“说出”动作。

    可动作 token 化不是小事：切得太粗，机器人手抖、对不准；切得太细，序列太长，推理变慢，模型学起来也更难；切的方式不对，还会让轨迹在块与块之间跳变。本节依次看文字、图像和动作的 token 化，重点放在动作上：均匀分箱、FAST（DCT + BPE）和 VQ 码本，以及它们在分辨率、序列长度和平滑性上的取舍。

    The transformer of Lesson 17.3 processes a sequence of vectors, and these vectors usually come from a lookup table: each position starts as an integer \(k\in\{0,\dots,V-1\}\), and its embedding vector is looked up. The output side is the same: a language model outputs at each position a softmax distribution over the \(V\) integers, is trained with cross-entropy (Lesson 16.1) and generates by sampling them one at a time (Lesson 19.4). That integer is a token, and the pair of functions that turns raw data into a token sequence and back is a tokenizer.

    For text, tokenizers have long been standard parts. The trouble with robots is that they must handle three completely different things at once: an instruction ("put the red cube in the bowl"), images from several cameras, and a stream of continuous actions, for example end-effector displacements and gripper commands 50 times a second. If you want to reuse a large model pretrained on internet text and images (the idea behind vision-language-action models, VLAs, such as RT-2 and OpenVLA), the easiest way is to turn actions into tokens too, so that the model "speaks" its actions the way it speaks words.

    Tokenizing actions is no small matter: too coarse and the robot shakes and misses; too fine and the sequence gets long, inference slows and learning gets harder; the wrong cut and the trajectory jumps between chunks. This lesson looks at tokenizing text, images and actions in turn, with the weight on actions: uniform binning, FAST (DCT + BPE) and VQ codebooks, and their trade-offs in resolution, sequence length and smoothness.

    2直觉：token 是一本字典里的页码Intuition: a token is a page number in a dictionary

    任何 tokenizer 都在做同一件事：准备一本有 \(V\) 个条目的字典，把输入切成片段，每个片段写成它在字典里的编号。字典越大（\(V\) 越大），每个条目能表示的东西越多，同样的内容用的 token 越少；字典越小，序列越长。对文字，最常用的字典是 BPE（byte-pair encoding，字节对编码）学出来的：从单个字符开始，统计语料里哪两个相邻符号一起出现得最多，就把它们合并成一个新符号 \(\hl{merge}{(a,b)\to ab}\)，加进字典，重复若干次。常见的词因此变成一个 token，罕见的词被拆成几个常见的片段，而且任何由已知字符组成的字符串都能被编码，不会出现“未登录词”。

    下图在一小份机器人指令语料（35 句，如“pick up the red cube”）上实时训练 BPE。拖动合并次数，看同一句话的切分怎样从一个个字符变成整词。“▁”标记一个词的开头（SentencePiece 的写法），所以“▁the”和词中间的“the”是两个不同的 token。

    Every tokenizer does the same thing: it prepares a dictionary of \(V\) entries, cuts the input into pieces, and writes each piece as its entry number. The larger the dictionary (the larger \(V\)), the more each entry can stand for and the fewer tokens the same content needs; the smaller the dictionary, the longer the sequence. For text the most common dictionary is learned by BPE (byte-pair encoding): start from single characters, count which two adjacent symbols occur together most often in the corpus, merge them into a new symbol \(\hl{merge}{(a,b)\to ab}\) added to the dictionary, and repeat. Common words thereby become one token, rare words are split into a few common pieces, and any string of known characters can be encoded: there are no "out-of-vocabulary" words.

    The figure trains BPE live on a tiny corpus of robot instructions (35 sentences such as "pick up the red cube"). Drag the number of merges and watch the same sentence go from single characters to whole words. "▁" marks the start of a word (the SentencePiece convention), so "▁the" and a "the" inside a word are different tokens.

    FIG 1BPE：从字符到子词，一次合并一对BPE: from characters to subwords, one pair merge at a time
    上：当前句子被切成的 token，每个方框是一个 token（四种颜色只是为了区分相邻的方框，没有别的含义）；红色实线框是最近一次合并产生的 token，红色虚线框是语料里从未见过的字符。下：这句话的 token 数随合并次数的变化（蓝线），琥珀点是当前设置。右：已学到的合并规则，最近的在最上面。语料只有 35 句，合并到出现次数不足 2 次的字符对为止，一共能学到 103 条规则，字典大小 = 23 个基本字符 + 合并次数。 Top: the tokens the current sentence is cut into; each box is one token (the four colours only separate neighbouring boxes and mean nothing else); a solid red outline marks tokens produced by the latest merge, a dashed red outline a character never seen in the corpus. Below: the token count of this sentence against the number of merges (blue line), with the current setting as the amber dot. Right: the merge rules learned so far, latest first. With only 35 sentences, merging stops when no pair occurs at least twice, after 103 rules; the dictionary size is 23 base characters + the number of merges.
    试一试Try this
    • 用默认句子，增加合并次数，直到“gripper”成为一个 token（第 36 条合并时出现）。这时整句从 33 个字符缩到 16 个 token；到 60 次合并时只剩 7 个，一个词一个 token。With the default sentence, raise the number of merges until "gripper" becomes a single token (it happens at merge 36). The sentence has then shrunk from 33 characters to 16 tokens; by 60 merges it is down to 7, one token per word.
    • 合并次数至少 50 时，在输入框里打一个语料里没有的英文词（例如 “grippers” 或 “redcube”）：它被拆成两个以上已知的片段，而不是变成“未知词”。With at least 50 merges, type an English word the corpus never contains (for example "grippers" or "redcube"): it is split into two or more known pieces instead of becoming an "unknown word".

    图像：切成小块，每块一个 tokenImages: cut into patches, one token per patch

    图像通常不查字典，而是直接切成 \(\hl{patch}{P\times P}\) 的小块，按行优先的顺序排成一串，每块的 \(3P^2\) 个像素值经过同一个线性层变成一个嵌入向量，再加上位置编码：这就是 ViT 的做法（17.3 节讲了它的 Attention 部分）。这里的“token”是连续向量而不是整数，但它在 Transformer 里扮演的角色和文字 token 完全一样。块越小，细节越多，token 越多，而 Attention 的代价按 token 数的平方增长。

    Images usually skip the dictionary: they are cut into \(\hl{patch}{P\times P}\) patches, laid out in row-major order, and the \(3P^2\) pixel values of each patch go through one shared linear layer to become an embedding vector, plus a position encoding. This is the ViT recipe (Lesson 17.3 covers its attention part). These "tokens" are continuous vectors rather than integers, but inside the transformer they play exactly the role of text tokens. Smaller patches keep more detail and give more tokens, and the cost of attention grows with the square of the token count.

    FIG 2图像 token：块的大小决定序列长度Image tokens: the patch size sets the sequence length
    左：一张 32×32 像素的俯视图（红色方块、蓝色杯子、绿色盘子和上方的夹爪），青色网格是块的边界。右：同样这些块按行优先排成的一维序列（太长时换行显示，编号就是位置）。把鼠标移到任一边的某个块上，两边对应的块都会加上琥珀色框，读数给出它的序号。真实的 ViT 输入是 224×224，16×16 的块给出 196 个 token。 Left: a 32×32-pixel top view (red cube, blue cup, green plate and the gripper above), with the patch boundaries as a cyan grid. Right: the same patches in row-major order as a one-dimensional sequence (wrapped when long; the number is the position). Hover a patch on either side and the matching patch on both gets an amber frame; the readout gives its index. A real ViT input is 224×224, and 16×16 patches give 196 tokens.
    • 选出恰好给出 64 个 token 的块大小，再把鼠标移到红色方块所在的一个块上，读出它在序列里的位置。Choose the patch size that gives exactly 64 tokens, then hover a patch containing the red cube and read off its position in the sequence.
    核心洞见Key insight

    所有 tokenizer 都在三样东西之间做取舍：保真度（还原回来和原来差多少）、序列长度（Transformer 的计算和延迟随它增长）和字典大小（输出 softmax 有多宽，每个条目要多少数据才学得好）。BPE 用更大的字典换更短的序列；更大的图像块用更少的细节换更短的序列。动作 token 化面对的是同一个三角，只是多了一条机器人特有的要求：还原出来的轨迹必须平滑、连续。

    Every tokenizer trades among three things: fidelity (how far the reconstruction is from the original), sequence length (the transformer's compute and latency grow with it) and dictionary size (how wide the output softmax is, and how much data each entry needs to be learned well). BPE buys shorter sequences with a larger dictionary; larger image patches buy shorter sequences with less detail. Action tokenization faces the same triangle, plus one requirement peculiar to robots: the reconstructed trajectory must be smooth and continuous.

    3动手：把一条轨迹切成 token 再还原Play: cut a trajectory into tokens and back

    现在轮到动作。设机器人的末端在平面里走一条 1 秒长的轨迹，控制频率为 \(f\)：每秒发出 \(f\) 个动作，每个动作有 \(D=2\) 维（\(x\) 和 \(y\)，已归一化到大约 \([-1,1]\)）。最简单的 tokenizer 是 RT-1 和 RT-2 用的按维度均匀分箱：对每一维，把训练数据的取值范围 \([a_{\min},a_{\max}]\) 等分成 \(N\) 个箱子，宽度为 \(\hl{delta}{\Delta}=(a_{\max}-a_{\min})/N\)，一个动作值就写成它所在箱子的编号；还原时取箱子的中点。于是每个时刻产生 \(D\) 个 token，1 秒的轨迹产生 \(D\cdot f\) 个。

    下图左边是轨迹在平面里的样子，紫色圆圈是可以拖动的路径点；右边是两维随时间的变化。改箱子数、取值范围的定法、控制频率，看还原误差、token 数和“重复率”怎样变。

    Now for actions. Let the robot's end effector follow a 1-second trajectory in the plane at control rate \(f\): \(f\) actions per second, each with \(D=2\) dimensions (\(x\) and \(y\), normalized to roughly \([-1,1]\)). The simplest tokenizer is the uniform per-dimension binning of RT-1 and RT-2: for each dimension, split the training-data range \([a_{\min},a_{\max}]\) into \(N\) equal bins of width \(\hl{delta}{\Delta}=(a_{\max}-a_{\min})/N\), and write an action value as the index of its bin; to reconstruct, take the bin's midpoint. Each time step thus yields \(D\) tokens, and a 1-second trajectory \(D\cdot f\) of them.

    On the left of the figure is the trajectory in the plane, with the purple circles as draggable waypoints; on the right, both dimensions over time. Change the number of bins, how the range is set, and the control rate, and watch the reconstruction error, the token count and the "repeat rate".

    FIG 3均匀分箱：每个时刻、每一维一个 tokenUniform binning: one token per dimension per time step
    左：蓝色是连续的轨迹和它在控制频率下的采样点，琥珀色是从 token 还原的点（箱子中点），青色虚线框是两维的取值范围，框里的青色细线是箱子边界（箱子不多于 32 个时画出）。右：\(x(t)\)（红）和 \(y(t)\)（绿），琥珀色阶梯是还原值，在每个控制周期内保持不变。下方列出每一维的 token 序列，琥珀色的数字与前一个 token 相同。“含故障的示范”在数据集里加入一条 \(x\) 被传感器故障记成 4.0 的示范：它不改变这条轨迹本身，只改变取值范围。范围统计用 100 Hz 的稠密记录，所以 1% 分位数能把那一个故障样本排除在外。 Left: blue is the continuous trajectory and its samples at the control rate, amber the points reconstructed from the tokens (bin midpoints), the dashed cyan box the range of the two dimensions, and the thin cyan lines inside it the bin edges (drawn for 32 bins or fewer). Right: \(x(t)\) (red) and \(y(t)\) (green), with the reconstruction as the amber staircase, held for each control period. Below, the token sequence of each dimension; amber numbers equal the previous token. "Glitchy demo" adds to the dataset one demonstration whose \(x\) was recorded as 4.0 by a sensor fault: it does not change this trajectory, only the range. Range statistics use a dense 100 Hz recording, so the 1% quantiles can leave that single faulty sample out.
    • 找出使 RMSE 低于 0.01 的最小箱子数（默认轨迹、20 Hz 时是 64；32 个箱子时为 0.011）。注意箱子数每翻一倍，误差大约减半。Find the smallest number of bins that brings the RMSE below 0.01 (64 for the default trajectory at 20 Hz; 32 bins give 0.011). Notice that each doubling of the bins roughly halves the error.
    • 在 16 个箱子、“最小–最大”范围下打开“含故障的示范”：RMSE 从 0.023 涨到 0.065，几乎三倍；再换成“1%–99% 分位数”：回到 0.022。With 16 bins and the "min–max" range, switch on the "glitchy demo": the RMSE rises from 0.023 to 0.065, almost three times; then switch to "1–99% quantiles": back to 0.022.
    • 把控制频率提到 40 Hz 以上、箱子数不超过 16：超过 40% 的 token 和前一个完全相同（50 Hz、8 个箱子时是 70%）。Raise the control rate to 40 Hz or more with at most 16 bins: over 40% of the tokens equal the previous one exactly (70% at 50 Hz with 8 bins).
    看到了什么What you just saw

    (1) 分辨率只由箱子宽度决定：误差不超过 \(\Delta/2\)，均方根约为 \(\Delta/\sqrt{12}\)（第 6 节）。默认 8 个箱子时公式给出 0.049，实测也是 0.049。(2) 范围和箱子数一样重要：一个离群值就能把箱子撑宽好几倍，所以 OpenVLA 用 1% 和 99% 分位数定范围。分位数的代价是最极端的 1% 被截断到两端的箱子里；对这条平滑的路径，这几乎没有损失。(3) 序列长度是 \(D\cdot f\)，和箱子数无关：频率越高，token 越多，而相邻 token 也越来越像，大量 token 只是在重复前一个。下一节会看到这对自回归模型很不利。

    (1) Resolution is set by the bin width alone: the error never exceeds \(\Delta/2\), and its RMS is about \(\Delta/\sqrt{12}\) (Section 6). With the default 8 bins the formula gives 0.049 and the lab measures 0.049 too. (2) The range matters as much as the bin count: one outlier can widen every bin several times, which is why OpenVLA sets the range by the 1st and 99th percentiles. The price of quantiles is that the most extreme 1% of values is clipped into the end bins; for this smooth path that costs almost nothing. (3) The sequence length is \(D\cdot f\), independent of the bin count: the higher the rate, the more tokens, and the more alike neighbouring tokens become, until many of them merely repeat the previous one. The next section shows why that is bad for an autoregressive model.

    4另外两种动作 tokenizer：FAST 与 VQ 码本Two more action tokenizers: FAST and VQ codebooks

    FAST：先变换到频域，再压缩FAST: move to the frequency domain, then compress

    FIG 3 的重复率揭示了逐时刻分箱的根本问题。自回归模型逐个预测 token，训练信号来自“给定前面的 token，下一个是什么”。频率高时，下一个 token 几乎总等于上一个，模型只要学会“抄上一个”就能把损失降得很低，真正有信息的变化反而被淹没。FAST（Pertsch 等，2025）的论文正是这样描述高频灵巧任务上分箱的失败：模型陷入抄写前一个动作的局部最优。

    FAST 的办法借自 JPEG：对 1 秒长的动作块，每一维单独做离散余弦变换（DCT），得到 \(H\) 个频率系数 \(\hl{coef}{c_{d,k}}\)。平滑的轨迹几乎所有能量都在最低的几个频率里，高频系数接近零。把系数乘以缩放因子 \(\gamma\) 后四舍五入为整数，大部分高频系数直接变成 0。再按“先低频、各维交错”的顺序展平成一串整数，最后用在大量动作数据上训练好的 BPE 压缩：成串的 0 和常见的组合被合并成一个 token。于是 token 数取决于轨迹有多复杂，而不是控制频率有多高。

    The repeat rate of FIG 3 exposes the basic problem of per-step binning. An autoregressive model predicts one token at a time, and its training signal is "given the previous tokens, what comes next". At high rates the next token almost always equals the last one, so the model can push the loss very low simply by learning to copy, and the changes that actually carry information are drowned out. The FAST paper (Pertsch et al., 2025) describes exactly this failure of binning on high-frequency dexterous tasks: the model gets stuck in a local optimum that copies the previous action.

    FAST borrows its remedy from JPEG. For a 1-second action chunk, apply a discrete cosine transform (DCT) to each dimension separately, giving \(H\) frequency coefficients \(\hl{coef}{c_{d,k}}\). A smooth trajectory puts almost all of its energy into the lowest few frequencies, and the high-frequency coefficients are close to zero. Multiply the coefficients by a scale \(\gamma\) and round to integers, and most high-frequency coefficients become exactly 0. Flatten them into one integer string, lowest frequencies first and dimensions interleaved, and finally compress it with a BPE trained on lots of action data: runs of zeros and common combinations merge into single tokens. The token count then depends on how complex the trajectory is, not on how high the control rate is.

    FIG 4FAST：DCT、取整、展平、BPE，再反过来还原FAST: DCT, round, flatten, BPE, and back again
    左上：取整后的 DCT 系数 \(q_{d,k}=\operatorname{round}(\gamma c_{d,k})\)，两行是 \(x\)、\(y\) 两维，从左到右频率升高；琥珀色为正、蓝色为负，颜色越实绝对值越大，空格是 0。左中：按“先低频、各维交错”展平的整数串，每个方框是 BPE 之后的一个 token（相邻 token 的下划线颜色交替）。左下：同一段 1 秒动作，逐时刻分箱需要的 token 数（紫）与 FAST 的 token 数（琥珀）。右：原轨迹 \(x(t)\)（红）、\(y(t)\)（绿）与反 DCT 还原的结果（琥珀色线和点）。BPE 在 200 条随机生成的平滑示范上训练（与被编码的这三条轨迹不同），每次改变频率或 \(\gamma\) 都会在浏览器里重新训练。 Top left: the rounded DCT coefficients \(q_{d,k}=\operatorname{round}(\gamma c_{d,k})\); the two rows are the dimensions \(x\) and \(y\), frequency rising to the right; amber is positive, blue negative, more solid means larger magnitude, and empty cells are 0. Middle left: the integer string flattened "low frequencies first, dimensions interleaved"; each box is one token after BPE (underline colours alternate between neighbours). Bottom left: for the same 1-second chunk, the token count of per-step binning (purple) and of FAST (amber). Right: the original \(x(t)\) (red) and \(y(t)\) (green) and the inverse-DCT reconstruction (amber line and dots). The BPE is trained on 200 randomly generated smooth demonstrations (not the three trajectories being encoded) and is retrained in your browser whenever the rate or \(\gamma\) changes.
    • 选 50 Hz（逐时刻分箱要 100 个 token），在 BPE 打开的情况下调 \(\gamma\)，使 FAST 只用不超过 20 个 token 且 RMSE 低于 0.02（\(\gamma=10\) 时平滑轨迹是 13 个 token、RMSE 0.015）。Choose 50 Hz (per-step binning needs 100 tokens) and, with BPE on, tune \(\gamma\) so that FAST uses at most 20 tokens with an RMSE below 0.02 (at \(\gamma=10\) the smooth trajectory takes 13 tokens with an RMSE of 0.015).
    • 体会取舍：把 \(\gamma\) 降到 2 以下，token 更少但 RMSE 超过 0.04（轨迹被抹平）；再升到 30 以上（频率 ≥ 20 Hz），RMSE 低于 0.01 但 token 变多。Feel the trade-off: lower \(\gamma\) to 2 or below and the tokens get fewer but the RMSE exceeds 0.04 (the trajectory is smoothed away); raise it to 30 or more (at 20 Hz or above) and the RMSE drops below 0.01 while the tokens increase.

    两个值得注意的细节。第一，把 BPE 关掉（合并数 0），FAST 的 token 数和逐时刻分箱一样多：DCT 本身不缩短序列，它只是把信息集中到少数系数上，让大部分整数变成 0；真正缩短序列的是 BPE。第二，换成“抖动”轨迹（叠加了 7 Hz、幅度 0.04 的颤动），高频系数不再为零，token 数随之上升：FAST 的序列长度反映的是轨迹的复杂程度。频率低于颤动频率的两倍（这里是 14 Hz）时颤动会混叠，采样根本表示不了它。

    Two details are worth noticing. First, switch BPE off (0 merges) and FAST needs as many tokens as per-step binning: the DCT by itself does not shorten the sequence; it concentrates the information into a few coefficients and turns most integers into 0, and it is the BPE that shortens the sequence. Second, switch to the "tremor" trajectory (a 7 Hz shake of amplitude 0.04 on top) and the high-frequency coefficients are no longer zero, so the token count goes up: FAST's sequence length reflects how complex the trajectory is. Below twice the tremor frequency (14 Hz here) the tremor aliases, and the samples cannot represent it at all.

    VQ：学一本动作片段的码本VQ: learn a codebook of action snippets

    分箱和 FAST 都是人工设计的变换。另一条路是让数据自己决定字典：把动作切成长度为 \(L\) 的小块（一个“动作片段”），在大量示范上用 k-means（18.1 节）聚出 \(K\) 个中心 \(\hl{code}{\mathbf e_k}\)，即码本，每个片段用离它最近的码字的编号表示。这是向量量化（VQ）。VQ-VAE 把同样的最近邻查表放进一个可训练的自编码器里，码本和编码器一起学；VQ-BeT 用多级的“残差 VQ”给动作片段编码。这里用最朴素的版本：直接对动作片段做 k-means。

    动作用的是常见的增量形式：每一步的动作是末端位移 \((\Delta x,\Delta y)\)，一个片段就是 \(L\) 个连续的位移，即 \(2L\) 维向量。每个片段只需一个 token，所以 20 Hz、\(L=4\) 时每秒只有 5 个 token。代价是：\(K\) 个码字要覆盖所有可能的片段，片段越长越难覆盖，误差越大；每个片段执行完时，末端位置和真值之间会有一个跳变。本图假设机器人在每个片段开始时重新观测到自己的真实位置（闭环执行），所以误差不会跨片段累积，但会在片段边界上形成跳变。“残差 VQ”再用第二本码本量化第一本留下的误差，每个片段 2 个 token。

    Binning and FAST are hand-designed transforms. The other road is to let the data choose the dictionary: cut the actions into snippets of length \(L\), run k-means (Lesson 18.1) on lots of demonstrations to get \(K\) centres \(\hl{code}{\mathbf e_k}\), the codebook, and represent each snippet by the index of its nearest codeword. This is vector quantization (VQ). VQ-VAE puts the same nearest-neighbour lookup inside a trainable autoencoder, learning the codebook together with the encoder; VQ-BeT encodes action chunks with a multi-stage "residual VQ". Here we use the plainest version: k-means directly on action snippets.

    The actions take the common delta form: each step's action is an end-effector displacement \((\Delta x,\Delta y)\), and a snippet is \(L\) consecutive displacements, a \(2L\)-dimensional vector. Each snippet needs only one token, so at 20 Hz with \(L=4\) there are only 5 tokens per second. The price: \(K\) codewords must cover every possible snippet, which gets harder the longer the snippet, so the error grows; and when a snippet ends, the end effector is off from the true position by a jump. The figure assumes the robot re-observes its true position at the start of each snippet (closed-loop execution), so errors do not accumulate across snippets, but they show up as jumps at the snippet boundaries. "Residual VQ" quantizes what the first codebook left over with a second codebook, at 2 tokens per snippet.

    FIG 5VQ 码本：一个 token 代表一整段动作A VQ codebook: one token for a whole snippet of motion
    左：测试轨迹（不在训练集里，20 Hz）。蓝色是真值，空心圆是每个片段的起点；琥珀色是从码字还原的各个片段；红色短线是片段结束时的位置与下一段真实起点之间的跳变。右：第一级码本，每格画出一个码字代表的动作片段（从格子中心出发的 \(L\) 步折线，所有格子用同一比例），琥珀色框是测试轨迹用到的码字，虚线框是没有分到任何训练片段的“死码字”。码本在 300 条随机示范上训练，按“训练”后每帧在约 6 ms 的预算内做若干次 Lloyd 迭代，直到收敛。 Left: the test trajectory (not in the training set, 20 Hz). Blue is the truth, with hollow circles at the start of each snippet; amber the snippets reconstructed from their codewords; the short red segments are the jumps between where a snippet ends and where the next one truly starts. Right: the first-stage codebook; each cell draws the snippet one codeword stands for (an \(L\)-step polyline from the cell centre, all cells at the same scale); an amber frame marks codewords the test trajectory uses, a dashed frame a "dead" codeword that no training snippet was assigned to. The codebook is trained on 300 random demonstrations; after "Train" each frame runs as many Lloyd iterations as fit in about 6 ms, until convergence.
    • 用 \(K=16\)、\(L=4\)、单级，训练到收敛：RMSE 约 0.14。再切到“2 级（残差）”并训练：每个片段多用一个 token，RMSE 降到一半左右（约 0.08）。With \(K=16\), \(L=4\) and one stage, train to convergence: RMSE about 0.14. Then switch to "2 stages (residual)" and train: one more token per snippet roughly halves the RMSE (to about 0.08).
    • 把片段长度改为 \(L=10\)（每秒只剩 2 个 token），\(K=16\)，训练到收敛：片段边界上的平均跳变约 0.8（\(L=4\) 时约 0.23），轨迹明显“断成几截”。Set the snippet length to \(L=10\) (only 2 tokens per second) with \(K=16\) and train to convergence: the mean jump at snippet boundaries is about 0.8 (about 0.23 at \(L=4\)), and the trajectory visibly "breaks into pieces".
    一个想法，三种切法One idea, three ways to cut

    分箱：每个时刻、每一维独立量化。简单、可解释、不需要训练，但序列长度 \(D\cdot f\) 随频率线性增长，相邻 token 高度冗余。 FAST：先用一个固定的正交变换（DCT）去掉冗余，再用 BPE 做无损压缩。token 数随轨迹复杂度变化，误差由 \(\gamma\) 控制，可证明有界。 VQ：从数据里学一本码本，一个 token 代表一整段动作，序列最短，但码本外的动作表示不好，片段边界会跳。三者在第 2 节的三角里各占一个角：分箱保真但长，VQ 短但粗，FAST 在中间，还能用 \(\gamma\) 调节。

    Binning: quantize each time step and each dimension independently. Simple, interpretable, no training, but the sequence length \(D\cdot f\) grows linearly with the rate and neighbouring tokens are highly redundant. FAST: remove the redundancy with a fixed orthogonal transform (the DCT) first, then compress losslessly with BPE. The token count follows the trajectory's complexity, and the error is set by \(\gamma\) with a provable bound. VQ: learn a codebook from data; one token stands for a whole snippet, giving the shortest sequences, but actions outside the codebook are represented poorly and snippet boundaries jump. The three sit in different corners of the triangle of Section 2: binning is faithful but long, VQ short but coarse, and FAST in between, adjustable through \(\gamma\).

    5再一个视角：token 给出的是一个分布One more view: tokens give you a distribution

    把动作离散化除了能复用语言模型，还有一个更深的好处。回归网络用 MSE 训练，学到的是条件均值 \(\mathbb E[a\mid s]\)（16.2 节）。如果示范者有时从左边绕过障碍、有时从右边，均值就落在两者中间，正好撞上障碍。而对 token 做分类时，网络输出的是一个完整的分布 \(\hl{prob}{p_\theta(k\mid s)}\)，两个峰可以同时存在。这是“多峰”问题的第一种解法；22.1 节会更仔细地讲 BC 为什么会平均掉多个模式，22.2 节的扩散策略是另一种解法。

    下图的状态 \(s\) 表示障碍相对机器人的位置，动作 \(a\) 是横向的转向量。\(s\) 越大，示范者越倾向于从右边（\(a>0\)）绕；\(s\) 在 0 附近时左右各占一半。左右两个模式之间 \(|a|<0.3\) 的红色带子代表撞上障碍。两个网络（同样的 [1, 32, 32] tanh 隐藏层）在同一批数据上实时训练：一个回归 \(a\)，一个把 \(a\) 均匀分成 \(B\) 个箱子后做分类。

    Discretizing actions has a deeper benefit than reusing language models. A regression network trained with MSE learns the conditional mean \(\mathbb E[a\mid s]\) (Lesson 16.2). If the demonstrator sometimes passes an obstacle on the left and sometimes on the right, the mean falls in between, straight into the obstacle. A network that classifies tokens instead outputs a whole distribution \(\hl{prob}{p_\theta(k\mid s)}\), in which both peaks can live at once. This is the first cure for the "multimodality" problem; Lesson 22.1 looks more closely at why BC averages modes, and the diffusion policy of Lesson 22.2 is another cure.

    In the figure the state \(s\) is where the obstacle sits relative to the robot, and the action \(a\) is a lateral steering command. The larger \(s\), the more the demonstrator prefers to pass on the right (\(a>0\)); near \(s=0\) it is half and half. The red band \(|a|<0.3\) between the two modes stands for hitting the obstacle. Two networks (the same [1, 32, 32] tanh hidden layers) train live on the same data: one regresses \(a\), the other classifies \(a\) after splitting it uniformly into \(B\) bins.

    FIG 6回归平均掉两个模式，分类把它们都留下Regression averages the two modes, classification keeps both
    左：横轴是状态 \(s\)，纵轴是动作 \(a\)。灰点是 400 个示范样本；青色格子是分类网络给出的 \(p_\theta(k\mid s)\)（越青概率越大：颜色强度正比于每个箱子的概率相对均匀分布 \(1/B\) 的倍数，8 倍以上为满色）；紫线是回归网络；琥珀色是按所选规则从分布里解码出的动作（取最大、采样或取期望）；红色带子是障碍。琥珀色竖虚线和它的把手可以左右拖动，选择右图查看的状态 \(s^\ast\)。右：\(s^\ast\) 处的分布（青色柱），紫色竖线是回归的输出，琥珀色竖线是解码出的动作。 Left: state \(s\) across, action \(a\) up. Grey dots are the 400 demonstration samples; cyan cells are the classifier's \(p_\theta(k\mid s)\) (more cyan = more probable: the colour strength is proportional to each bin's probability relative to the uniform \(1/B\), full at 8 times or more); the purple line is the regression network; amber is the action decoded from the distribution by the chosen rule (argmax, sample or expectation); the red band is the obstacle. The amber dashed vertical line and its handle can be dragged sideways to choose the state \(s^\ast\) shown on the right. Right: the distribution at \(s^\ast\) (cyan bars), the regression output as the purple vertical line and the decoded action as the amber one.
    • 按“训练”至少 1500 步：在 \(s=0\) 处，分类网络把至少 20% 的概率分给左边（\(a<0\)），也把至少 20% 分给右边；回归网络的输出却停在两者之间的红色带子里。Press "Train" for at least 1500 steps: at \(s=0\) the classifier gives at least 20% of the probability to the left (\(a<0\)) and at least 20% to the right, while the regression output sits between them, inside the red band.
    • 训练至少 1500 步后，把解码规则改为“期望”：在 \(s=0\) 处，\(\sum_k p_k\hat a_k\) 也落进了红色带子（\(|a|<0.3\)），和回归一样撞上障碍。After at least 1500 training steps, set the decoding rule to "expectation": at \(s=0\), \(\sum_k p_k\hat a_k\) also falls into the red band (\(|a|<0.3\)) and hits the obstacle just like the regression.
    反 token 化也是设计的一部分De-tokenization is part of the design

    有了分布还得决定怎么把它变回一个动作。取最大（argmax）总选同一个模式，结果确定，但在两个模式概率接近时，\(s\) 的微小变化会让它在两个模式之间来回跳。采样真正保留了多峰性，但相邻两步可能各抽到一边，所以通常和动作分块（一次采样一整段，22.2 节）配合。取期望 \(\hl{mean}{\sum_k p_k\hat a_k}\) 看起来“更精确”，却正好把多峰性又平均掉了：它最小化的就是平方误差，和回归是一回事。

    Having a distribution, you still have to decide how to turn it back into an action. Argmax always picks one mode and is deterministic, but where the two modes are nearly equally likely a tiny change in \(s\) flips it from one mode to the other. Sampling truly keeps the multimodality, but two consecutive steps may draw opposite sides, so it is usually paired with action chunking (sample a whole chunk at once, Lesson 22.2). The expectation \(\hl{mean}{\sum_k p_k\hat a_k}\) looks "more precise" but averages the multimodality right back out: it minimizes squared error, which is exactly what regression does.

    6再看数学The math

    提示：把鼠标悬停在公式中带底色的符号上，图中对应的元素会被高亮（最近一次合并、图像块网格、箱子边界、DCT 系数、码本、分类分布、期望解码）。Tip: hover the highlighted symbols in the formulas; matching elements in the figures light up (the latest merge, the patch grid, the bin edges, the DCT coefficients, the codebook, the class distribution, the expectation decoding).

    tokenizer 与训练目标Tokenizer and training objective

    一个 tokenizer 是一对映射：编码器 \(\mathcal E\) 把数据 \(\mathbf x\)（一句话、一张图、一段动作）变成整数序列 \(k_{1:n}\)，\(k_i\in\{0,\dots,V-1\}\)；解码器 \(\mathcal D\) 把整数序列变回数据 \(\hat{\mathbf x}=\mathcal D(\mathcal E(\mathbf x))\)。文字的 BPE 是无损的（\(\hat{\mathbf x}=\mathbf x\)），动作 tokenizer 都是有损的。自回归模型按链式法则分解序列的概率（19.4 节），以观测 \(o\)（图像和指令的 token）为条件，用交叉熵训练：

    A tokenizer is a pair of maps: an encoder \(\mathcal E\) turns data \(\mathbf x\) (a sentence, an image, a stretch of actions) into an integer sequence \(k_{1:n}\) with \(k_i\in\{0,\dots,V-1\}\), and a decoder \(\mathcal D\) turns it back, \(\hat{\mathbf x}=\mathcal D(\mathcal E(\mathbf x))\). Text BPE is lossless (\(\hat{\mathbf x}=\mathbf x\)); action tokenizers are all lossy. An autoregressive model factorizes the sequence probability by the chain rule (Lesson 19.4), conditioned on the observation \(o\) (the image and instruction tokens), and is trained with cross-entropy:

    $$ p_\theta(k_{1:n}\mid o)=\prod_{i=1}^{n}p_\theta(k_i\mid k_{<i},o) $$ $$ \mathcal L(\theta)=-\sum_{i=1}^{n}\log p_\theta(k_i\mid k_{<i},o) $$

    每个 token 最多携带 \(\log_2V\) 比特（4.11 节）。动作 tokenizer 的“码率”是每秒 token 数乘以每个 token 的比特数；推理时每个 token 需要一次前向计算，所以每秒 token 数也直接决定了自回归解码的延迟。

    Each token carries at most \(\log_2V\) bits (Lesson 4.11). The "bit rate" of an action tokenizer is tokens per second times bits per token; at inference every token costs one forward pass, so tokens per second also directly sets the latency of autoregressive decoding.

    BPE 与图像块BPE and image patches

    BPE 从基本符号表（字符或字节）出发，每一步找出语料中相邻出现次数最多的一对符号，把它合并成新符号，直到达到预定的字典大小（\(\#(a\,b)\) 是语料里 \(b\) 紧跟在 \(a\) 后面出现的次数）：

    BPE starts from a base alphabet (characters or bytes) and at each step finds the pair of adjacent symbols that occurs most often in the corpus and merges it into a new symbol, until the planned dictionary size is reached (\(\#(a\,b)\) is the number of times \(b\) directly follows \(a\) in the corpus):

    $$ \hl{merge}{(a,b)}=\arg\max_{(a,b)}\ \#(a\,b),\qquad a\,b\ \to\ ab,\qquad V\leftarrow V+1 $$

    编码新文本时按学到的顺序依次应用这些合并。以字节作为基本符号（GPT-2 的“字节级 BPE”）时，任何字符串都能编码，FIG 1 里红色虚线框那种“没见过的字符”就不会出现。图像没有字典，\(H\times W\times3\) 的图片切成 \(n=HW/\hl{patch}{P}^2\) 块，每块拉直成 \(3P^2\) 维向量 \(\mathbf x_i\)，经过同一个线性层 \(\mathbf z_i=W_e\mathbf x_i+\mathbf p_i\)（\(\mathbf p_i\) 是位置编码）。注意力的代价约为 \(n^2\)，所以把 \(P\) 减半，token 数变成 4 倍，注意力的代价变成 16 倍。

    New text is encoded by applying the learned merges in the order they were learned. With bytes as the base alphabet (GPT-2's "byte-level BPE") every string can be encoded, and "unseen characters" like the dashed red boxes in FIG 1 cannot occur. Images have no dictionary: an \(H\times W\times3\) picture is cut into \(n=HW/\hl{patch}{P}^2\) patches, each flattened to a \(3P^2\)-vector \(\mathbf x_i\) and passed through one shared linear layer, \(\mathbf z_i=W_e\mathbf x_i+\mathbf p_i\) (\(\mathbf p_i\) the position encoding). Attention costs about \(n^2\), so halving \(P\) quadruples the token count and multiplies the attention cost by 16.

    均匀分箱Uniform binning

    对动作的每一维 \(d\)，用训练数据定出范围 \([a_{\min},a_{\max}]\)（最小–最大，或 1% 与 99% 分位数），分成 \(N\) 个箱子：

    For each action dimension \(d\), set the range \([a_{\min},a_{\max}]\) from the training data (min–max, or the 1st and 99th percentiles) and split it into \(N\) bins:

    $$ \hl{delta}{\Delta}=\frac{a_{\max}-a_{\min}}{N},\qquad k=\min\!\Big(N-1,\ \max\!\Big(0,\Big\lfloor\frac{a-a_{\min}}{\Delta}\Big\rfloor\Big)\Big),\qquad \hat a=a_{\min}+\big(k+\tfrac12\big)\Delta $$

    范围内的值误差 \(|a-\hat a|\le\Delta/2\)。如果值在每个箱子里大致均匀地分布，误差就是 \([-\Delta/2,\Delta/2]\) 上的均匀分布，方差 \(\Delta^2/12\)，均方根 \(\Delta/\sqrt{12}\approx0.29\Delta\)：箱子数翻倍，误差减半，代价只是每个 token 多 1 比特。范围外的值被截断到端点的箱子里，误差可以任意大，所以分位数范围只适合截掉真正罕见的值。每秒的 token 数是 \(D\cdot f\)，与 \(N\) 无关。

    Inside the range the error obeys \(|a-\hat a|\le\Delta/2\). If values spread roughly evenly within each bin, the error is uniform on \([-\Delta/2,\Delta/2]\) with variance \(\Delta^2/12\) and RMS \(\Delta/\sqrt{12}\approx0.29\Delta\): doubling the bins halves the error at the cost of just 1 more bit per token. Values outside the range are clipped into the end bins, with arbitrarily large error, so a quantile range should only cut off values that are truly rare. The tokens per second are \(D\cdot f\), independent of \(N\).

    FAST：正交变换 + 取整 + 无损压缩FAST: orthogonal transform + rounding + lossless compression

    对一维动作 \(a_0,\dots,a_{H-1}\)，正交归一的 DCT-II 及其逆变换是

    For one action dimension \(a_0,\dots,a_{H-1}\), the orthonormal DCT-II and its inverse are

    $$ \hl{coef}{c_k}=s_k\sum_{n=0}^{H-1}a_n\cos\frac{\pi(n+\frac12)k}{H},\qquad a_n=\sum_{k=0}^{H-1}s_k\,c_k\cos\frac{\pi(n+\frac12)k}{H},\qquad s_0=\sqrt{\tfrac1H},\ s_{k>0}=\sqrt{\tfrac2H} $$ $$ q_k=\operatorname{round}(\gamma\,c_k),\qquad \hat c_k=q_k/\gamma $$ $$ \sum_n(a_n-\hat a_n)^2=\sum_k(c_k-\hat c_k)^2\le\frac{H}{4\gamma^2}\ \Longrightarrow\ \sqrt{\tfrac1H\textstyle\sum_n(a_n-\hat a_n)^2}\le\frac{1}{2\gamma} $$

    写成矩阵，DCT 就是一个 \(H\times H\) 的正交矩阵（1.10 节），它保持长度，所以系数上的误差平方和等于时间域里的误差平方和（Parseval 等式）。每个系数取整的误差不超过 \(1/(2\gamma)\)，于是每个样本的均方根误差不超过 \(1/(2\gamma)\)，与 \(H\) 无关：\(\gamma=10\) 时上界是 0.05，FIG 4 里各种设置下实测在 0.014 到 0.029 之间（被取成 0 的小系数误差小于上界）。平滑信号的 \(c_k\) 随 \(k\) 增大迅速变小，满足 \(|\gamma c_k|<\tfrac12\) 的系数都变成 0，非零整数的个数大致只取决于轨迹里有多少“弯”，这才是 BPE 能压缩的东西。FAST 把系数矩阵按列展平（先放所有维的第 0 个系数，再放所有维的第 1 个……），论文的解释是先预测低频成分，生成的轨迹更稳定。

    As a matrix, the DCT is an \(H\times H\) orthogonal matrix (Lesson 1.10); it preserves length, so the sum of squared errors over the coefficients equals the one over time (Parseval's identity). Rounding errs by at most \(1/(2\gamma)\) per coefficient, so the per-sample RMS error is at most \(1/(2\gamma)\), independent of \(H\): at \(\gamma=10\) the bound is 0.05, and FIG 4 measures between 0.014 and 0.029 across its settings (small coefficients rounded to 0 err by less than the bound). For a smooth signal \(c_k\) shrinks quickly with \(k\), every coefficient with \(|\gamma c_k|<\tfrac12\) becomes 0, and the number of nonzero integers depends roughly only on how many "bends" the trajectory has. That is what BPE can compress. FAST flattens the coefficient matrix column by column (first coefficient 0 of every dimension, then coefficient 1 of every dimension, and so on); the paper's explanation is that predicting the low-frequency parts first makes the generated trajectories more stable.

    向量量化Vector quantization

    码本 \(\{\hl{code}{\mathbf e_k}\}_{k=1}^{K}\subset\R^{2L}\) 由 k-means 学出，它最小化训练片段 \(\mathbf v_j\) 到最近码字的平方距离；编码就是最近邻查表。残差 VQ 再用第二本码本量化剩余误差：

    The codebook \(\{\hl{code}{\mathbf e_k}\}_{k=1}^{K}\subset\R^{2L}\) is learned by k-means, minimizing the squared distance from each training snippet \(\mathbf v_j\) to its nearest codeword; encoding is the nearest-neighbour lookup. Residual VQ quantizes the remaining error with a second codebook:

    $$ \min_{\mathbf e_{1:K}}\sum_j\min_k\|\mathbf v_j-\mathbf e_k\|^2 $$ $$ k^{(1)}=\arg\min_k\|\mathbf v-\mathbf e^{(1)}_k\|,\qquad k^{(2)}=\arg\min_k\|\mathbf v-\mathbf e^{(1)}_{k^{(1)}}-\mathbf e^{(2)}_k\| $$

    每级每个片段 1 个 token、\(\log_2K\) 比特，每秒 \(S f/L\) 个 token（\(S\) 是级数）。FIG 5 里 \(K=16\)、\(L=4\)、单级的码率只有每秒 \(5\times4=20\) 比特；同样 20 Hz 下 256 个箱子的分箱是 \(40\times8=320\) 比特。码率低当然有代价：这本码本只能表示 16 种“动作片段”。

    Each stage costs 1 token of \(\log_2K\) bits per snippet, so there are \(Sf/L\) tokens per second (\(S\) the number of stages). In FIG 5, \(K=16\), \(L=4\) and one stage give a bit rate of only \(5\times4=20\) bits per second; binning with 256 bins at the same 20 Hz uses \(40\times8=320\). The low rate has its price: this codebook can express only 16 kinds of "snippet".

    分类头与解码The classification head and decoding

    $$ \hl{prob}{p_\theta(k\mid s)}=\operatorname{softmax}\big(\mathbf z_\theta(s)\big)_k,\qquad \mathcal L=-\log p_\theta\big(k(a)\mid s\big) $$ $$ \bar a=\hl{mean}{\textstyle\sum_k p_k\,\hat a_k}=\arg\min_{\hat a}\ \mathbb E_{k\sim p}\big[(\hat a_k-\hat a)^2\big] $$

    交叉熵的最优解是 \(p_\theta(k\mid s)\) 等于数据里 \(a\) 落进第 \(k\) 个箱子的条件概率，多少个峰都能表示。而期望解码最小化的是平方误差，它和 MSE 回归的最优解 \(\mathbb E[a\mid s]\) 只差箱子的量化误差，所以又回到了“平均掉模式”的老问题。也要注意：交叉熵不知道箱子是有顺序的。把真值在第 128 个箱子的动作预测成第 129 个，和预测成第 0 个，受到的惩罚完全一样，网络必须从数据里自己学到“相邻箱子相似”。

    The optimum of cross-entropy sets \(p_\theta(k\mid s)\) to the conditional probability that the data's \(a\) falls into bin \(k\), with as many peaks as needed. Expectation decoding minimizes squared error instead, and it differs from the MSE regression optimum \(\mathbb E[a\mid s]\) only by the binning error, which brings back the old problem of averaging the modes. Note also that cross-entropy does not know the bins are ordered: predicting bin 129 for an action whose true bin is 128 is penalized exactly as much as predicting bin 0, so the network must learn from the data alone that neighbouring bins are similar.

    7机器人中的它：VLA 怎样“说出”动作In robotics: how VLAs "speak" actions

    下表是几种有代表性的做法。它们几乎都从“每维 256 个均匀箱子”开始，后来的工作主要在改范围（分位数）、改序列长度（FAST、VQ）或干脆不用离散 token（ACT、扩散策略）。

    The table lists some representative designs. Almost all start from "256 uniform bins per dimension"; later work mostly changes the range (quantiles), the sequence length (FAST, VQ), or drops discrete tokens altogether (ACT, diffusion policies).

    工作Work动作怎样变成 tokenHow actions become tokens
    Trajectory Transformer (Janner et al., 2021)把状态、动作和回报的每一维分别离散化，整条轨迹当作一个序列建模。Discretizes every dimension of states, actions and rewards separately and models the whole trajectory as one sequence.
    Gato (Reed et al., 2022)连续值先做 μ-law 压扩到 \([-1,1]\)，再均匀分成 1024 个箱子，箱子的编号排在文字 token 之后，和文字共用同一个 Transformer。Continuous values are μ-law companded into \([-1,1]\) and then binned uniformly into 1024 bins, whose indices are placed after the text tokens, sharing one transformer with text.
    RT-1 (Brohan et al., 2022)机械臂、移动底座和模式切换的每个动作维度各分成 256 个均匀箱子；模型一次输出当前时刻的全部动作 token。Every action dimension (arm, mobile base, and a mode switch) is split into 256 uniform bins; the model outputs all action tokens for the current step.
    RT-2 (Brohan et al., 2023)同样的 256 个箱子，但直接写成语言模型的文字 token：PaLI-X 版本用表示整数的 token，PaLM-E 版本挪用 256 个最少使用的 token，让 VLM 原样“说出”动作。The same 256 bins, but written directly as text tokens of the language model: the PaLI-X version uses the tokens for integers, the PaLM-E version takes over the 256 least-used tokens, so the VLM "says" actions as it is.
    OpenVLA (Kim et al., 2024)7 维动作，每维 256 个箱子，范围取训练数据的 1% 和 99% 分位数（FIG 3 的做法），覆盖 Llama 分词器里 256 个最少使用的 token。7-dimensional actions, 256 bins each, with the range set by the 1st and 99th percentiles of the training data (the FIG 3 recipe), overwriting the 256 least-used tokens of the Llama tokenizer.
    BeT (Shafiullah et al., 2022)用 k-means 把动作聚成若干类，Transformer 预测类别（离散，处理多峰）再加一个连续的偏移量（补回精度）。Clusters actions with k-means; the transformer predicts the cluster (discrete, handling multimodality) plus a continuous offset (restoring precision).
    VQ-BeT (Lee et al., 2024)用残差 VQ 把动作块编码成几级码字，再预测码字和偏移量。Encodes action chunks with residual VQ into a few levels of codewords, then predicts the codes and an offset.
    FAST / π0-FAST (Pertsch et al., 2025)DCT + 取整 + BPE；另外发布了在约 100 万条 1 秒动作块上训练的通用版 FAST+。论文报告，与扩散式 VLA 性能相当，训练时间最多缩短 5 倍。DCT + rounding + BPE; also released a universal FAST+ trained on about 1M one-second action chunks. The paper reports performance matching diffusion VLAs with up to 5 times less training time.
    ACT (Zhao et al., 2023)不做离散化：一次回归出一整块连续动作（动作分块），并用时间集成平滑块与块之间的衔接，见 22.2 节。No discretization: regresses a whole chunk of continuous actions at once (action chunking) and smooths the transitions between chunks by temporal ensembling; see Lesson 22.2.

    FAST 论文的表 I 给出了 1 秒动作块的 token 数，可以和 FIG 3 的 \(D\cdot f\) 公式对照：BridgeV2（7 维、5 Hz）逐步分箱 35 个、FAST 20 个；DROID（7 维、15 Hz）105 对 29；叠 T 恤（双臂 14 维、50 Hz）700 对 53。分箱的 token 数严格等于 \(D\cdot f\)，而 FAST 在不同频率下大致都在每条手臂 20 到 30 个之间，和 FIG 4 看到的一样：它跟着轨迹的复杂度走，不跟着频率走。

    Table I of the FAST paper lists token counts for 1-second action chunks, which you can check against FIG 3's \(D\cdot f\) formula: BridgeV2 (7 dimensions, 5 Hz) needs 35 tokens with per-step binning versus 20 with FAST; DROID (7 dimensions, 15 Hz) 105 versus 29; T-shirt folding (bimanual, 14 dimensions, 50 Hz) 700 versus 53. The binning counts are exactly \(D\cdot f\), while FAST stays at roughly 20 to 30 per arm across rates, just as in FIG 4: it follows the trajectory's complexity, not the rate.

    真实系统里In real systems

    序列预算：一个 VLA 的输入通常是几路相机，每路几百个图像 token（224×224、14×14 的块是 256 个），加上几十个指令 token；动作 token 排在最后逐个生成。逐步分箱时，一步 7 个 token 意味着 7 次顺序的前向计算，这限制了控制频率，也是 FAST 和动作分块想要解决的问题之一。 归一化统计量是模型的一部分：分箱和 FAST 的范围、分位数、\(\gamma\) 都是从训练数据算出来的，部署时必须用同一套统计量反 token 化；换一台机器人、换一个数据集，往往要重新计算。 夹爪：夹爪开合常是接近二值的阶跃，对分箱无所谓，但对 DCT 来说阶跃需要很多高频系数，还原时会在跳变两侧出现振荡（吉布斯现象）。 不离散化的路线：π0 用流匹配（19.7 节）直接生成连续的动作块，Diffusion Policy 用扩散（22.2 节），它们不需要动作 tokenizer，但也就不能直接复用语言模型的输出头。

    The sequence budget: a VLA's input usually has several cameras with a few hundred image tokens each (224×224 with 14×14 patches gives 256), plus a few dozen instruction tokens; the action tokens come last and are generated one by one. With per-step binning, 7 tokens per step means 7 sequential forward passes, which limits the control rate; this is one of the problems FAST and action chunking set out to solve. Normalization statistics are part of the model: the ranges, quantiles and \(\gamma\) of binning and FAST are computed from the training data, and deployment must de-tokenize with exactly the same statistics; a new robot or dataset often needs them recomputed. The gripper: gripper commands are often near-binary steps; binning does not mind, but for the DCT a step needs many high-frequency coefficients, and the reconstruction rings on both sides of the jump (the Gibbs phenomenon). The no-discretization route: π0 generates continuous action chunks directly by flow matching (Lesson 19.7) and Diffusion Policy by diffusion (Lesson 22.2); they need no action tokenizer, but they also cannot reuse the language model's output head as it is.

    8常见误区Common pitfalls

    误区 1：用最小–最大值定范围Pitfall 1: setting the range by min and max

    真实数据里总有传感器故障、遥操作时的猛拉、单位写错的示范。FIG 3 里一个故障样本就让 \(x\) 维的箱子宽了三倍，正常动作的误差几乎变成三倍。用分位数定范围（OpenVLA 用 1% 和 99%），或者先清洗数据。反过来，分位数会截断真正极端的动作；如果任务需要偶尔的大动作（快速撤回），要检查它们没有被截掉。

    Real data always contains sensor faults, jerks during teleoperation, and demos recorded in the wrong units. In FIG 3 a single faulty sample made the \(x\) bins three times wider and nearly tripled the error of normal actions. Set the range by quantiles (OpenVLA uses 1% and 99%) or clean the data first. Conversely, quantiles clip truly extreme actions; if the task needs the occasional large motion (a quick retreat), check that it is not clipped away.

    误区 2：以为箱子越多越好、频率越高越好Pitfall 2: assuming more bins and higher rates are always better

    箱子多了，每个箱子分到的训练样本就少，softmax 更宽，而交叉熵又不知道相邻箱子相似，所以需要更多数据才能学好。频率高了，逐步分箱的序列按 \(D\cdot f\) 变长，而且大部分 token 只是重复前一个（FIG 3），模型容易学成“抄上一个”。256 个箱子是常见的默认值，不是定律；高频任务应该先考虑 FAST 这类压缩，或者动作分块。

    More bins mean fewer training samples per bin and a wider softmax, and cross-entropy does not know neighbouring bins are similar, so more data is needed to learn them. Higher rates make per-step binning sequences grow as \(D\cdot f\), with most tokens merely repeating the previous one (FIG 3), and the model is tempted to learn "copy the last one". 256 bins is a common default, not a law; for high-rate tasks consider compression such as FAST, or action chunking, first.

    误区 3：用期望值解码多峰分布Pitfall 3: decoding a multimodal distribution by its expectation

    FIG 6：期望解码最小化平方误差，和回归一样把左右两个模式平均成正中间。想保留多峰性就要采样或取最大，并且要保证一段动作内部的选择一致（一次采样一整块），否则机器人会在两个模式之间来回切换，哪个都完成不了。

    FIG 6: expectation decoding minimizes squared error and, like regression, averages the left and right modes into the middle. To keep the multimodality, sample or take the argmax, and keep the choice consistent within a stretch of motion (sample a whole chunk at once); otherwise the robot switches back and forth between modes and completes neither.

    误区 4：不处理“非法”的 token 序列Pitfall 4: not handling "illegal" token sequences

    语言模型可以在动作的位置上输出一个文字 token，或者在 FAST 里生成一串解码后系数个数不等于 \(D\times H\) 的 token。部署时要约束解码（只允许动作 token 出现在动作位置），并对 FAST 解码出的系数做补零或截断。分箱 tokenizer 也一样：输出的编号必须落在 \(0,\dots,N-1\) 里。

    A language model can emit a text token where an action belongs, or, with FAST, a token string that decodes to a coefficient count other than \(D\times H\). At deployment, constrain decoding (only action tokens allowed at action positions) and pad or truncate the coefficients FAST decodes. Binning tokenizers need the same care: an output index must lie in \(0,\dots,N-1\).

    误区 5：忽略块与块之间的衔接Pitfall 5: ignoring the seams between chunks

    一个 token 代表的动作越长（VQ 的片段、FAST 的 1 秒块、动作分块），块内越平滑，块与块之间的接缝就越显眼：FIG 5 里 \(K=16\)、\(L=10\) 时片段边界的平均跳变约 0.8，是 \(L=4\) 时的三倍多。常见的补救：让块之间重叠并做时间集成（ACT），只执行每块的前一部分再重新规划（滚动时域，Diffusion Policy），或者在下一块的条件里加入上一块的结尾。

    The longer the motion a token stands for (a VQ snippet, a FAST 1-second chunk, an action chunk), the smoother the inside of a chunk and the more visible the seams between chunks: in FIG 5 with \(K=16\) and \(L=10\) the mean jump at snippet boundaries is about 0.8, more than three times that at \(L=4\). Common remedies: overlap the chunks and ensemble them over time (ACT), execute only the first part of each chunk and replan (receding horizon, Diffusion Policy), or condition the next chunk on the end of the previous one.

    9小练习Check yourself

    1. 一条 7 维动作的机械臂以 50 Hz 控制，每维 256 个均匀箱子，逐时刻分箱。1 秒的动作需要多少个 token？1. An arm with 7-dimensional actions runs at 50 Hz with 256 uniform bins per dimension and per-step binning. How many tokens does 1 second of action take?
    每个时刻每一维一个 token：\(D\cdot f=7\times50=350\)。箱子数 256 只决定每个 token 能取多少个值（8 比特），不影响 token 个数（选项 D 把两者混淆了）。FAST 论文里 14 维、50 Hz 的叠 T 恤任务正好是 700 个。One token per dimension per time step: \(D\cdot f=7\times50=350\). The 256 bins only set how many values each token can take (8 bits), not how many tokens there are (option D confuses the two). The 14-dimensional, 50 Hz T-shirt folding task in the FAST paper comes to exactly 700.
    2. 为什么 OpenVLA 用训练数据的 1% 和 99% 分位数，而不是最小值和最大值，来确定每一维的箱子范围？2. Why does OpenVLA set each dimension's bin range by the 1st and 99th percentiles of the training data instead of its minimum and maximum?
    箱子宽度 \(\Delta=(a_{\max}-a_{\min})/N\)。FIG 3 里一个记成 4.0 的故障值让 \(x\) 维范围从约 1.55 变成约 4.75，16 个箱子时 RMSE 从 0.023 涨到 0.065。分位数不改变 token 数（选项 A），也不会让误差为零（选项 C），代价是截断最极端的 1%。The bin width is \(\Delta=(a_{\max}-a_{\min})/N\). In FIG 3 one faulty value recorded as 4.0 stretches the \(x\) range from about 1.55 to about 4.75, and with 16 bins the RMSE rises from 0.023 to 0.065. Quantiles change neither the token count (option A) nor make the error zero (option C); their price is clipping the most extreme 1%.
    3. 在 FAST 里把缩放因子 \(\gamma\) 加倍，会怎样？3. What happens in FAST when the scale \(\gamma\) is doubled?
    \(q_k=\operatorname{round}(\gamma c_k)\)：\(\gamma\) 越大，取整越细，误差越小，但满足 \(|\gamma c_k|<\tfrac12\) 而变成 0 的系数越少，整数也越大，BPE 能压缩的就越少。系数个数永远是 \(D\times H\)（选项 C 错）。BPE 无损，但损失发生在取整这一步（选项 D 错）。FIG 4 里 50 Hz 时 \(\gamma\) 从 10 到 30，平滑轨迹的 token 从 13 增到 17，RMSE 从 0.015 降到 0.0055。\(q_k=\operatorname{round}(\gamma c_k)\): a larger \(\gamma\) rounds more finely and errs less, but fewer coefficients satisfy \(|\gamma c_k|<\tfrac12\) and become 0, and the integers grow, leaving BPE less to compress. There are always \(D\times H\) coefficients (option C is wrong). BPE is lossless, but the loss happens at the rounding step (option D is wrong). In FIG 4 at 50 Hz, going from \(\gamma=10\) to 30 takes the smooth trajectory from 13 to 17 tokens and its RMSE from 0.015 to 0.0055.
    4. 一个对动作箱子做分类的策略在左右两种示范上训练得很好。部署时用期望值 \(\sum_kp_k\hat a_k\) 解码，在两种示范各占一半的状态下会发生什么？4. A policy that classifies action bins has been trained well on demonstrations going left and right. At deployment it decodes by the expectation \(\sum_kp_k\hat a_k\). What happens in a state where the two kinds of demonstration are equally common?
    期望最小化平方误差，等于把分布平均成一个点。分布本身是多峰的（网络学对了），是解码把它毁掉了。选项 A 描述的是采样，选项 B 描述的是取最大。FIG 6 里切到“期望”就能看到 \(s=0\) 处的解码动作落进红色带子。The expectation minimizes squared error, collapsing the distribution to a single point. The distribution itself is multimodal (the network learned it correctly); the decoding destroyed it. Option A describes sampling, option B argmax. Switch FIG 6 to "expectation" and the decoded action at \(s=0\) lands in the red band.
    5. 20 Hz 的 2 维动作，用 \(K=256\) 的单级 VQ 码本、片段长度 \(L=10\)。每秒多少个 token、多少比特？5. 2-dimensional actions at 20 Hz, a one-stage VQ codebook with \(K=256\) and snippet length \(L=10\). How many tokens and how many bits per second?
    每 \(L=10\) 步（0.5 秒）一个 token，每秒 \(f/L=2\) 个，每个 \(\log_2256=8\) 比特，共 16 比特。选项 A 是同样频率下每维 256 个箱子的逐步分箱：码率高了 20 倍。码率这么低，片段内的细节只能靠码本里那 256 种形状来近似，所以 FIG 5 里 \(L=10\) 时边界跳变明显变大。One token per \(L=10\) steps (0.5 s), so \(f/L=2\) per second, each of \(\log_2256=8\) bits: 16 bits. Option A is per-step binning with 256 bins per dimension at the same rate, 20 times the bit rate. At such a low rate the detail inside a snippet can only be approximated by the 256 shapes in the codebook, which is why the boundary jumps in FIG 5 grow markedly at \(L=10\).

    10延伸阅读Further reading

    • P. Gage, A New Algorithm for Data Compression, The C Users Journal, 1994 — 字节对编码最早作为数据压缩算法提出byte-pair encoding first proposed as a data-compression algorithm
    • R. Sennrich, B. Haddow & A. Birch, Neural Machine Translation of Rare Words with Subword Units, ACL 2016 — 把 BPE 用于神经网络的子词切分BPE for subword segmentation in neural models
    • A. Dosovitskiy et al., An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, ICLR 2021 — ViT：图像块作为 tokenViT: image patches as tokens
    • N. Ahmed, T. Natarajan & K. R. Rao, Discrete Cosine Transform, IEEE Transactions on Computers, 1974 — DCT 的原始论文the original DCT paper
    • A. van den Oord, O. Vinyals & K. Kavukcuoglu, Neural Discrete Representation Learning, NeurIPS 2017 — VQ-VAE：学出来的离散码本VQ-VAE: learned discrete codebooks
    • M. Janner, Q. Li & S. Levine, Offline Reinforcement Learning as One Big Sequence Modeling Problem, NeurIPS 2021 — Trajectory Transformer：逐维离散化的轨迹序列模型the Trajectory Transformer: sequence modelling of per-dimension discretized trajectories
    • S. Reed et al., A Generalist Agent, Transactions on Machine Learning Research, 2022 — Gato：文字、图像和连续动作共用一套 tokenGato: text, images and continuous actions in one token space
    • A. Brohan et al., RT-1: Robotics Transformer for Real-World Control at Scale, RSS 2023 (arXiv 2022) — 每维 256 个箱子的动作 tokenaction tokens with 256 bins per dimension
    • A. Brohan et al., RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control, CoRL 2023 (arXiv 2307.15818; the proceedings list B. Zitkovich first) — 把动作写成语言模型的 tokenactions written as language-model tokens
    • M. J. Kim et al., OpenVLA: An Open-Source Vision-Language-Action Model, CoRL 2024 — 分位数范围的动作分箱action binning with quantile ranges
    • N. M. Shafiullah, Z. J. Cui, A. Altanzaya & L. Pinto, Behavior Transformers: Cloning k Modes with One Stone, NeurIPS 2022 — k-means 动作类别加连续偏移k-means action classes plus continuous offsets
    • S. Lee, Y. Wang, H. Etukuru, H. J. Kim, N. M. Shafiullah & L. Pinto, Behavior Generation with Latent Actions, ICML 2024 — VQ-BeT：残差 VQ 编码的动作块VQ-BeT: action chunks coded by residual VQ
    • K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn & S. Levine, FAST: Efficient Action Tokenization for Vision-Language-Action Models, RSS 2025 (arXiv 2501.09747) — DCT + BPE 的动作 token 化DCT + BPE action tokenization
    • T. Z. Zhao, V. Kumar, S. Levine & C. Finn, Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware, RSS 2023 — ACT 与动作分块ACT and action chunking