Learning the natural history of human disease with generative transformers

方法图解Visual method notes

研究背景Background

问题Problem

医疗决策需要了解个体在整个疾病谱上的风险及其随时间的演变；人口老龄化正在推高多病共存和疾病负担（例如预计到 2050 年全球癌症诊断增加 77%）。

Healthcare decisions need individual risk across the full spectrum of disease and its evolution over time, and ageing populations are driving up multimorbidity and disease burden (e.g., a projected 77% rise in global cancer diagnoses by 2050).

研究空白Gap

现有算法大多只预测单一疾病，很少覆盖 ICD-10 顶层的 1,000 多种诊断；全面的、生成式的多病共存建模尚未在大规模数据上评估过。

Existing algorithms mostly predict single diseases; few cover the more than 1,000 top-level ICD-10 diagnoses, and comprehensive generative multimorbidity modelling had not been assessed at scale.

研究目标Objective

扩展 GPT 来建模终生健康轨迹（Delphi-2M），在 UK Biobank 上训练，不重新训练直接在丹麦登记数据上验证，并考察其准确性、生成能力、可解释性和偏倚。

Extend GPT to model lifetime health trajectories (Delphi-2M), train it on UK Biobank, validate it on Danish registries without retraining, and examine its accuracy, generative ability, interpretability and biases.

研究方法Methods

研究速览Study at a glance

方法分步讲解Step by step

1. 1

   把病历变成带时间的词元 · Turn records into timed tokens

   输入是每位 UK Biobank 参与者的记录：每种诊断首次出现的时间（来自全科医生、住院、癌症登记、死亡登记或自我报告的 1,256 个 ICD-10 编码），以及性别、BMI、吸烟和饮酒。每一项都被变成一个词元（token，即一张固定清单里的一个符号，清单共 1,270 个），并配上记录时以天计的年龄；此外大约每 5 年插入一个“无事件”（no event）词元，让没有生病的平静时期也出现在记录里。输出是每人一份带时间戳的清单，像一本每行写着“45.7 岁：哮喘”的日记；与句子里的词不同，行与行之间隔着真实的时间，而且这个间隔有意义，所以每个词元都带着年龄。由于参与者必须活着才能入组，作者把吸烟、饮酒和 BMI 词元的记录年龄在入组时点前后随机平移（−20 至 +40 年），以免模型把它们和入组后才开始出现的死亡联系起来。

   The input is each UK Biobank participant's record: the first date of each diagnosis (1,256 ICD-10 codes from GP, hospital, cancer-registry and death records or self-report), plus sex, BMI, smoking and alcohol. Each item becomes a token (one entry from a fixed list of 1,270 symbols) paired with the age in days when it was recorded, and a 'no event' token is slipped in roughly every 5 years so that quiet, healthy stretches also appear in the record. The output is a time-stamped list per person, like a diary whose lines read 'age 45.7: asthma'; unlike words in a sentence, the gaps between lines are real time and carry meaning, which is why the age travels with every token. Because participants had to be alive to join the study, the authors randomly shifted the recorded ages of the smoking, alcohol and BMI tokens (by −20 to +40 years around enrolment) so that the model would not tie them to the deaths that only start to appear after enrolment.

   为什么这样做Why this step

   Transformer 只能读符号序列，所以必须先把一生的诊断整理成这种形式；“无事件”词元则让模型即使在没有任何记录的年份，也能随着年龄增长定期更新风险。

   A transformer can only read sequences of symbols, so a lifetime of diagnoses has to be put in that form first, and the 'no event' tokens give the model regular moments to update risk as a person ages even when nothing is recorded.

   技术细节Technical detail · M4, M6, M7, M11, M12, M13, M17, M18, M19, M20

   每个数据点是一个（词元, 以天计的年龄）对；词表包括 1,257 个 ICD-10 三位码疾病词元（UK Biobank category 1712 的首次发生数据，加上癌症登记字段 40005/40006）、9 个生活方式词元（BMI、吸烟、饮酒各 3 档）、2 个性别词元、1 个“无事件”词元和 1 个不携带信息的填充词元。“无事件”词元按每 5 年 1 个的速率加入：在 (0, 36,525) 天内均匀抽取 20 个时间点，再与每人的数据时间范围取交集；性别词元放在出生时刻，生活方式词元的时间在入组时点前后随机平移 −20 至 +40 年，以打断与永生时间偏倚（immortal time bias）之间的混杂。图 1b 报告 UK Biobank 共 502,485 人、7,426,036 个词元。

   Each datapoint is a (token, age-in-days) pair; the vocabulary holds 1,257 ICD-10 level-3 disease tokens (first occurrences from UK Biobank category 1712 plus cancer-registry fields 40005/40006), 9 lifestyle tokens (three levels each of BMI, smoking and alcohol), 2 sex tokens, 1 'no event' token and 1 non-informative padding token. 'No event' tokens are added at 1 per 5 years by sampling 20 uniform times in (0, 36,525) days, intersected with each person's data range; sex tokens sit at birth, and lifestyle token times are randomized by −20 to +40 years around enrolment to break confounding with immortal time bias. Fig. 1b reports 502,485 UK Biobank individuals with 7,426,036 tokens.

   C1 · 词元与序列（token / sequence） · Tokens and sequences
2. 2

   划分训练、验证与检验数据 · Split data for honest testing

   输入是第 1 步得到的词元序列。UK Biobank 参与者被随机分成两份：80%（402,799 人）用来教模型，20%（100,639 人）留出来检查和调整模型；此外还有两次“面向未来”的检验：把某个截点日期之后的记录全部藏起来，空出 1 年，再让模型预测下一年里的新发诊断——一次针对 2020 年年中仍在世的 471,057 名 UK Biobank 参与者，一次针对 1.93 million（193 万）名 50–80 岁的丹麦居民，其登记编码已转换成相同格式。这就像先用没见过的题考学生，再用下学期的试卷考，最后用另一个国家学校出的试卷考；不同的是，英国的“未来”检验里有些人的早年记录模型学过，只有他们后来的结局是新的。

   The input is the token sequences from step 1. UK Biobank participants are randomly split so that 80% (402,799 people) are used to teach the model and 20% (100,639) are held back to check and tune it; on top of that, two 'future' tests hide everything after a cut-off date, skip one year, and ask the model to predict new diagnoses in the following year — once for the 471,057 UK Biobank participants still alive in mid-2020 and once for 1.93 million Danish residents aged 50–80, whose registry codes were converted to the same format. It is like marking students on unseen questions, then on next term's exam, and finally on an exam set by a school in another country; unlike a clean exam, the UK 'future' test includes people whose earlier years the model studied, and only their later outcomes are new.

   为什么这样做Why this step

   模型在学过的人身上可能表现很好，换一批人就失灵；留出的个体、更晚的时间窗和另一个国家的医疗系统，依次检验难度越来越大的泛化能力。

   A model can look excellent on the people it learned from yet fail elsewhere, so held-out people, a later time window and another country's health system test progressively harder kinds of generalization.

   技术细节Technical detail · M1, M2, M5, M8, M9, M10, M15, M16, M21, M38, M39, M40, M41, M74

   UK Biobank 个体按 80/20 随机划分（训练 402,799 人、验证 100,639 人；数据自出生至 2020 年 6 月 30 日；验证集同时用于超参数优化）。内部纵向检验：截点时仍在世的全部 471,057 人，预测 1 年时间框内的发生率，间隔 1 年，评估 2021 年 7 月 1 日至 2022 年 7 月 1 日 50–80 岁人群的发病；外部纵向检验：2016 年 1 月 1 日时 50–80 岁的丹麦居民（1.93 million；女性 51%），输入截至该日，评估窗口为 2017 年 1 月 1 日至 2018 年 1 月 1 日，直接沿用英国训练得到的权重、不作任何修改。丹麦数据按 UK Biobank 格式统一（只保留首次发生、ICD-8 映射到 ICD-10、取三位码、删除 UK Biobank 中没有的编码、无生活方式词元）。

   Random 80/20 split of UK Biobank individuals (402,799 training, 100,639 validation; data from birth to 30 June 2020; the validation set also served for hyperparameter optimization). Internal longitudinal test: all 471,057 individuals alive at the cut-off, rates predicted over a 1-year frame, a 1-year gap, and incidence evaluated from 1 July 2021 to 1 July 2022 for ages 50–80; external longitudinal test: Danish residents aged 50–80 on 1 January 2016 (1.93 million; 51% female), inputs up to that date and evaluation from 1 January 2017 to 1 January 2018, using the UK-trained weights unchanged. Danish data were harmonized to UK Biobank (first occurrences only, ICD-8 mapped to ICD-10, level-3 codes, codes absent from UK Biobank dropped, lifestyle tokens absent).

   C1 · 词元与序列（token / sequence） · Tokens and sequencesC2 · 训练集、验证集、检验集与外部验证 · Training, validation and test sets; external validation
3. 3

   把 GPT-2 改造成病史模型 · Adapt GPT-2 to health timelines

   起点是 GPT-2——OpenAI 在 2019 年公开的文本生成模型，也是 ChatGPT 背后那类模型的前身：它把每个词元变成一串数字（嵌入，embedding），再用注意力机制（attention）衡量前面哪些词元对预测下一个词元最重要。作者改了三处：告诉模型“这是第几个词”的位置信号换成了这个人的真实年龄；预测某个条目时，禁止模型参考与它同一时刻记录的其他条目；再加一个输出，估计距下一个事件还有多久。改造后的 Delphi 能在病史的任何时间点，给出 1,256 种疾病及死亡各自的每日发生率，由此同时推出“下一件是什么”和“多快发生”。这就像把手机输入法的联想功能升级，不仅猜你下一个词，还猜你什么时候会打出来；不同的是，病史条目之间隔着的是真实的时间。

   The starting point is GPT-2, a text generator released by OpenAI in 2019 and a forerunner of the models behind ChatGPT: it turns each token into a list of numbers (an embedding) and uses attention to weigh which earlier tokens matter for predicting the next one. The authors change three things — the signal that tells the model a word's position is replaced by the person's actual age, the model is barred from looking at other entries recorded at the same moment as the one being predicted, and a second output estimates how long until the next event. The result, Delphi, gives for any point in a history a rate per day for each of 1,256 diseases plus death, from which both 'what comes next' and 'how soon' follow. It is like upgrading phone autocomplete so that it suggests not only the next word but also when you will type it; unlike text, the space between entries is real clock time.

   为什么这样做Why this step

   标准 GPT-2 只知道词的先后顺序，不知道两词之间隔了多久，所以它本身既无法回答疾病大约何时发生，也处理不好同一天记录的多个诊断。

   Standard GPT-2 knows only the order of words, not the time between them, so on its own it could neither say when a disease is likely to occur nor handle several diagnoses recorded on the same day.

   技术细节Technical detail · M24, M26, M27, M28, M30, M45

   结构沿用 nanoGPT 实现的 GPT-2（词元嵌入与输出层共享权重（weight tying）；N 个前置层归一化的模块，每个含多头因果自注意力和前馈层），图 1c 中标红的改动有三处：用正弦/余弦函数（最低频率 1/365）经可训练线性层组合而成的年龄编码取代位置编码；因果连续时间掩码，额外屏蔽与被预测词元时间戳相同的词元；以及输出距事件时间的 Exp 头。logits 即对数速率，λi = exp(logit_i)：下一个词元为 j 的概率是 λj/Σiλi（即 softmax），等待时间服从速率为 Σiλi 的指数分布，一年内发生的概率为 P = 1 − exp(−λ × 365.25)。

   The architecture follows nanoGPT's GPT-2 (token embedding weight-tied to the output layer; N× pre-LayerNorm blocks of multi-head causal self-attention and feed-forward layers), with three changes marked red in Fig. 1c: an age encoding of sine and cosine functions (lowest frequency 1/365) mixed by a trainable linear layer in place of positional encoding, a causal continuous-time mask that also hides tokens sharing the predicted token's timestamp, and an Exp head for time to event. Logits are log-rates, λi = exp(logit_i): the next token is j with probability λj/Σiλi (the softmax), the waiting time is exponential with rate Σiλi, and P(event within a year) = 1 − exp(−λ × 365.25).

   C1 · 词元与序列（token / sequence） · Tokens and sequencesC4 · 下一词元预测与交叉熵（cross-entropy） · Next-token prediction and cross-entropyC5 · Transformer 与注意力机制（attention） · Transformer and attentionC6 · 嵌入（embedding）与 UMAP 降维 · Embeddings and UMAPC9 · 位置编码与年龄编码 · Positional encoding vs age encodingC10 · 竞争风险与指数等待时间 · Competing risks and exponential waiting times
4. 4

   调模型大小并检验改动 · Tune size, test design choices

   作者用训练集和验证集训练了 486 个大小不同的模型版本——区别在于每个词元用多长的一串数字表示、叠了多少层注意力、每层有几个注意力“头”（head，即几条并行的回看通道）——并在留出的验证集人群上逐一打分。结果显示，对 UK Biobank 这样规模的数据，约 2 million（200 万）个可调数字（参数，parameter）最合适，因此选定了一个有 2.2 million（220 万）参数的版本，即 Delphi-2M；作者还把新加的部件逐个去掉，并改变表示平静期的“无事件”词元出现的频率，以检验每处改动是否真的有用。这就像为一门课挑笔记本：太小记不下，太大就开始抄无关的噪声；不同的是，这里的“合适大小”取决于数据有多少。

   Using the training and validation sets, the authors trained 486 versions of the model that differ in size — how long a list of numbers each token gets, how many attention layers are stacked, and how many attention 'heads' (parallel ways of looking back) each layer has — and scored each on the held-out validation people. The scores showed that, for data of UK Biobank's size, about 2 million adjustable numbers (parameters) is the sweet spot, so they kept a 2.2-million-parameter version, Delphi-2M; they also removed the new parts one at a time, and varied how often the quiet-period 'no event' tokens appear, to check that each change helps. It is like choosing a notebook for a course: too small and the notes do not fit, too large and you start copying noise instead of the lesson — except that here the right size depends on how much data there is.

   为什么这样做Why this step

   模型相对数据太大，就会记住个人而不是学到规律；逐个去掉新部件，则能看出提升究竟来自这些改动，还是只来自模型规模。

   A model too large for its data memorizes individuals instead of learning patterns, and removing each new part in turn shows whether the modifications, rather than sheer size, account for the gains.

   技术细节Technical detail · M22, M25, M34, M35, M36

   超参数筛查覆盖上下文长度（48、56、64）、嵌入维度（60–156）、层数（6–16）和注意力头数（6–16），共 486 个模型，以验证集交叉熵比较；在训练数据比例 0.05–1.0 下的规模定律（scaling law）分析提示约 2 million 参数最优；Delphi-2M 的嵌入维度为 120、12 层、12 个头（2.2 million 参数；扩展数据图 2 中上下文长度为 48，正文未定义该设置）。消融实验以各年龄上相对于年龄–性别基线的交叉熵，比较 Delphi-2M、NanoGPT-2M、加填充的 NanoGPT-2M 以及去掉指数损失的 Delphi-2M；填充实验则以每 1–50 年 1 个或不加“无事件”词元训练模型（每种设置 3 个随机种子）。

   The hyperparameter screen covered context size (48, 56, 64), embedding dimension (60–156), layers (6–16) and heads (6–16) in 486 models compared by validation cross-entropy, and a scaling-law analysis over training fractions 0.05–1.0 indicated about 2 million parameters as optimal; Delphi-2M has embedding dimension 120, 12 layers and 12 heads (2.2 million parameters; context size 48 in Extended Data Fig. 2, a setting the text does not define). The ablation compared Delphi-2M with NanoGPT-2M, NanoGPT-2M with padding and Delphi-2M without the exponential loss by age-wise cross-entropy relative to an age–sex baseline, and the padding experiment trained models with 'no event' rates of 1 per 1–50 years or none (3 seeds each).

   C2 · 训练集、验证集、检验集与外部验证 · Training, validation and test sets; external validationC4 · 下一词元预测与交叉熵（cross-entropy） · Next-token prediction and cross-entropyC5 · Transformer 与注意力机制（attention） · Transformer and attention
5. 5

   训练：猜下一事件及时间 · Train: guess next event, timing

   把 402,799 份训练病史输入模型；在每份病史的每个时间点，模型都要说出下一件事是什么、还要多久发生，每轮之后把内部数字微调一点，让真实答案的可能性变大——共 200,000 轮，每轮 128 份病史。性别、BMI、吸烟和饮酒只作为背景被读入，模型从不预测它们。这就像实习医生逐行读几千份病历，每读一行就用卡片盖住下一行，先猜再看；不同的是，模型还得猜出下一行上写的日期。

   The 402,799 training histories are fed to the model; at every point in each history it must say what comes next and how long until it happens, and after each round its internal numbers are nudged slightly so that the true answer becomes more likely — 200,000 rounds of 128 histories each. Sex, BMI, smoking and alcohol are only read as context and are never themselves predicted. It is like a trainee reading thousands of charts line by line, covering the next line with a card and guessing it before looking; unlike the trainee, the model must also guess the date written on that next line.

   为什么这样做Why this step

   同时预测下一事件“是什么”和“何时发生”，迫使模型输出可以当作发病率来读的绝对速率，而不只是哪种病更可能发生的排序。

   Having to predict both the identity and the timing of the next event forces the model to output absolute rates that can be read as incidences, not just a ranking of which disease is more likely.

   技术细节Technical detail · M28, M31, M32, M33

   总损失 = lossj + lossT，其中 lossj = −log P(j) 是下一个词元的交叉熵，lossT = −log p(T*) = −(logsumexp(logits) − sum(exp(logits)) × T*) 是等待时间的指数分布对数似然；性别、生活方式和不携带信息的填充词元的 logits 设为 −Inf、不进入损失，因此两项损失只在疾病词元和“无事件”词元上计算。优化使用 Adam（标准参数），共 200,000 次迭代，批量 128，预热 1,000 次迭代后学习率按余弦从 6 × 10−4 衰减到 6 × 10−5，采用 32 位精度；同一时刻发生的多个事件都从最后一个非同时发生的词元出发预测。

   Total loss = lossj + lossT, where lossj = −log P(j) is the cross-entropy of the next token and lossT = −log p(T*) = −(logsumexp(logits) − sum(exp(logits)) × T*) is the exponential log-likelihood of the waiting time; sex, lifestyle and non-informative padding tokens have logits set to −Inf and are excluded, so both terms are evaluated only on disease and 'no event' tokens. Optimization used Adam (standard parameters) for 200,000 iterations, batch size 128, 1,000 warmup iterations and cosine decay of the learning rate from 6 × 10−4 to 6 × 10−5, in 32-bit precision; co-occurring events are each predicted from the last non-co-occurring token.

   C4 · 下一词元预测与交叉熵（cross-entropy） · Next-token prediction and cross-entropyC10 · 竞争风险与指数等待时间 · Competing risks and exponential waiting times
6. 6

   对照基准评估预测 · Score predictions against benchmarks

   训练好的模型给出的发生率，要在留出的英国人群、英国“未来”检验和丹麦检验中接受两方面检查：后来得了某病的人，是否比没得的人被给了更高的发生率（AUC，在各性别的 5 年年龄段内分别计算，免得只靠年龄就能区分）；预测的发生率是否与观察到的发生率一致（校准，calibration）。同样的检查也用于简单的年龄–性别发病率表、现成的临床工具（心血管病的 QRisk3、SCORE2、PREVENT、Framingham，痴呆的 UKBDRS，死亡的 Charlson 与 Elixhauser），以及机器学习模型（从数据中学规律的计算机模型），其中包括用自然语言提问、让它给出风险估计的类 ChatGPT 模型 LLama3.1。这就像评价天气预报员：既看下雨的日子是否事先被给了更高的降雨概率，也看报“30% 降雨概率”的日子是否真有大约 30% 下了雨。

   The trained model's predicted rates are checked on the held-out UK people, the UK 'future' test and the Danish test in two ways: whether people who went on to get a disease had been given higher rates than people who did not (AUC, computed inside 5-year age bands for each sex so that age alone cannot do the work), and whether predicted rates match observed rates (calibration). The same checks are run on simple age-and-sex incidence tables, on established tools — QRisk3, SCORE2, PREVENT and Framingham for cardiovascular disease, UKBDRS for dementia, Charlson and Elixhauser for death — and on machine-learning models (computer models that learn patterns from data), including a ChatGPT-like model, LLama3.1, asked in plain words for a risk estimate. It is like judging a weather forecaster both on whether rainy days had received higher rain chances and on whether '30% chance' days really saw rain about 30% of the time.

   为什么这样做Why this step

   单个总体 AUC 可能掩盖年龄的作用，排序好也不代表绝对风险对，所以需要区分度、校准和与基准的正面比较三方面一起判断这些发生率能否实际使用。

   A single overall AUC can hide the effect of age, and a good ranking can still give wrong absolute risks, so discrimination, calibration and head-to-head baselines are all needed to judge whether the rates are usable in practice.

   技术细节Technical detail · M23, M42, M46, M47, M49, M50, M51, M52, M53, M54, M55, M60, M61, M62, M63, M64, M65, M66, M67, M68, M69, M70, M71, M72, M73

   非纵向 AUC：按发病年龄把病例分入 50–80 岁的 5 年年龄段并加入年龄匹配的对照，用预测发生率计算 AUC，对病例多于 2 例的年龄段取平均，男女分开（部分分析只用结局发生前至少 N 个月记录的词元），DeLong 方差合并为 N(μ, σ²)；校准在验证集上用对数线性风险分箱，在纵向检验中用预测发生率十分位组，发病率交叉熵则比较模型与观测的年发病率。纵向 ROC-AUC 与平均精确率（scikit-learn）纳入至少 25 例的疾病；Nelson–Aalen 年龄–性别基线英国部分来自训练数据、丹麦部分来自 2010–2016 年全体丹麦人口，对照模型的协变量缺失用 MICE 插补（5 份，按 Rubin 规则合并），正面比较的终点为 CVD、痴呆和死亡。其他比较包括 MILTON 已发表的 AUC（410 种疾病）、HbA1c 等生物标志物、总体健康自评，以及在多基因风险评分或生物标志物之外加入 Delphi logits 的线性模型；另用 4 种不同划分重新训练以检验可重复性。

   Non-longitudinal AUC: cases stratified into 5-year age brackets (50–80 years) with age-matched controls, AUC from predicted rates averaged over brackets with more than 2 cases, by sex (some analyses used only tokens recorded N or more months before the outcome), with DeLong variances combined as N(μ, σ²);calibration used log-linear risk bins (validation) or deciles of predicted rate (longitudinal), and incidence cross-entropy compared modelled with observed annual incidence. Longitudinal ROC-AUC and average precision (scikit-learn) covered diseases with at least 25 cases; the Nelson–Aalen age–sex baseline came from the training data (UK) or the 2010–2016 Danish population, comparators' missing covariates were imputed by MICE (5 copies, Rubin's rules), and the head-to-head end points were CVD, dementia and death. Further comparisons used MILTON's published AUCs (410 diseases), biomarkers such as HbA1c, self-rated health and linear models adding Delphi logits to polygenic scores or biomarkers, and retraining on 4 different splits checked reproducibility.

   C2 · 训练集、验证集、检验集与外部验证 · Training, validation and test sets; external validationC3 · AUC 与校准（calibration） · AUC and calibration
7. 7

   生成未来与合成病史 · Simulate future and synthetic histories

   给定某人到某个年龄为止的病史（或只给出生时的性别），模型按预测的发生率随机抽出下一个事件及其时间，把它接到病史后面，再重复这一过程，就像手机输入法的联想功能一个词接一个词地写出一整段话。作者对 63,622 名验证集参与者从 60 岁开始这样模拟（每人一条未来轨迹，到 80 岁截止），再与他们的真实经历比较；又从出生开始生成 400,000 份完全虚构的人生病史（合成数据），用来训练一个从未见过真实病人的新模型，然后在真实的验证数据上检验。就像飞行模拟器，每次模拟只是一条可能的路径，而不是确定的预言，所以要把模拟出的发病率与观察到的发病率比较，并把每个人模拟出的诊断与其后来真实得到的诊断对照。

   Given a history up to some age, or just a person's sex at birth, the model draws the next event and its timing at random in line with its predicted rates, adds it to the history and repeats, the way phone autocomplete can write a whole paragraph one word at a time. This was done for 63,622 validation participants from age 60 (one simulated future each, stopped at age 80) and compared with what actually happened to them, and 400,000 entirely artificial life histories (synthetic data) were generated from birth to train a fresh model that never saw a real patient, which was then tested on the real validation data. Like a flight simulator, each run is one plausible path rather than a certain forecast, so the simulations are checked by comparing simulated with observed disease rates and each person's simulated diagnoses with the ones that person actually received.

   为什么这样做Why this step

   能把病史向前推演的模型，可以估计人群未来的疾病负担，还能产生不照搬任何真实个人记录的训练数据；把模拟结果与观察结果对照，则能检验它是否学到了疾病如何逐步展开，而不只是下一步。

   A model that can roll histories forward can estimate future disease burden for groups and produce training data that do not copy any real person's record, and comparing simulated with observed patterns tests whether it has captured how diseases unfold rather than only the next step.

   技术细节Technical detail · M29, M37, M43, M44, M55

   抽样时，下一个疾病词元从疾病 logits 的 softmax 分布中抽取（舍弃性别和生活方式的 logits），时间取各速率为 λi 的指数分布样本中的最小值，反复迭代得到轨迹，80 岁后截断，并用同样的发病率比较方法评估。60 岁实验以验证集个体 60 岁前的全部数据为条件（据图 3a 图注 n = 63,622，均在 60 岁前后都有记录；第 4 页正文写作 63,662）；合成数据模型用 400,000 条从出生开始、性别随机分配的抽样轨迹训练，超参数不变，并在真实验证数据上评估。

   Sampling draws the next disease token from the softmax over disease logits (sex and lifestyle logits discarded) and the time as the minimum of samples from exponentials with rates λi, iterated to build trajectories that are truncated after age 80 and evaluated with the same incidence comparisons. The age-60 experiment conditioned on all validation data before age 60 (n = 63,622 with records before and after 60 per the Fig. 3a caption; the p. 4 text says 63,662), and the synthetic-data model was trained on 400,000 trajectories sampled from birth with random sex, using unchanged hyperparameters, and evaluated on the observed validation data.

   C2 · 训练集、验证集、检验集与外部验证 · Training, validation and test sets; external validationC4 · 下一词元预测与交叉熵（cross-entropy） · Next-token prediction and cross-entropyC8 · 合成数据（synthetic data） · Synthetic dataC10 · 竞争风险与指数等待时间 · Competing risks and exponential waiting times
8. 8

   解释预测并排查偏倚 · Explain predictions and probe biases

   作者用 100,639 份验证集病史，一次藏起一个或几个既往条目，测量每个条目能让预测发生率变化多少（SHAP）；又把模型为每个诊断学到的那串数字画成二维地图上的点，模型看来相似的疾病会挨在一起（UMAP）；这些效应再与经典的生存分析相互印证，即比较有无某个诊断的人随着时间推移发生事件的快慢。作者还用统计检验比较不同族裔、不同贫困程度人群中的准确度是否不同，并考察记录来源（全科医生、医院或自我报告）是否左右了预测。SHAP 回答的是“如果这个诊断从未被记录，预测会怎样变化”——它描述的是模型的推理，可能反映数据的收集方式，并不能证明一种病导致另一种病。

   Using the 100,639 validation histories, the authors hide past entries one or several at a time and measure how much each one moves the predicted rate (SHAP), and they draw each diagnosis's learned list of numbers as a point on a 2-D map where diseases the model treats alike sit close together (UMAP); these effects are cross-checked against classical survival analyses, which compare how quickly people with and without a given diagnosis go on to have an event over time. They also use a statistical test to see whether accuracy differs across ethnic groups and levels of deprivation, and look at whether the source of a record (GP, hospital or self-report) shapes the predictions. SHAP answers 'how would this prediction change if this diagnosis had never been recorded?' — it describes the model's reasoning, which can mirror how the data were collected, and is not proof that one disease causes another.

   为什么这样做Why this step

   风险模型在用于决策之前，需要弄清它学到的关联是否合理、准确度在不同人群间是否有差异，以及它是否吸收了记录方式带来的假象。

   Before a risk model informs decisions, one needs to know whether its learned associations are plausible, whether its accuracy differs between population groups, and whether it has absorbed artefacts of record-keeping.

   技术细节Technical detail · M14, M56, M57, M58, M59, M75, M76, M77, M78

   SHAP 使用 PartitionExplainer，被屏蔽的词元替换为“无事件”（性别词元换成相反性别），应用于全部 100,639 条验证集轨迹，被解释的始终是最后一个词元；平均 SHAP 值与惩罚性时间依赖 Cox 模型的系数，以及粗略的 Nelson–Aalen 风险比（每个病例配 5 名年龄–性别匹配者，高斯核平滑）相比较；UMAP（余弦度量）作用于 1,270 × 120 的词元嵌入矩阵。亚组异质性检验基于 DeLong 法得到的亚组 AUC、逆方差加权的标准化残差，以及自由度为 (n − 1) 的 χ² 统计量，阈值经 Bonferroni 校正（剔除病例少于 6 例的年龄–性别组）；族裔和多重剥夺指数只用于分层；数据来源偏倚分析只在图注中有描述。

   SHAP used the PartitionExplainer with masked tokens replaced by 'no event' (sex tokens swapped to the opposite sex) on all 100,639 validation trajectories, always explaining the last token; averaged SHAP values were compared with penalized time-dependent Cox coefficients and with crude Nelson–Aalen hazard ratios (5 age–sex-matched controls per case, Gaussian-kernel smoothing), and UMAP (cosine metric) was applied to the 1,270 × 120 token-embedding matrix. Subgroup heterogeneity used DeLong subgroup AUCs, inverse-variance weighted standardized residuals and a χ² statistic with (n − 1) degrees of freedom at Bonferroni-corrected thresholds (age–sex brackets with fewer than 6 cases filtered), with ethnicity and the index of multiple deprivation used only for stratification; the data-source bias analyses are described only in figure captions.

   C2 · 训练集、验证集、检验集与外部验证 · Training, validation and test sets; external validationC3 · AUC 与校准（calibration） · AUC and calibrationC6 · 嵌入（embedding）与 UMAP 降维 · Embeddings and UMAPC7 · SHAP（Shapley 加性解释） · SHAP (Shapley additive explanations)

读懂方法需要的前置知识Concepts you need first

研究结果Results

• 在 1,000 多种诊断上，Delphi-2M 判断谁会先发生某种疾病的能力，在内部验证中年龄-性别分层平均 AUC 约为 0.76（死亡为 0.97）；97% 的诊断 AUC 高于 0.5。 p. 2, 4

  Across more than 1,000 diagnoses, Delphi-2M ranked who would develop a disease next with an average age–sex-stratified AUC of about 0.76 in internal validation (death: 0.97); 97% of diagnoses scored above 0.5. p. 2, 4
• 预测距离越远，准确度逐渐下降：从下一事件的 0.76 降到 10 年后约 0.70。在死亡、痴呆和心血管疾病上与现有评分接近（例如心血管疾病 0.70 对 QRISK3 的 0.71），但在糖尿病上不如 HbA1c（0.75 对 0.84）。 p. 3–4, 18

  Accuracy fell gradually with forecast distance, from 0.76 for the next event to about 0.70 ten years ahead. For death, dementia and cardiovascular disease it performed close to established scores (e.g., CVD 0.70 vs QRISK3 0.71), but worse than HbA1c for diabetes (0.75 vs 0.84). p. 3–4, 18
• 不做任何调整直接用于丹麦登记数据中的 193 万人，平均 AUC 为 0.67（UK Biobank 时间外推测试为 0.69），两地逐病 AUC 的相关系数 r = 0.76（95% CI 0.72–0.80）。 p. 7

  Applied unchanged to 1.93 million people in Danish registries, the average AUC was 0.67 (vs 0.69 in UK Biobank temporal testing), and per-disease AUCs correlated at r = 0.76 (95% CI 0.72–0.80). p. 7
• 从 60 岁开始模拟的轨迹再现了 70–75 岁时的人群发病率；第一年约 17% 的疾病标记被正确预测，20 年后降到 14% 以下（仅用年龄和性别为 12–13%）；只用合成数据训练的模型 AUC 达到 0.74。 p. 5

  Trajectories simulated from age 60 reproduced population disease rates at ages 70–75 and correctly predicted about 17% of disease tokens in the first year, falling below 14% after 20 years (age and sex alone: 12–13%); a model trained only on synthetic data reached an AUC of 0.74. p. 5
• 可解释性分析显示，疾病按器官系统聚集，影响有持久和短暂之分（癌症多年持续抬高死亡率，败血症的影响约 5 年内消退）；同时也发现模型学到了数据收集方式的规律：在有其他住院记录的人身上，仅见于住院记录的疾病预测发生率约高 10 倍。 p. 6–8

  Explainability analyses showed diseases clustering by organ system and effects that are either lasting (cancers raise mortality for years) or short-lived (septicaemia's effect fades within about 5 years), but also that the model learned data-collection patterns: hospital-only diseases were predicted about ten times higher when other hospital records existed. p. 6–8
• 作者认为，该模型是一个有前景、可跨体系迁移的多病预测与卫生规划工具，应作为现有临床工具的补充而非替代。 p. 8–9

  The authors conclude that the model is a promising, transferable tool for multi-disease prediction and health planning that should add to, not replace, existing clinical tools. p. 8–9

关键数字Key numbers

效应量与置信区间Estimates with confidence intervals

研究局限Limitations

研究设计Study design · 2

• 健康志愿者及入组选择偏倚限制了人群代表性。

  Healthy-volunteer and recruitment selection biases limit representativeness.
  p. 7–8

• 入组前缺少死亡记录，可能扭曲死亡风险估计。

  Missing pre-recruitment deaths can distort mortality estimates.
  p. 7, 10–11

数据Data · 2

• 80 岁以上的随访不足，难以支持高龄人群应用。

  Limited follow-up above age 80 restricts use in older populations.
  p. 7

• 诊断数据来源与覆盖差异可能形成预测伪影。

  Differences in diagnostic sources and coverage can create prediction artefacts.
  p. 7–8

外推性Generalizability · 1

• 不同祖源与社会经济群体间的预测表现可能不一致。

  Performance may vary across ancestry and socioeconomic groups.
  p. 7–8

解读Interpretation · 2

• 个体未来轨迹具有较大的偶然性，不能当作确定预后。

  Individual future trajectories remain uncertain and should not be treated as definite prognoses.
  p. 8

• 疾病之间的预测关联不等于因果关系。

  Predictive associations between diseases do not establish causation.
  p. 8

其他Other · 1

• 临床部署仍需监管框架及进一步验证。

  Clinical deployment still requires regulatory oversight and further validation.
  p. 9

我的思考My thoughts

整体评价Overall assessment

我认为还可能存在的不足Further limitations I see

• 内部评估与模型开发并非完全独立，表现可能偏乐观。

  Internal evaluation is not fully independent of model development, so performance may be optimistic.
  p. 2, 10, 16
• 只保留首次诊断和粗略的生活方式记录，难以反映复发与病程变化。

  First diagnoses and coarse lifestyle records miss recurrence and changes over time.
  p. 10
• 跨医疗体系的校准仍不充分；AUC 尚可，也可能高估或低估绝对风险。

  Calibration across healthcare systems remains insufficiently established; acceptable AUC can coexist with over- or underestimated absolute risk.
  p. 7, 17

优先考虑的改进Most feasible improvement

• 优先用一份独立的本地数据检查少数目标结局的校准：比较预测与实际发生率，报告校准截距和斜率；若偏差明显，先尝试简单再校准，再决定是否需要重训。

  Start with a calibration check for a few target outcomes in independent local data: compare predicted and observed rates and report calibration intercepts and slopes. If there is systematic bias, try simple recalibration before considering model retraining.

应用Applications

1. 临床文本与语音：把非结构化记录整理成事件时间线。

   Clinical text and speech: turn unstructured records into event timelines.
   先在小样本中人工核对事件、诊断编码和日期抽取，再考虑接入序列模型。

   First manually check event, code, and date extraction in a small sample before using the timelines in a sequence model.
2. 设备与可穿戴数据：研究带时间的低氧或报警事件序列。

   Device and wearable data: study timed sequences of hypoxia or alarm events.
   先做好信号质量控制，再提取事件；是否能预测后续事件，需要单独验证。

   Quality-check the signals before extracting events, then separately test whether the sequences help predict later events.
3. 卫生服务规划：探索人群层面的疾病负担和服务需求。

   Health-service planning: explore population-level disease burden and service demand.
   用本地发生率校准，并与历史趋势比较；不将人群模拟解释为个人命运。

   Calibrate to local rates and compare with historical trends; population simulations are not individual forecasts.