先看左侧的文字规则与右侧的词结构，再比较中间两种词形处理，最后沿底部理解语料库怎样在选定规则下形成词汇表。

✎ 阅读提示根据我的手写笔记整理；箭头表示概念之间的联系，不是必须依次执行的步骤。

Start with text patterns on the left and word structure on the right. Compare the two word-form methods in the middle, then follow the bottom row from a corpus to a vocabulary under chosen rules.

✎ 阅读提示Tidied up from my handwritten page; arrows show links between ideas, not a fixed sequence of steps.

学习语言时，我们会关注词如何构成、句子如何组织，以及说话者到底想表达什么。用这些问题理解 NLP，可以建立一张清晰的概念地图。

层面	关注的问题	例子
形态学 Morphology	词内部如何构成？	walk 加上 -ed 成为 walked
句法 Syntax	词如何组成短语和句子？	The cat sat on the mat. 的主语和谓语
语义 Semantics	词和句子表达什么意义？	谁做了什么，动作涉及什么对象
语用 Pragmatics	在此情境下，说话者想做什么？	“能把盐递给我吗？”通常是在请求
篇章 Discourse	多个句子如何连接？	后一句的“她”指向前文哪一个人

自然语言理解（NLU）与自然语言生成（NLG）分别关注解释语言和产生语言。“从词到意义，再从意图到表达”有助于入门，但不是所有系统都遵循的固定流水线，也不能证明计算机以与人相同的方式学习语言。

✎ 阅读提示读图说明：图中的箭头是概念关系。语用与篇章可以分别讨论；理解与生成也不是严格互逆的计算过程。

When learning a language, we notice how words form, how sentences fit together, and what a speaker intends. These questions also provide a useful map for approaching NLP.

Level	Question	Example
Morphology	How are words built internally?	walk plus -ed becomes walked
Syntax	How do words form phrases and sentences?	The subject and predicate in The cat sat on the mat.
Semantics	What do words and sentences mean?	Who did what, and what was involved?
Pragmatics	What is the speaker trying to do here?	“Can you pass the salt?” is usually a request
Discourse	How do sentences connect?	Which previously mentioned person does “she” refer to?

Natural language understanding (NLU) concerns interpreting language; natural language generation (NLG) concerns producing it. Moving from words toward meaning, or from an intention toward an expression, is a useful introductory picture. It is not a fixed pipeline followed by every system, nor evidence that computers learn language in the same way people do.

✎ 阅读提示Reading note: the arrows show conceptual relationships. Pragmatics and discourse can be discussed separately. Understanding and generation are not strictly inverse computational procedures.

词素（morpheme）是承载意义或语法功能的最小语言单位。词根承载词的核心意义；词缀附着在其他成分上，改变意义或语法功能。例如 cats 包含 cat 和复数词缀 -s。词干（stem）是进一步添加屈折词缀的基础，不一定等于词根。Words and Tokens

屈折变化表达数、时态等语法信息，例如 cat → cats；派生构成新词，例如 happy → unhappy。派生可以改变词性，也可以保持词性。中文学习中，偏旁部首帮助我们认识字形，但它们不等于词素，不能把英文的词根词缀分析直接套到汉字结构上。

✎ 阅读提示读图说明：lies 既可能是名词复数，也可能是动词形式，需结合上下文。图中的“提取词根”和“提高搜索准确性”是简化说法：词干提取不保证得到真正词根，也不保证提高检索效果。图示 unhappy → happy 不能当作 Porter 算法的通用规则。

A morpheme is a minimal unit carrying meaning or grammatical function. A root carries a word's central meaning; an affix attaches to another element and changes meaning or grammatical function. Cats contains cat and the plural affix -s. A stem is the base to which an inflectional affix can be added, and need not be identical to a root. Words and Tokens

Inflection expresses grammatical information such as number or tense: cat → cats. Derivation creates a new word: happy → unhappy. Derivation may change word class, but does not always do so. Chinese radicals help us recognize written character structure; they are not equivalent to morphemes. English root-and-affix analysis cannot simply be transferred to Chinese character components.

✎ 阅读提示Reading note: lies can be a plural noun or a verb form, depending on context. Derivation does not always change word class. Stemming need not recover a true root, is not required by every search system, and does not guarantee better retrieval. The Chinese figure's unhappy → happy illustration is not a general Porter rule.

分词（tokenization）把文本切分成处理单位。单位可以是词、子词或字符；token 不一定等于语言学上的词素。句子切分确定句界，而文本规范化统一选定的形式，例如多余空格或大小写。Words and Tokens

这些操作需要服务于研究问题。“清理得越彻底越好”并不是通用原则。以访谈分析为例，删除否定词可能改变受访者的意思；统一词形也可能丢失研究所需的时态差异。应保留原文，并明确每一步改变了什么。

语料库（corpus）是为分析而收集的文本集合；词汇表（vocabulary）则记录按既定规则得到的不同词项。采用不同的切分和规范化规则，同一份材料可能产生不同的词汇表。

Tokenization divides text into processing units, which may be words, subwords, or characters. Tokens need not correspond to linguistic morphemes. Sentence segmentation identifies sentence boundaries. Normalization standardizes selected forms, such as excess spaces or capitalization. Words and Tokens

These choices should serve the research question. More aggressive cleaning is not automatically better. In interview analysis, deleting a negation can change what a participant said; merging word forms can remove a tense distinction that matters. Keep the original text and make each transformation explicit.

A corpus is a collection of texts assembled for analysis. A vocabulary records the distinct items obtained under chosen processing rules. Different tokenization and normalization choices can produce different vocabularies from the same material.

方法	核心做法	如何理解结果
词干提取 Stemming	按规则删改词的形式	结果可能不是一个词典词
词形还原 Lemmatization	利用词典和形态信息寻找基本词形	结果取决于词性信息和所用工具

Porter 是面向英语的经典词干提取算法，按顺序和条件应用后缀规则。它不是简单地删除所有词缀，也不适用于所有语言。词形还原则可将动词 was 对应到 be；但不同工具在缺少词性信息时可能给出不同结果。Porter 算法；NLTK 词形处理示例

两种方法都可能帮助归并词形，但不能仅凭方法名称判断谁更准确或更快。应检查它们是否把研究中需要区分的表达错误地合并。

Method	Approach	Interpreting the output
Stemming	Apply rules that remove or rewrite parts of word forms	The result may not be a dictionary word
Lemmatization	Use lexical and morphological information to find a base form	Results depend on the tool and available part-of-speech information

Porter is a classic English stemming algorithm that applies ordered suffix rules with conditions. It does not simply remove every affix and is not a solution for every language. Lemmatization can map the verb was to be, although tools may behave differently without part-of-speech information. Porter algorithm; NLTK examples

Both approaches can group related forms. Neither name alone tells us which will be more accurate or faster in a particular workflow. Check whether the method merges expressions that the research needs to distinguish.

正则表达式（regular expression，简称 regex）用模式描述要寻找的字符串，适合搜索、替换和格式检查。它匹配的是文字形式；匹配到一个关键词，不等于理解了整句话。

下面用 Python 的正则语法演示。[wW] 选择单个字符，s? 允许一个可选的 s；\b 表示词边界。Python re 文档

import re

text = "Woodchuck woodchucks woodchuckish"
matches = re.findall(r"\b[wW]oodchucks?\b", text)
print(matches)  # ['Woodchuck', 'woodchucks']

print(re.sub(r" +", " ", "follow   up"))  # follow up

✎ 阅读提示读图说明：[wW] 本身只匹配 w 或 W；应写 [wW]oodchucks?，图中的 woodchunks 拼写有误。[^A-Z] 只匹配一个非 ASCII 大写字母字符。\. 匹配句点，不匹配整个 1.5。^、$ 的行为与多行模式有关；. 默认不匹配换行。英文配图中的 color/colour 示例应写 colou?r；[^aeiou] 也会匹配数字等非元音字符。

例如，在英文访谈中搜索 pain，会同时找到 “I have pain” 和 “I have no pain”。规则能定位待阅读的片段，是否存在某种经历或观点仍需结合上下文判断。

A regular expression, or regex, describes a string pattern for searching, replacement, or format checking. Matching a pattern does not establish what an entire sentence means.

This example uses Python syntax. [wW] selects one character, s? allows an optional s, and \b marks a word boundary. Python re documentation

import re

text = "Woodchuck woodchucks woodchuckish"
matches = re.findall(r"\b[wW]oodchucks?\b", text)
print(matches)  # ['Woodchuck', 'woodchucks']

print(re.sub(r" +", " ", "follow   up"))  # follow up

✎ 阅读提示Reading note: the color/colour pattern should be colou?r, not colo?r. [^aeiou] also matches digits and other non-vowel characters. In the Chinese figure, [wW] matches only w or W, woodchunks should be woodchucks, and [^A-Z] matches a single character outside ASCII uppercase letters. \. matches a period, not all of 1.5. Anchor behavior depends on multiline mode; a dot excludes newlines by default.

Searching an English interview for pain finds both “I have pain” and “I have no pain.” A rule can locate passages for review. Determining whether a passage describes an experience or viewpoint still requires context.

我开始学习这些内容，是想理解 NLP 到底如何工作。过去我觉得它的方法很复杂；现在，词的结构、句子的组织和意义之间的联系，让这个领域更容易进入。把它类比为学习语言，是对我有帮助的起点，但这个类比不足以解释现代模型的全部机制。

对我来说，这一期不必以记住每一种方法为目标。代码助手可以协助实现简单规则，而我更想理解规则为什么这样设计，以及哪些判断仍需要由人来做。

了解文本处理的基础逻辑后，我可以把研究问题拆解成明确的处理步骤，让计算机按照我设定的规则处理健康相关文本或笔记。我需要决定寻找什么、保留什么，以及如何检查结果。

例如：从患者访谈中整理预约等待的经历

假设我想了解患者预约看诊时的等待经历。下面是三个虚构的访谈片段，以及人工阅读时需要保留的区别：

访谈片段	需要保留的信息
“我等了三周才约到医生。”	提到预约等待，时长为三周；是否归为“等待过长”，还需先明确研究中的判断标准。
“这次没有等很久，很快就约上了。”	明确否定等待很久，不能只因为出现“等”就标记为等待困难。
“以前要等很久，现在好多了。”	过去等待较久，目前有所改善，需要保留时间上的区别。

我可以把处理过程设计成几个步骤：

1. 明确目标：寻找预约等待的经历，先定义哪些表达属于研究范围。
2. 统一形式并保留原文：例如把“三周”和“3 周”记录为相同的时长，同时保留原句供回查。
3. 定位候选片段：用关键词或正则表达式寻找预约、等待和时长相关的表达。
4. 检查否定和时间：结合上下文区分“没有等很久”和“等很久”，以及“以前”和“现在”。
5. 整理并核对：输出原句、等待时长、否定表达和时间信息，再与一小批人工阅读结果比较，检查漏选和误选。

这个例子让我更清楚地理解自己在其中的作用：定义处理目标、制定规则，并检查结果。正则表达式能帮助定位文字模式，但“现在好多了”具体指什么，仍可能需要更复杂的方法或人工判断。这是我想尝试的应用思路，还没有经过实际验证。

我仍想弄清：为了更有效地使用 NLP，我需要掌握哪些基础知识？下一步可以继续学习分词和上下文处理，观察简单规则在哪些地方不再够用。

I started studying this topic because I wanted to understand how NLP works. Its methods used to feel complicated. Seeing the relationships between word structure, sentence organization, and meaning makes the field more approachable. Comparing it with learning a language helps me get started, although that analogy does not explain all the mechanisms of modern models.

For this stage, memorizing every method is not my priority. A coding assistant can help implement simple rules. I want to understand why those rules are designed as they are, and which judgments still belong to the person using them.

Understanding the basic logic of text processing helps me break a research question into explicit steps and have a computer process health-related text or notes according to rules I define. I need to decide what to look for, what to preserve, and how to check the results.

For example: organizing accounts of appointment waits in patient interviews

Suppose I want to explore patients' experiences of waiting to get an appointment. These three fictional interview excerpts illustrate distinctions that a human reader would need to preserve:

Interview excerpt	Information to preserve
"I waited three weeks to get an appointment with a doctor."	An appointment wait of three weeks is reported. Classifying it as excessive would require an explicit research criterion.
"I didn't wait long this time. I got an appointment quickly."	A long wait is explicitly negated. The word "wait" alone should not trigger a label of difficulty.
"I used to wait a long time, but things are better now."	A long wait in the past and improvement now are distinct pieces of information.

I could design the process in a few steps:

1. Define the target: specify which accounts of appointment waits fall within the research question.
2. Normalize selected forms and keep the original: record "three weeks" and "3 weeks" as the same duration while retaining the source sentence.
3. Find candidate passages: use keywords or regular expressions to locate expressions about appointments, waiting, and duration.
4. Check negation and time: use context to distinguish "didn't wait long" from "waited a long time," and past experiences from current ones.
5. Organize and review: output the original sentence, waiting duration, negation, and temporal information, then compare a small sample with manual reading to identify missed and incorrectly selected passages.

This example clarifies my role: define the processing goal, design the rules, and check the output. Regex can help locate text patterns, but deciding what "things are better now" refers to may require more complex methods or human judgment. This is an application I would like to explore; I have not yet tested it in practice.

My remaining question is which foundations I need to use NLP effectively. Next, I can explore tokenization and context handling, and examine where simple rules stop being sufficient.

✎ 阅读提示保留最初的学习路径。整理后的版本写明：词汇表由分词和规范化规则共同决定，分句本身不会直接产生词汇表；蓝色的正则表达式箭头连到下方的规范化流程。词根、词干、词素及算法效果的区分，以正文中的解释为准。

✎ 阅读提示Kept to show the original learning path. In the tidied version, tokenization and normalization together determine the vocabulary (sentence splitting alone does not produce it), and the blue regex arrow connects to the normalization workflow below. For the differences between roots, stems and morphemes, and the limits of the algorithms, follow the article.

本篇由 UBC “Natural Language Processing to Improve Patient Care” 微证书课程学习笔记整理而成，母稿记录的课程为 “Overview of NLP and Large Language Models”。技术核对使用正文链接的 Stanford 教材、Python 官方文档、Porter 算法说明和 NLTK 示例。

动手探索

想亲手探索正则表达式，可以使用 regex101 在线练习工具：输入表达式和示例文本，查看匹配结果与逐项解释。尝试本文示例时，先将 Flavor 选择为 Python，以便与代码语法保持一致。

This article grew out of notes for UBC's Natural Language Processing to Improve Patient Care micro-certificate. The source draft names the course Overview of NLP and Large Language Models. Technical checks used the Stanford textbook, official Python documentation, Porter algorithm description, and NLTK examples linked above.

Explore it yourself

To explore regular expressions interactively, try regex101. Enter a pattern and sample text to inspect matches and explanations. For the examples in this note, choose the Python flavor to match the code syntax.