Large Language Models: Tokens, Attention, Prompting and Reasoning

Start at the top left. The first block lists the three ways an LLM improves on a simple neural language model: tokens instead of words (the violet lines define tokenization, tokens and embeddings), a much longer preceding context, and a dynamic context, where attention works out how much each token should influence the others. Below it are prompting and in-context learning, the everyday way of using an LLM. On the right, the Transformer and the two model families from the course (GPT and Llama), then chain-of-thought. Along the bottom, models trained to reason (DeepSeek-R1 as the example), the two challenges named in the lecture, and my takeaway at the bottom right.

A large language model is a neural language model trained on a very large amount of text to predict the next token. Built on the Transformer architecture, it has billions of weights, sometimes more than a trillion. The basic job has not changed since the last note: take the tokens so far, compute a probability distribution for the next one, pick one, and repeat. What changed is scale and design. More weights and far more training data let it learn much richer patterns of language, so one model can answer questions, summarize, translate and classify when asked in plain language, without a separate model for each task.

"Large" refers to two things in the course: the number of weights (parameters), and the length of context the model can handle.

A simple neural language model takes whole words as input. An LLM takes tokens. A token is a chunk of characters that often occurs together: a whole word, part of a word, or punctuation. Tokenization splits the text into tokens, and each token is then turned into a vector, its embedding.

GPT-3's vocabulary has 50,257 tokens: frequent words such as love and are, affixes such as dis, and punctuation. Sub-word tokens have two advantages. An unknown word can still be built from known pieces, and pieces such as un carry meaning of their own. This is the tokenization from the text-processing notes, now as the actual input of the model.

The simple neural language model in the last note looked at only the previous three words. Asked to continue "The promise of large language models is that they…", it would see only "is that they", which says almost nothing about the topic. An LLM can take thousands of preceding tokens into account.

The more context the model can see, the more consistent and coherent its next word can be. Context length has been one of the main drivers of improvement.

In a simple neural language model, every word has a single static embedding. The mouse in "I'm using both the keyboard and the mouse" and the mouse in "the cat chased away the mouse" get the same vector, which mixes features of a computer device with features of an animal.

An LLM computes a contextual embedding for each occurrence instead: the vector for a token depends on the tokens around it. Next to keyboard, mouse leans toward the device; next to chased and cat, it leans toward the animal. This matters even more for words with little meaning on their own, such as pronouns. In "The promise of large language models is that they…", they means nothing until we know it refers to models.

The part of the network that does this is the attention mechanism. It learns how strongly each token in the context should influence the representation of another token, and combines them by those weights. In the course example, about 75% of the meaning of they comes from models. Attention is the key idea of the Transformer (Vaswani et al., 2017).

✎ reading noteReading note: attention weights are learned scores, not an exact measure of how much one word's meaning influences another. GPT-4 was released in an 8,192-token version and a 32,768-token version; the 30,000+ figure refers to the larger one.

Architecturally, an LLM is a huge neural network called the Transformer, built as a stack of blocks. Each block contains an attention layer, which builds the contextual embeddings, and feed-forward layers that work much like the simple network in the last note. All blocks have the same structure but their own weights. The input text, as embeddings, flows up through the blocks, and the top produces the next token.

Training works the same way as before: predict the next (or a missing) word in a huge amount of text, often web-scale, measure the error, and backpropagate to revise the weights.

The course shows two popular families:

"Open" here means open weights: you can download the trained model and run or fine-tune it yourself. Llama is released under Meta's own license, which, for example, requires companies with more than 700 million monthly users to ask Meta for permission. OpenAI also released two open-weight models, gpt-oss, in 2025, though its flagship GPT models remain closed. Other well-known families include Claude, Gemini, Qwen and DeepSeek. Most share the same Transformer basis and differ in size, training data, training method and openness.

✎ reading noteReading note: the scale row is rough. Llama models range from about 1 billion to 405 billion weights, and OpenAI has not published the size of its recent GPT models. The current spelling is Llama.

The basic way to use an LLM follows from what any language model computes. You give it a sequence of tokens, the prompt. It predicts the next token, appends it, predicts the next one, and continues until it produces an end-of-sequence token. Typing "I have been having a fever for four days, what should I do?" into ChatGPT does exactly this: the answer is generated one token at a time.

The prompt also sets the scope. A model trained on a huge range of text can go in many directions; telling it what we want, and in what form, narrows the space and usually improves the answer.

In-context learning (ICL) goes one step further: put a few solved examples in the prompt, then the new case. The model picks up the task from the examples.

The key point: no retraining, and no weights change. The model adapts while answering, from the context alone; once the prompt is gone, so is the "learning". This is why LLMs are called fast learners.

Reasoning means analysing evidence, building an argument and applying logic or probability to reach a conclusion. Early LLMs were weak at it, and simply making pre-training bigger was not the best way to improve it. Chain-of-thought (CoT) prompting showed that changing the prompt alone can draw out step-by-step reasoning, with no extra training.

There are two common forms:

The course's word problem shows the difference. With a plain in-context example (Roger has 5 tennis balls and buys 2 cans of 3; answer: 11), the model answers a new cafeteria question with 27, which is wrong. When the example also shows the steps (2 cans × 3 = 6, 5 + 6 = 11), the model writes out its own steps for the new question and reaches the correct answer, 9. In a zero-shot example on a US Medical Licensing Examination question, "Let's think step by step" led the model to reason through how HIV infection works before choosing the correct option.

Chain-of-thought improves arithmetic, commonsense and symbolic reasoning across many benchmarks, and its success set off a large wave of research on LLM reasoning. Two cautions: the gains appear mainly in large models, so scale still matters, and a written chain of steps can still contain errors, so it needs checking rather than trusting.

✎ reading noteReading note: the lines "Goal…" and "Target Audience…" at the top, and the labels KEY_FINDING, PROCESS_STEP and DEFINITION, are left over from the image's production brief, not content. "Scaling is not the solution" is too strong: the course says scaling pre-training is not the best way to improve reasoning, and the chain-of-thought paper found the benefit appears only in sufficiently large models. The medical example shows one exam question answered correctly, not a model passing the exam.

Chain-of-thought gets reasoning out of a model by prompting. The next step was to train models to reason. The course's example is DeepSeek-R1, released in January 2025.

Its weights are open. It has 671 billion weights in total, but only 37 billion are active for each token, so it is cheaper to run than its total size suggests; the course's "37 billion" refers to this active part. The widely quoted training cost of about 6 million US dollars covers only the final training run of its base model (DeepSeek-V3), and the course notes that such estimates have been questioned. During training, the model started to re-check and revise its own steps inside its thinking section, a behaviour the authors describe as emerging from reinforcement learning rather than being programmed. That is the self-correction I had in mind when I described chain-of-thought; strictly, it belongs to reasoning models like this one.

The course also lists open questions about such models: training details are only partly published; smaller models may depend on distilling bigger ones; and benchmark results may be inflated by data contamination.

Both are easy to miss because the model's text looks fine. Contamination is a problem in all of machine learning, but more serious for LLMs, which are trained on so much of the web that public benchmarks are easily included. A separate issue is poor-quality or biased training data, which can also lead to wrong or skewed answers. The next lecture covers hallucination, stale information, training data and bias in more detail.

An LLM builds on the neural language model and evolves it. The first big step is from words to tokens. In the earlier notes on language models we kept saying "words", because we started from how we ourselves learn language, and in daily life we use whole words. But to teach a machine language, we break sentences apart, and break words apart too, for example into roots and affixes; then we build connections between the pieces. That is what the earlier notes covered. A large language model is still a neural language model, and it rests on that same early idea: tokenization. As a quick recap, a token doesn't have to be a whole word; it can be a phrase-like chunk or part of a word. Splitting text this way makes learning more efficient, and it lets the model use a longer context and predict more precisely.

The second step is understanding context. A word no longer has one fixed meaning; the model works out what it means in this particular context. The attention mechanism is the milestone here: it calculates how much tokens influence each other and weights them accordingly, which shapes the output. In the end, it comes back to the same basic point as the earlier notes. We first learned relationships between words and between sentences; now the model uses mechanisms to handle the relationships between tokens.

Prompting means giving the model instructions when I interact with it. The model was built from a very broad range of information; a prompt makes that information more precise by setting a scope and saying what I want. This does not retrain the model. The more detailed the instructions, the more the model can tell what kind of task it is facing, and the better it adapts.

Sometimes the answers are still not good enough. So developers ask the model to break its answer down and show how it reasoned, step by step, the way we solve a problem. Writing out each step makes the final answer more accurate. Models trained to reason go further, noticing problems in their own steps and correcting them.

Common challenges remain, such as data contamination and hallucination; I will leave them for the next note. For model families, it is enough for now to know that GPT and Llama are two of the most common, and that they differ in things such as whether their weights are open.

This note is based on my study notes from the University of British Columbia (UBC) micro-certificate program "Natural Language Processing to Improve Patient Care," in the course "Overview of NLP and Large Language Models" (the Large Language Models lecture by Giuseppe Carenini). Technical details were checked against:

• Jurafsky and Martin, Speech and Language Processing (3rd edition draft): Chapter 7, Transformers and Pretraining.
• Vaswani et al. (2017), Attention Is All You Need.
• Brown et al. (2020), Language Models are Few-Shot Learners (GPT-3, in-context learning).
• Wei et al. (2022), Chain-of-Thought Prompting Elicits Reasoning in Large Language Models; Kojima et al. (2022), Large Language Models are Zero-Shot Reasoners.
• DeepSeek-AI (2025), DeepSeek-R1 and the DeepSeek-V3 technical report.
• Meta, Llama 3.1 license; OpenAI, gpt-oss-120b model card.

Try it yourself

• Tiktokenizer: paste any sentence and see how a GPT tokenizer splits it into tokens, with the token count. Try unhappy, a drug name, or a sentence in Chinese.
• Transformer Explainer: runs a small GPT-2 in your browser; type a prompt and watch the attention and the next-token probabilities.
• The Illustrated Transformer: Jay Alammar's step-by-step visual walk-through of attention and the Transformer blocks.