What Do LLM Weights Do?
Large language models (LLMs) are often described as “just weights.” That phrase sounds dismissive, but it’s accurate in a technical sense: the model’s learned behavior is stored in a huge collection of numbers. When you ask an LLM a question, those numbers guide how it turns your text into a response.
What “weights” are in a published LLM
Weights are the learned parameters of a neural network. They are typically floating‑point numbers (sometimes stored in reduced precision) arranged in matrices and vectors. Each weight influences how strongly one internal feature affects another.
During training, the model is shown many examples of text and repeatedly adjusts these weights to reduce prediction error. After training, the weights are “frozen” and published as files that can be loaded for inference (answering prompts). If two people load the same weights and use the same settings, they will get highly similar behavior.
Weights as a compressed record of patterns
Weights don’t store a library of sentences. They store statistical patterns: how words tend to follow each other, how concepts relate, and how to produce structured outputs like lists, code, or dialogue.
This is compression in a practical sense. Many gigabytes of training text get distilled into a smaller set of numbers. The model can generalize because the weights capture reusable patterns rather than memorized copies of every line.
What happens when you ask a question
When you send a prompt, the model does not “look up” an answer. It performs a series of calculations that transform your input into a probability distribution over the next token (a token is a chunk of text, often a word or part of a word).
A simplified flow looks like this:
- Tokenization: Your text is split into tokens.
- Embedding: Each token is mapped to a vector using an embedding matrix (weights). This converts discrete token IDs into continuous numbers the network can process.
- Transformer layers: The vectors pass through many layers containing attention and feed‑forward submodules, each with its own weights.
- Output projection: A final matrix converts the last hidden representation into scores (“logits”) for every token in the vocabulary.
- Sampling/decoding: The system picks the next token based on those scores and repeats the process until it decides to stop.
Why attention weights matter
Transformers rely on self‑attention to decide which earlier tokens should influence the next token. Attention uses learned weight matrices to create three representations for each token: queries, keys, and values.
During inference, attention computes similarity between queries and keys to decide what to focus on. The values then contribute information to the current position. This is how the model keeps track of relationships like:
- A pronoun referring to a noun earlier in the paragraph
- A question asking for a specific item in a list
- Code requiring consistent variable names
The learned matrices shape what kinds of relationships the model is able to express and how efficiently it can retrieve relevant context from the prompt.
Feed‑forward weights build features
Between attention blocks, each layer usually includes a feed‑forward network (often two linear transformations with a nonlinearity). These weights build and refine internal features: representations of concepts, styles, and task patterns. Attention chooses where to look; feed‑forward parts compute what to do with that information.
Why the same model can answer many tasks
Instruction following, summarization, translation, and coding all rely on patterns present in training and fine‑tuning. Fine‑tuning updates some or all weights so the probability distributions shift toward preferred behaviors (for example, being more helpful, structured, or safe).
Some published models also include separate adapter weights (such as LoRA). These are smaller weight sets that modify the base model’s behavior without changing every parameter.
What weights do not do
Weights do not guarantee truth. The model outputs tokens that are statistically plausible given the prompt and its learned patterns. If the prompt lacks needed facts, or the training patterns are misleading, the produced text may be incorrect while still sounding confident.
Why weights are the “model”
Code defines the architecture, but the weights define the learned content. When you ask a question, you are triggering a deterministic chain of matrix multiplications and nonlinear functions driven by those weights, producing one token after another until an answer is formed.
