How Do LLMs “Think”?

Large language models (LLMs) can write, reason, code, and chat in a way that feels close to human thought. Yet their “thinking” is not a stream of conscious ideas. It is a structured statistical process that turns text into numbers, runs those numbers through many layers of computation, and then produces the next token in a sequence.

Text Becomes Tokens

LLMs do not read words the way people do. They break text into tokens: chunks such as “cat”, “ing”, “tion”, punctuation marks, or even whitespace patterns. Tokenization matters because the model’s vocabulary is built from these pieces.

When you type a prompt, it is converted into a token list. Each token is mapped to an integer ID. This gives the model a discrete sequence it can process.

Tokens Become Vectors (Embeddings)

The token IDs are then converted into embeddings, which are dense vectors (lists of numbers). An embedding places tokens in a high-dimensional space where the model can represent similarity and usage patterns. For example, tokens used in similar contexts end up with vectors that often point in related directions.

These vectors are not fixed “definitions” of words. They are flexible representations tuned to help the model predict what comes next given context.

Position and Order: More Than Just Words

A sequence is not only about which tokens appear, but also where they appear. The model adds positional information so it can treat “dog bites man” differently from “man bites dog”.

Modern systems use positional encodings (or related methods) so each token vector carries some sense of order and distance.

Attention: Selecting What Matters

The headline feature of many powerful LLMs is self-attention. Attention lets the model decide which earlier tokens are most relevant to each current token position.

In practice, attention builds a set of weighted links between tokens. If you write:

• “Sara put the book on the table. She picked it up later.”

The model can assign high attention to connect “She” with “Sara”, and “it” with “book”. This is not a symbolic pointer in a database; it is a learned pattern of weights computed during inference.

Attention is computed in multiple “heads,” each specializing in different relations (syntax, topic tracking, quotation structure, code brackets, and so on). Stacking many layers yields increasingly abstract features.

Layers Build Features, Not Facts

Each transformer layer transforms the vectors through:

• Attention (mixing information across the sequence)
• Feed-forward networks (nonlinear transforms applied per position)
• Residual connections and normalization (to keep training stable and information flowing)

As layers accumulate, the representation of each token becomes a rich summary of the prompt plus the token’s role in it. This is closer to “feature building” than to retrieving explicit stored sentences.

Prediction: The Next Token Is a Probability Distribution

When the model is ready to respond, it produces a probability distribution over the vocabulary for the next token. The top candidates might be words, punctuation, or partial word pieces. Decoding chooses one token using a strategy such as:

• Greedy selection (pick the highest probability)
• Sampling (introduce controlled randomness)
• Beam search (explore several candidates)

Temperature, top-k, and top-p settings shape how conservative or creative the output feels.

Why It Looks Like Reasoning

Reasoning-like behavior can appear because the model has learned patterns that match reasoning steps found in its training data: explanations, proofs, stepwise problem solving, and self-correction language. When prompted to “think step by step,” the model is guided to produce tokens that resemble intermediate reasoning.

Yet the mechanism remains the same: predicting the next token from a context-conditioned distribution. Multi-step answers are just longer sequences where each new token updates the context for the next prediction.

What “Thinking” Is in Practice

In powerful LLMs, “thinking” is:

• Compressing your prompt into vectors
• Repeatedly computing attention-based mixes of context
• Transforming those representations through deep layers
• Emitting tokens that best fit learned patterns

There is no inner voice in the human sense—only a high-dimensional calculation that can produce text that reads like thought.