Short answer: a large language model (LLM) is a learned numerical model of language. A typical text-generating LLM predicts tokens using the input and the tokens already generated. Training changes its learned parameters; inference uses them. A useful AI application adds software around that model, such as conversation storage, retrieval, tools, permissions, and a user interface.
Start here. No programming, machine learning, or networking background is required.
Think of a dubbing studio
A dubbing studio needs a performer, recording equipment, a script, and an editor. Rehearsing a large range of material changes what the performer can do. Recording a particular scene uses those abilities with today's script. The editor chooses what reaches the audience.
Use that separation for AI: training is the rehearsal, the trained model supplies learned patterns, the inference runtime is the equipment doing the computation, and the application assembles the script and presents the result. The analogy stops at human understanding. A transformer calculates with arrays of numbers; it doesn't acquire a performer's lived experience or guarantee that its lines are true.
AI, machine learning, and generative AI
| Term | What it means here | Concrete example |
|---|---|---|
| Artificial intelligence (AI) | The broad field of building systems that perform tasks associated with intelligence. Not every approach learns from data. | A rule-based planning system can use explicitly written rules |
| Machine learning (ML) | Learning patterns from examples instead of writing every decision rule | A classifier learns to label messages as spam or not spam |
| Deep learning | ML using neural networks with many processing layers | A transformer processes token representations through attention and feed-forward layers |
| Generative AI | Models that produce content, such as text, images, or audio | A diffusion model generates an image; a causal language model generates text |
| Large language model | A language model trained at substantial scale. There is no universal parameter-count threshold for "large." | The Qwen2.5 family includes several model sizes and instruction-tuned variants |
The studio can classify a recording, transcribe it, or create a new performance. Those are different jobs. Likewise, not every AI task needs a chat model: a classifier, search engine, calculator, or ordinary rule may be more reliable and cheaper for a narrowly defined task.
Training changes the model; inference uses it
Parameters, often called weights, are learned numbers inside the model. During causal-language-model pretraining, the system predicts subsequent tokens in training text. A loss function measures prediction error, and an optimizer updates weights to reduce that error across examples. This is not the same as building a searchable folder of every training document, although models can memorize and reproduce some training material.
Post-training adapts the pretrained model, for example with instruction-and-response examples or preference-based training. An instruction-tuned checkpoint is usually more suitable for a beginner's chat task than its base-model counterpart. Fine-tuning changes weights or trained adapter parameters. It isn't just adding a sentence to the prompt.
Inference loads a checkpoint and computes an output. In an ordinary chat request, the weights stay fixed. Providing an example in the current context can change the answer without changing those weights. Think of handing the performer a new script, not sending them through another rehearsal program.
How one answer is produced
- Prepare input: the application assembles instructions, the current question, and selected conversation history. A chat template encodes role boundaries in the model's expected format.
- Tokenize: a tokenizer converts text to token IDs. A token may represent a word, a word fragment, punctuation, or bytes. It is not necessarily one word.
- Process context: in a typical decoder-only transformer, causal attention combines information from permitted earlier positions. Feed-forward layers also transform the representations. Attention is a numerical operation, not a truth checker.
- Select a next token: the model produces scores called logits. The runtime applies the decoding strategy, such as greedy selection or sampling.
- Continue: the selected token becomes part of the sequence for the next step. Generation stops on an end token, a configured stop condition, or a limit. The tokenizer decodes the output into text.
The studio records a line using the script available at that moment. The model similarly works from the input it actually receives, not from every conversation ever shown in an application's sidebar. A runtime commonly uses a key/value (KV) cache to reuse intermediate attention data while generating; that cache is not a permanent update to the model's weights.
Model, runtime, application
| Layer | Responsibility | Example and limit |
|---|---|---|
| Model package | Weights, architecture metadata, tokenizer, and often a chat template | Qwen2.5-0.5B-Instruct identifies a particular instruction-tuned model, not a chat website |
| Inference runtime / serving software | Load a compatible model, allocate memory, execute generation, expose an interface | llama.cpp is an inference engine; Ollama packages model management and serving around inference backends |
| Application | Collect input, manage history, authorize tools/retrieval, display output | A chat interface might call Ollama's /api/chat. The interface's history policy is not a model capability. |
The same performer sounds different with a different script or edit. The same model can answer differently because an application changes its system instructions, truncates history, retrieves documents, or uses different sampling settings. Troubleshooting starts by identifying which layer changed.
Choose a task you can check
Give a model this synthetic note: Maintenance starts at 09:00 UTC and lasts 30 minutes. Ask it to extract the start time and duration. You can verify both values directly. An answer that adds a date, location, or service name has introduced information the note never supplied.
Now ask, "Is tomorrow's maintenance approved?" The note cannot answer that. A confident yes is a hallucination: plausible output unsupported by the available evidence. Fluency and token probability are not factual confidence. Adding "be accurate" doesn't give the model access to the approval system.
For high-impact claims, check authoritative evidence or use a deterministic tool. For an exact sum, calculate it. For a current policy, retrieve the current policy and verify the cited passage. Never execute generated shell commands merely because they look familiar; inspect their targets, permissions, and side effects first.
Local and hosted describe where computation runs
A local runtime uses your machine's RAM, processor, and possibly GPU. A hosted endpoint runs elsewhere and introduces a service's pricing, retention, identity, and network boundaries. Neither is automatically more accurate. Both consume resources during inference, even when no per-request bill appears.
A studio in your house can still upload recordings. Likewise, a local chat application can use a cloud model, remote embedding service, web search, synchronization, or telemetry. Verify the complete data path before using sensitive material. Downloadable weights also do not automatically mean unrestricted licensing or fully open training data.
Wrong facts, lost history, and a failed model load are different problems. Check evidence for the first, the application's submitted context for the second, and runtime compatibility and memory for the third.
Continue reading
Next: Tokens, Context & Prompting. Then connect the computation to local hardware, or see how RAG supplies external evidence.