Short answer: on a supported Windows PC, install Ollama from its official site, download a small local instruction model, and call its loopback API from PowerShell. A completed response proves that this request ran; checking its content is a separate step. Keep the service local and use non-sensitive sample text.
Read Local AI Hardware and Tokens, Context & Prompting. You need permission to install software, internet access for the installer/model, and enough free disk and working memory.
A home cinema, one component at a time
A home cinema needs a player, a film, and a working connection to the display. Installing the player doesn't download every film. A film appearing in the library doesn't prove it can play. A picture on screen doesn't prove it is the film you intended.
Here, Ollama is the player and local server, the downloaded model is the film, and PowerShell is the remote control. We will check the remote, the server connection, the model library, and finally generation. This separation makes an error actionable instead of turning every failure into "AI doesn't work."
1. Check the platform and choose the model
The official Windows guide lists Windows 10 22H2 or newer and separate GPU/driver requirements. Prefer a currently supported Windows release. Ollama can use supported CPU or GPU backends; a dedicated GPU is not required for this small example. Check the current Windows guide and hardware matrix for your exact machine.
This walkthrough uses qwen2.5:0.5b, a small instruction-tuned text model. The Ollama library entry reviewed for this article lists roughly 398 MB and Q4_K_M quantization. That is an artifact size, not the installer size or total runtime memory requirement. It is chosen to make a first download manageable, not as a recommendation for the strongest current model.
The specific Qwen2.5-0.5B-Instruct model card specifies a 32,768-token context, despite broader family descriptions advertising larger limits. We will request only 2,048. Model tags can change: record the ID shown by your installation and inspect ollama show. Check the model's license before reuse or redistribution.
2. Install and verify the runtime
Download the Windows installer from Ollama's official download page, run it under your organization's software policy, and open a new PowerShell window. The standard installer runs under your user account and starts a background application. Reserve space for the runtime as well as models; the Windows guide currently calls for at least 4 GB for the binary installation.
Get-Command ollama
ollama --version
Invoke-RestMethod -Uri 'http://127.0.0.1:11434/api/version' -TimeoutSec 10
ollama listGet-Command should locate the executable. The version command reports the installed CLI, while /api/version checks the server and returns a version field. ollama list lists downloaded models; an empty list is normal on a fresh install. None of these checks proves generation works yet.
If the server is unavailable, launch Ollama from the Start menu. Don't start another ollama serve process while the tray application's server already owns port 11434. A "port in use" error is a server-start problem, not evidence that a model is too large.
3. Keep the request local
Ollama binds to 127.0.0.1:11434 by default. A previous installation may have changed that. Check the listener before entering private material:
Get-NetTCPConnection -State Listen -LocalPort 11434 |
Select-Object LocalAddress, LocalPort, OwningProcessLoopback addresses 127.0.0.1 and ::1 stay on the host. 0.0.0.0, ::, or a LAN address indicates a broader binding. For this exercise, keep the configured host at 127.0.0.1:11434; don't add firewall exceptions, public tunnels, or wildcard browser origins. The local API does not require authentication. Loopback narrows exposure but is not a boundary against other processes on your computer.
For Ollama's local-only mode, set the user environment variable OLLAMA_NO_CLOUD to 1 through Windows Environment Variables, then quit and restart the Ollama application. The official FAQ documents this control for cloud models and web search. It doesn't block installer updates, model downloads, or network activity from a separate chat application. Use the local model tag below, and verify your application's full data path before handling sensitive information.
4. Download and inspect one model
The next command downloads model data and consumes disk space. On a metered connection, check the current library size first. No cloud account is needed for this public local model.
ollama pull qwen2.5:0.5b
ollama list
ollama show qwen2.5:0.5bConfirm that the model appears in the local list and that the shown architecture, parameters, and quantization match your intended artifact. Ollama's usual Windows model location is %USERPROFILE%\.ollama\models, unless OLLAMA_MODELS changes it. The cinema now has a film in its library; playback remains the next test.
5. Send a bounded PowerShell request
This example uses /api/generate, constructs JSON with PowerShell's serializer, and disables streaming so the response is one JSON object. It doesn't attach files, access websites, or execute generated text.
$body = @{
model = 'qwen2.5:0.5b'
prompt = 'Note: The cabinet has 3 shelves. How many shelves does it have? Reply with the number only.'
stream = $false
keep_alive = '2m'
options = @{
num_ctx = 2048
num_predict = 32
temperature = 0
}
} | ConvertTo-Json -Depth 4
$result = Invoke-RestMethod -Method Post `
-Uri 'http://127.0.0.1:11434/api/generate' `
-ContentType 'application/json' -Body $body -TimeoutSec 120
$result.response
$result | Select-Object model, done, done_reason, prompt_eval_count, eval_count
ollama psThe backtick at the end of each continued PowerShell line must be its final character. Run the block in the same PowerShell session. The configured timeout limits how long the client waits; it is not a guaranteed deadline for server-side cleanup.
The answer key is 3. This is a check to perform, not a claim that every run of this small model returns exactly that string. done: true means generation completed, not that the answer passed the check. A length-related stop may mean the output cap was reached. eval_count counts generated tokens; it need not equal the character or word count.
ollama ps should show the loaded model while its keep-alive period remains active, along with placement information. That is the cinema actually playing the film, rather than merely listing it. If the model unloads before inspection, repeat the short request instead of assuming the earlier run failed.
6. Test a missing fact
Replace the prompt with: Note: The cabinet has 3 shelves. What color is it? Use only the note. If absent, reply "not stated". Rebuild $body and repeat the request. The answer key is not stated. Inventing a color is a grounding failure even if the endpoint returns HTTP 200.
Then try the incident-summary prompt with a larger output cap such as num_predict = 192. Check every factual claim. This tiny model may struggle with multiple constraints; that is evidence about the model/task combination, not automatically an installation defect. Each /api/generate request here supplies its own prompt; it doesn't implicitly include earlier requests.
Locate the failed boundary
| Symptom | Check | Interpretation / next step |
|---|---|---|
ollama not recognized | Get-Command ollama; installer completion | Open a new terminal after installation. This is executable discovery, not model generation. |
| Connection refused | Tray app, /api/version, listener on 11434 | Start the local application. Don't solve a local failure by exposing the server to the LAN. |
| Model not found | ollama list and exact tag spelling | Pull the intended model and use that full tag in the request |
| Download or certificate error | Connectivity, approved proxy, disk space | Follow organizational proxy/trust setup; don't disable certificate validation |
| Load failure or very slow request | Available memory, backend logs, ollama ps | Keep context small; investigate placement and compatibility before increasing the timeout |
| JSON lines instead of one object | stream setting | Default streaming returns incremental events; this example requires false |
| Wrong or invented answer | Exact prompt, model ID, answer key, stop reason | Separate quality failures from transport or installation failures |
On Windows, logs are under %LOCALAPPDATA%\Ollama, including server.log. Inspect locally first; remove sensitive paths, prompts, and identifiers before sharing diagnostics. Record both client and server versions when investigating mismatches.
Stop and clean up deliberately
ollama stop qwen2.5:0.5b
ollama psstop unloads the model from working memory; it doesn't delete the download or stop the Ollama server. Quit the tray application to stop the background application when you are finished. To remove this model's local entry and unneeded data, use ollama rm qwen2.5:0.5b only when you no longer need it. Shared model data may remain for other installed models.
First, verify that the chosen local model completed the request. Second, compare the answer with the supplied evidence. A functioning inference server is not a factual guarantee.
Continue reading
Next: RAG Explained, for choosing source passages before generation. Revisit memory planning before trying larger models or longer contexts.