What Actually Runs When You Send Ollama a Prompt?

Ollama serves requests and selects a compatible model runner; the runner executes numerical work. Current documented local paths include llama.cpp-backed GGUF support and an MLX engine for supported Apple-silicon models. A Go service does not mean the matrix kernels are Go code, and an Ollama API request does not prove inference is local: cloud endpoints are separate.
Supported local route

Showing a compatible GGUF model on CPU: 6 stages, 5 steps.

Your local request sends local API request Ollama Go service Ollama Go service validates and schedules Runner scheduling Runner scheduling uses compatible GGUF engine llama.cpp GGUF path Runner scheduling uses MLX engine MLX engine llama.cpp GGUF path uses CPU backend ggml CPU kernels llama.cpp GGUF path uses CUDA backend CUDA kernels MLX engine uses Metal primitives MLX Metal kernels ggml CPU kernels executes on CPU CUDA kernels executes on NVIDIA GPU MLX Metal kernels executes on Apple silicon GPU Your local request application Ollama Go service runtime Runner scheduling runtime llama.cpp GGUF path runtime MLX engine runtime ggml CPU kernels kernel CUDA kernels kernel MLX Metal kernels kernel CPU hardware NVIDIA GPU hardware Apple silicon GPU hardware

The selected path, step by step

  1. Your local request sends local API request Ollama Go service: This path uses the local endpoint, not an Ollama-hosted cloud request.
  2. Ollama Go service validates and schedules Runner scheduling: The Go service resolves a compatible runner after validating options and capabilities.
  3. Runner scheduling uses compatible GGUF engine llama.cpp GGUF path: This selection describes the documented llama.cpp-backed GGUF integration.
  4. llama.cpp GGUF path uses CPU backend ggml CPU kernels: The CPU-only route executes supported work through CPU kernels.
  5. ggml CPU kernels executes on CPU: The CPU-only numerical path terminates at CPU hardware.

Is Ollama itself the model or the inference engine?

Ollama provides request handling, model management and compatible runners. The Go server source shows validation and scheduling before inference. The model’s weights are data consumed by an execution engine; neither the model nor the chat endpoint is a programming language.

The diagram distinguishes the service layer from the numerical runtime. This matters when answering what Ollama is written in: Go is evidence for its service implementation, while the selected native engine has its own implementation and kernels. The llama.cpp path and MLX path explain those layers in more detail.

Does every Ollama model use llama.cpp?

No. The GGUF support announcement documents llama.cpp integration alongside Ollama’s MLX engine. The MLX announcement describes an Apple-silicon path for supported architectures. Model compatibility and the installed version determine which claims apply.

The controls compare documented paths; they do not expose a real Ollama engine-selection API. A compatible GGUF route and an MLX route need not accept identical model files or support identical features. Do not infer the engine from a model’s marketing name alone.

Why does Ollama use the CPU instead of the GPU?

A CPU-only model can run in system memory. GPU execution additionally needs a compatible engine, hardware support, dependencies and enough available memory for the configured workload. The FAQ documents ollama ps and its PROCESSOR column, including fully CPU, fully GPU and mixed placement. That report is more informative than assuming a GPU installation guarantees full offload.

The CUDA choice illustrates supported offloaded work. Real deployments can use a mixture of CPU and GPU, especially when the model does not fit entirely on a device. The diagram excludes that partial-offload detail rather than asserting that all CUDA requests are GPU-only.

What changes on Apple silicon?

The documented MLX engine uses Apple’s array framework and unified-memory architecture for supported models. MLX GPU primitives are implemented using Metal, as the extension guide explains. This is different from treating a Mac as an NVIDIA CUDA system.

Other local routes can still exist on Macs. The point of the MLX branch is to show one verified engine path, not to claim every model on every Apple computer uses it. Check installed-version release notes and model support before applying this diagram to a specific deployment.

Does using the Ollama API mean my request stays local?

No. The API introduction documents both local and cloud base URLs, and local servers can also be used with cloud models. This page deliberately chooses a downloaded local model. Endpoint, model selection and serving configuration establish the execution boundary; the client library name alone does not.

How do GGUF, model size and hardware fit together?

GGUF is a model file format consumed by a compatible runtime. It does not describe one fixed processor or make all model architectures interchangeable. Memory placement and the chosen backend determine where computation occurs. This page avoids model rankings, speed promises and guessed internals, and records the engine and device conditions for each illustrated path.

Every stage

Your local request

application

A client sends a chat request to the local Ollama service using an already downloaded supported model.

Ollama Go service

runtime

The service handles the request and validates model capabilities and options before selecting a runner.

In the atlas: Ollama

Runner scheduling

runtime

Ollama schedules a compatible engine for the chosen model. This is not a promise that every architecture uses llama.cpp.

MLX engine

runtime

Supported Apple-silicon model paths can use Ollama’s MLX engine, which is distinct from the GGUF/llama.cpp route.

In the atlas: MLX

ggml CPU kernels

kernel

The selected CPU-only llama.cpp route uses CPU numerical kernels. No universal GPU requirement is implied.

In the atlas: ggml

CUDA kernels

kernel

The compatible CUDA route uses NVIDIA GPU acceleration through llama.cpp’s configured GPU backend.

In the atlas: CUDA

MLX Metal kernels

kernel

MLX primitives for the illustrated Apple-silicon GPU work use Metal kernels.

CPU

hardware

The CPU-only example keeps the selected model in system memory and executes its numerical work on CPU.

NVIDIA GPU

hardware

The CUDA example executes supported offloaded work on a compatible NVIDIA GPU; partial CPU offload is not shown.

Apple silicon GPU

hardware

The supported MLX route executes its GPU primitives through Metal on Apple silicon.

Every step

FromStepToWhenWhat happensSource
Your local requestsends local API requestOllama Go serviceany supported local routeThis path uses the local endpoint, not an Ollama-hosted cloud request.Project docs
Ollama Go servicevalidates and schedulesRunner schedulingany supported local routeThe Go service resolves a compatible runner after validating options and capabilities.Source repository
Runner schedulinguses compatible GGUF enginellama.cpp GGUF pathGGUF / CPU or GGUF / NVIDIA CUDAThis selection describes the documented llama.cpp-backed GGUF integration.Project docs
Runner schedulinguses MLX engineMLX engineMLX / Apple siliconThis selection assumes a model supported by the MLX engine on Apple silicon.Project docs
llama.cpp GGUF pathuses CPU backendggml CPU kernelsGGUF / CPUThe CPU-only route executes supported work through CPU kernels.Source repository
llama.cpp GGUF pathuses CUDA backendCUDA kernelsGGUF / NVIDIA CUDAThe configured CUDA route executes supported device work through GPU kernels.Source repository
MLX engineuses Metal primitivesMLX Metal kernelsMLX / Apple siliconMLX’s supported GPU primitives are implemented with Metal kernels.Project docs
ggml CPU kernelsexecutes onCPUany supported local routeThe CPU-only numerical path terminates at CPU hardware.Project docs
CUDA kernelsexecutes onNVIDIA GPUany supported local routeThe configured CUDA backend executes device work on the NVIDIA GPU.Source repository
MLX Metal kernelsexecutes onApple silicon GPUany supported local routeMetal primitives execute on the Apple-silicon GPU.Project docs

What this does not show

Local text inference with downloaded models supported by the selected engine: GGUF/llama.cpp on CPU or NVIDIA CUDA, or the documented MLX engine on Apple silicon. These are version- and model-dependent illustrative paths, not a user-facing engine-switch command or a guarantee of universal model compatibility. Cloud models, AMD/Vulkan routes, partial offload, multimodal processing and detailed token sampling are not depicted.

How this page is checked

Every stage and step above quotes the source that supports it. The quotes were checked against the Ollama API and FAQ, Go server source, GGUF and MLX announcements, llama.cpp build documentation and MLX kernel documentation on 2026-11-11. The data is a versioned JSON file under the same CC BY 4.0 license as the dataset. Found something wrong or out of date? Open an issue with the better source.

See this path in the graph explorer →

What these tools are written in is in the main atlas: Ollama, llama.cpp, MLX, ggml, CUDA. More execution paths: How Does a Rust LLM Generate Tokens with Candle, What Actually Happens When You Run JAX Code, What Runs Underneath a Keras 3 Model, How Does llama.cpp Run a Model on Your Hardware, How Does MLC LLM Turn a Compiled Model into Generated Tokens, What Actually Happens When You Run MLX on a Mac, How Does ONNX Runtime Execute a Model Across Hardware, What Actually Happens When You Run a PyTorch Model, What Actually Happens When SGLang Serves a Request, How Does TensorFlow Run Python on a CPU or GPU, What Actually Happens When TensorRT-LLM Runs a Model, What Actually Happens When You Run a Hugging Face Pipeline, What Actually Happens When vLLM Serves a Request, or compare them all.

Explore the full graph →