What Actually Happens When You Run a Hugging Face Pipeline?
A pipeline() call runs four documented stages: tokenization, model inference, post-processing and output. The tokenizer turns text into token ids with Hugging Face's Rust tokenizers library. The model is a PyTorch module whose weights from_pretrained() loads from safetensors files when they exist. generate() then calls the model one step at a time, by default greedily keeping the most likely next token, up to 20 new tokens unless the model's config says otherwise. Attention runs through PyTorch's SDPA, which picks the fastest kernel on a CUDA GPU and the C++ implementation on the CPU; on Apple silicon the operations run as MPS kernels. The output ids are decoded back into text.
Showing pipeline() on an NVIDIA GPU (CUDA): 10 stages, 10 steps.
Your program
Runtime
Framework core
Model file
Kernel
Hardware
The selected path, step by step
Your pipeline() call creates and calls Pipeline: Your code builds a pipeline for a task and calls it with text.
Pipeline loads safetensors weights: The model's weights are loaded with from_pretrained(), from safetensors files when available.
safetensors weights fill PyTorch model: The weights are loaded into the model class on the chosen device.
Pipeline tokenizes with Tokenizer (Rust core): Input text is tokenized first.
Tokenizer (Rust core) token ids go to generate() loop: The token ids start the generation loop.
generate() loop calls forward, one token at a time PyTorch model: Each step runs the model and, with greedy search, keeps the most likely next token.
PyTorch model computes attention with PyTorch SDPA: Unless another attention implementation is set, attention layers use PyTorch's SDPA.
PyTorch SDPA picks Fastest CUDA kernel: On a CUDA GPU, SDPA selects the fastest available attention kernel.
Fastest CUDA kernel runs on NVIDIA GPU: The kernels run on the NVIDIA GPU.
generate() loop hands output ids to Post-processing: The finished token ids go to post-processing.
Which device it picks
If you do not pass a device, the pipeline places the model on the first accelerator it finds (a CUDA GPU, Apple silicon's MPS, or others) and uses the CPU only when there is none. Passing device="cpu" or device="mps" forces a choice.
Where Python hands off
Most of the work in this path is not Python: tokenization runs in Rust, and the model's layers run as PyTorch kernels. For what PyTorch does with those operations on each device, see what happens when you run a PyTorch model. For serving many requests at once, engines such as vLLM and SGLang add batching schedulers and KV-cache management on top of the same kind of model.
Every stage
Your pipeline() call
programming language
Python code such as pipeline(task="text-generation", model=...), which picks a default model and preprocessor for the task unless you name one.
Each task is configured to use a default pretrained model and preprocessor, but this can be overridden with the model parameter if you want to use a different model.
Pipeline workflow is defined as a sequence of the following operations: Input -> Tokenization -> Model Inference -> Post-Processing (task dependent) -> Output
Produces new tokens one step at a time by calling the model; by default it is greedy search, up to 20 new tokens unless the model's generation config says otherwise.
Greedy search is the default decoding strategy. It selects the next most likely token at each step.
On the mps device, PyTorch runs operations as Metal Performance Shaders graphs and kernels.
It introduces a new device to map Machine Learning computational graphs and primitives on highly efficient Metal Performance Shaders Graph framework and tuned kernels provided by Metal Performance Shaders framework respectively.
Used only when no accelerator is available, or when you pass device="cpu".
By default, when device is left unset, Pipeline automatically places the model on the first available accelerator (CUDA GPU, Apple Silicon MPS, XPU, …) and only falls back to a CPU when none is available.
The first choice when device is left unset and a CUDA GPU is present.
By default, when device is left unset, Pipeline automatically places the model on the first available accelerator (CUDA GPU, Apple Silicon MPS, XPU, …) and only falls back to a CPU when none is available.
Documented path for one text-generation pipeline() call with the PyTorch backend, default greedy decoding and the default SDPA attention, on a CPU, an NVIDIA GPU or an Apple silicon GPU. Other tasks, other attention implementations (FlashAttention, paged attention), quantization, multi-GPU placement with device_map, Accelerate and other frameworks are out of scope. What happens inside PyTorch on each device is on the PyTorch page.
How this page is checked
Every stage and step above quotes the source that supports it. The quotes were checked against the Transformers documentation (Pipeline tutorial, Pipelines reference, Loading models, Tokenizers, Attention backends, Generation strategies) and the PyTorch MPS backend notes on 2026-11-12. The data is a versioned JSON file under the same CC BY 4.0 license as the dataset. Found something wrong or out of date? Open an issue with the better source.