What Actually Happens When You Run a Hugging Face Pipeline?

A pipeline() call runs four documented stages: tokenization, model inference, post-processing and output. The tokenizer turns text into token ids with Hugging Face's Rust tokenizers library. The model is a PyTorch module whose weights from_pretrained() loads from safetensors files when they exist. generate() then calls the model one step at a time, by default greedily keeping the most likely next token, up to 20 new tokens unless the model's config says otherwise. Attention runs through PyTorch's SDPA, which picks the fastest kernel on a CUDA GPU and the C++ implementation on the CPU; on Apple silicon the operations run as MPS kernels. The output ids are decoded back into text.
Device

Showing pipeline() on an NVIDIA GPU (CUDA): 10 stages, 10 steps.

Your pipeline() call creates and calls Pipeline Pipeline loads safetensors weights safetensors weights fill PyTorch model Pipeline tokenizes with Tokenizer (Rust core) Tokenizer (Rust core) token ids go to generate() loop generate() loop calls forward, one token at a time PyTorch model PyTorch model computes attention with PyTorch SDPA PyTorch SDPA picks Fastest CUDA kernel PyTorch SDPA falls back to C++ attention PyTorch SDPA runs as MPS kernels Fastest CUDA kernel runs on NVIDIA GPU C++ attention runs on CPU MPS kernels run on Apple silicon GPU generate() loop hands output ids to Post-processing Your pipeline() call programming language Pipeline framework component safetensors weights model format Tokenizer (Rust core) framework component PyTorch model framework component generate() loop runtime Post-processing framework component PyTorch SDPA kernel C++ attention kernel MPS kernels kernel Fastest CUDA kernel kernel CPU hardware Apple silicon GPU hardware NVIDIA GPU hardware

The selected path, step by step

  1. Your pipeline() call creates and calls Pipeline: Your code builds a pipeline for a task and calls it with text.
  2. Pipeline loads safetensors weights: The model's weights are loaded with from_pretrained(), from safetensors files when available.
  3. safetensors weights fill PyTorch model: The weights are loaded into the model class on the chosen device.
  4. Pipeline tokenizes with Tokenizer (Rust core): Input text is tokenized first.
  5. Tokenizer (Rust core) token ids go to generate() loop: The token ids start the generation loop.
  6. generate() loop calls forward, one token at a time PyTorch model: Each step runs the model and, with greedy search, keeps the most likely next token.
  7. PyTorch model computes attention with PyTorch SDPA: Unless another attention implementation is set, attention layers use PyTorch's SDPA.
  8. PyTorch SDPA picks Fastest CUDA kernel: On a CUDA GPU, SDPA selects the fastest available attention kernel.
  9. Fastest CUDA kernel runs on NVIDIA GPU: The kernels run on the NVIDIA GPU.
  10. generate() loop hands output ids to Post-processing: The finished token ids go to post-processing.

Which device it picks

If you do not pass a device, the pipeline places the model on the first accelerator it finds (a CUDA GPU, Apple silicon's MPS, or others) and uses the CPU only when there is none. Passing device="cpu" or device="mps" forces a choice.

Where Python hands off

Most of the work in this path is not Python: tokenization runs in Rust, and the model's layers run as PyTorch kernels. For what PyTorch does with those operations on each device, see what happens when you run a PyTorch model. For serving many requests at once, engines such as vLLM and SGLang add batching schedulers and KV-cache management on top of the same kind of model.

Every stage

Your pipeline() call

programming language

Python code such as pipeline(task="text-generation", model=...), which picks a default model and preprocessor for the task unless you name one.

In the atlas: Python

  • Each task is configured to use a default pretrained model and preprocessor, but this can be overridden with the model parameter if you want to use a different model.
    Pipeline (Transformers documentation) Project docs

Pipeline

framework component

Transformers' Pipeline class: tokenization, model inference and post-processing in one call, on the device it chooses or you set.

In the atlas: Transformers

safetensors weights

model format

The model's configuration and weights from the Hugging Face Hub or a local folder, read from safetensors files when they exist.

In the atlas: Safetensors

Tokenizer (Rust core)

framework component

Turns text into token ids. Fast tokenizers run on the Rust tokenizers library, which parallelizes batches across threads.

In the atlas: HF Tokenizers

PyTorch model

framework component

The model class (for text generation, a causal language model) holding the loaded weights on the chosen device.

In the atlas: PyTorch

generate() loop

runtime

Produces new tokens one step at a time by calling the model; by default it is greedy search, up to 20 new tokens unless the model's generation config says otherwise.

Post-processing

framework component

Turns the output token ids back into text for the result.

MPS kernels

kernel

On the mps device, PyTorch runs operations as Metal Performance Shaders graphs and kernels.

  • It introduces a new device to map Machine Learning computational graphs and primitives on highly efficient Metal Performance Shaders Graph framework and tuned kernels provided by Metal Performance Shaders framework respectively.
    MPS backend (PyTorch 2.15 documentation) Project docs

Fastest CUDA kernel

kernel

On CUDA, SDPA automatically picks the fastest available attention kernel.

In the atlas: CUDA

CPU

hardware

Used only when no accelerator is available, or when you pass device="cpu".

  • By default, when device is left unset, Pipeline automatically places the model on the first available accelerator (CUDA GPU, Apple Silicon MPS, XPU, …) and only falls back to a CPU when none is available.
    Pipeline (Transformers documentation) Project docs

Apple silicon GPU

hardware

The GPU of an Apple silicon Mac, through PyTorch's mps device.

NVIDIA GPU

hardware

The first choice when device is left unset and a CUDA GPU is present.

  • By default, when device is left unset, Pipeline automatically places the model on the first available accelerator (CUDA GPU, Apple Silicon MPS, XPU, …) and only falls back to a CPU when none is available.
    Pipeline (Transformers documentation) Project docs

Every step

FromStepToWhenWhat happensSource
Your pipeline() callcreates and callsPipelineany deviceYour code builds a pipeline for a task and calls it with text.Project docs
Pipelineloadssafetensors weightsany deviceThe model's weights are loaded with from_pretrained(), from safetensors files when available.Project docs
safetensors weightsfillPyTorch modelany deviceThe weights are loaded into the model class on the chosen device.Project docs
Pipelinetokenizes withTokenizer (Rust core)any deviceInput text is tokenized first.Project docs
Tokenizer (Rust core)token ids go togenerate() loopany deviceThe token ids start the generation loop.Project docs
generate() loopcalls forward, one token at a timePyTorch modelany deviceEach step runs the model and, with greedy search, keeps the most likely next token.Project docs
PyTorch modelcomputes attention withPyTorch SDPAany deviceUnless another attention implementation is set, attention layers use PyTorch's SDPA.Project docs
PyTorch SDPApicksFastest CUDA kernelNVIDIA GPUOn a CUDA GPU, SDPA selects the fastest available attention kernel.Project docs
PyTorch SDPAfalls back toC++ attentionCPUOn the CPU, SDPA uses PyTorch's C++ implementation.Project docs
PyTorch SDPAruns asMPS kernelsApple GPU (MPS)On Apple silicon, the model's operations run as MPS kernels.Project docs
Project docs
Fastest CUDA kernelruns onNVIDIA GPUNVIDIA GPUThe kernels run on the NVIDIA GPU.Project docs
C++ attentionruns onCPUCPUThe C++ code runs on the CPU.Project docs
MPS kernelsrun onApple silicon GPUApple GPU (MPS)MPS kernels run on the Apple silicon GPU.Project docs
generate() loophands output ids toPost-processingany deviceThe finished token ids go to post-processing.Project docs

What this does not show

Documented path for one text-generation pipeline() call with the PyTorch backend, default greedy decoding and the default SDPA attention, on a CPU, an NVIDIA GPU or an Apple silicon GPU. Other tasks, other attention implementations (FlashAttention, paged attention), quantization, multi-GPU placement with device_map, Accelerate and other frameworks are out of scope. What happens inside PyTorch on each device is on the PyTorch page.

How this page is checked

Every stage and step above quotes the source that supports it. The quotes were checked against the Transformers documentation (Pipeline tutorial, Pipelines reference, Loading models, Tokenizers, Attention backends, Generation strategies) and the PyTorch MPS backend notes on 2026-11-12. The data is a versioned JSON file under the same CC BY 4.0 license as the dataset. Found something wrong or out of date? Open an issue with the better source.

See this path in the graph explorer →

What these tools are written in is in the main atlas: Python, Transformers, Safetensors, HF Tokenizers, PyTorch, C++, CUDA. More execution paths: How Does a Rust LLM Generate Tokens with Candle, What Actually Happens When You Run JAX Code, What Runs Underneath a Keras 3 Model, How Does llama.cpp Run a Model on Your Hardware, How Does MLC LLM Turn a Compiled Model into Generated Tokens, What Actually Happens When You Run MLX on a Mac, What Actually Runs When You Send Ollama a Prompt, How Does ONNX Runtime Execute a Model Across Hardware, What Actually Happens When You Run a PyTorch Model, What Actually Happens When SGLang Serves a Request, How Does TensorFlow Run Python on a CPU or GPU, What Actually Happens When TensorRT-LLM Runs a Model, What Actually Happens When vLLM Serves a Request, or compare them all.

Explore the full graph →