What Actually Happens When You Run a PyTorch Model?

In eager mode, each tensor operation in your Python code is a separate call into ATen, PyTorch's C++ core, which runs a precompiled C++ kernel on the CPU or a CUDA kernel on the GPU. With torch.compile, TorchDynamo captures runs of operations as an FX graph, and TorchInductor generates new kernels for it: C++ with OpenMP for CPUs, Triton for GPUs. On an NVIDIA GPU, Triton lowers each kernel through MLIR and LLVM IR to PTX, which NVIDIA's driver turns into the GPU's own machine code.

Showing torch.compile on an NVIDIA GPU: 12 stages, 12 steps.

Your program
Framework core
Kernel
Compiler
Intermediate representation
Hardware
Fallback for some operations

The selected path, step by step

Your model code runs in CPython interpreter: Your model is ordinary Python, executed by the CPython interpreter.
ATen CUDA kernels launched on NVIDIA GPU: The kernel is queued on the GPU; Python continues without waiting for it to finish.
CPython interpreter intercepted by TorchDynamo: With torch.compile, TorchDynamo hooks CPython's frame evaluation and rewrites the function's bytecode before it runs.
TorchDynamo captures FX graph: Sequences of PyTorch operations become an FX graph. Code that cannot be traced causes a graph break and runs in normal Python between graphs.
FX graph compiled by TorchInductor: The graph goes to the backend compiler; by default that is TorchInductor.
TorchInductor generates Triton kernels: For GPUs, Inductor generates Triton kernels.
TorchInductor some ops stay ATen CUDA kernels: Some operations stay ATen calls under torch.compile, for example certain matrix multiplications that remain cuBLAS calls.
Triton kernels compiled by Triton compiler: Each Triton kernel is compiled by the Triton compiler, which runs a series of MLIR passes.
Triton compiler lowers to LLVM IR: Triton lowers the kernel to LLVM IR and uses LLVM for code generation.
LLVM IR lowers to PTX: For NVIDIA GPUs the final stage of Triton's pipeline is PTX (for AMD GPUs it is AMDGCN instead).
PTX translated to GPU machine code: NVIDIA's translator and driver turn portable PTX into the machine code of the installed GPU.
GPU machine code launched on NVIDIA GPU: The compiled kernel is launched on the GPU, asynchronously, like any other CUDA kernel.

Where Python ends

In eager mode, Python's part ends at each operation's call into ATen: from there on, everything is precompiled C++ and CUDA that shipped inside PyTorch. Python is still in charge of the order of operations, one call at a time, so the interpreter's overhead is paid on every operation.

Under torch.compile, Python's part ends earlier for the code TorchDynamo can capture: the captured operations never run as individual Python calls again. They become an FX graph, then new code generated by TorchInductor, and on a GPU that code passes through three compiler stages (Triton, LLVM, PTX) before NVIDIA's driver produces machine code for the specific GPU. Code Dynamo cannot trace causes a graph break and still runs in normal Python between compiled graphs.

Either way the GPU work is asynchronous: Python queues kernels and moves on, which is why a GPU program can look fast until something forces it to wait for a result.

Every stage

Your model code

programming language

The nn.Module and training or inference loop you write in Python. Every tensor operation in it is a call into PyTorch.

In the atlas: Python

ATen is fundamentally a tensor library, on top of which almost all other Python and C++ interfaces in PyTorch are built.
PyTorch C++ API: ATen (PyTorch documentation) Project docs

CPython interpreter

runtime

Runs your Python bytecode. In eager mode it calls PyTorch one operation at a time; under torch.compile, TorchDynamo hooks into it before each frame runs.

In the atlas: Python

Dynamo hooks into the frame evaluation API in CPython (PEP 523) to dynamically modify Python bytecode right before it is executed.
Dynamo Overview (PyTorch documentation) Project docs

ATen dispatch

framework component

PyTorch's C++ tensor library. Each operation is looked up and sent to the implementation for the tensor's device and type.

In the atlas: PyTorch

Most of these operations have both CPU and GPU implementations, to which the Tensor class will dynamically dispatch based on its type.
PyTorch C++ API: ATen (PyTorch documentation) Project docs

ATen CPU kernels

kernel

Precompiled C++ implementations of each operation for the CPU, shipped inside the PyTorch library.

In the atlas: C++

Most of these operations have both CPU and GPU implementations, to which the Tensor class will dynamically dispatch based on its type.
PyTorch C++ API: ATen (PyTorch documentation) Project docs

ATen CUDA kernels

kernel

Precompiled GPU implementations of each operation. Some, such as matrix multiplications, call NVIDIA's cuBLAS library.

In the atlas: CUDA

Most of these operations have both CPU and GPU implementations, to which the Tensor class will dynamically dispatch based on its type.
PyTorch C++ API: ATen (PyTorch documentation) Project docs
Under torch.compile, FP32 matmuls using bfx9 remain ATen/cuBLAS calls
CUDA semantics (PyTorch 2.15 documentation) Project docs

TorchDynamo

compiler

A Python-level JIT compiler. It rewrites your function's bytecode so that runs of PyTorch operations are captured as a graph instead of executed one by one.

It rewrites Python bytecode to extract sequences of PyTorch operations into an FX Graph which is then compiled with a customizable backend.
Dynamo Overview (PyTorch documentation) Project docs

FX graph

intermediate representation

The captured sequence of PyTorch operations, as a graph a backend compiler can optimize. Code Dynamo cannot trace causes a graph break and runs in normal Python between graphs.

It rewrites Python bytecode to extract sequences of PyTorch operations into an FX Graph which is then compiled with a customizable backend.
Dynamo Overview (PyTorch documentation) Project docs
Dynamo performs a graph break when it encounters code that can't be traced.
Working with Graph Breaks (PyTorch 2.15 documentation) Project docs

TorchInductor

compiler

The default torch.compile backend. It generates new kernels for the graph: Triton for GPUs and C++ for CPUs. For training, AOT Autograd also captures the backward pass ahead of time.

The default backend is called TorchInductor
torch.compiler (PyTorch 2.15 documentation) Project docs
TorchInductor is the default compiler backend for TorchDynamo, which translates PyTorch programs into OpenAI's Triton for GPUs and C++ for CPUs.
PyTorch 2 paper and tutorial (PyTorch blog) Project docs
AOT Autograd captures not only the user-level code, but also backpropagation
torch.compiler (PyTorch 2.15 documentation) Project docs
Pointwise ops can be fused with: other pointwise ops; reduction ops; and matmul/conv templates.
TorchInductor design post by its lead author (PyTorch dev-discuss) Project docs

C++/OpenMP kernels

kernel

On CPU, Inductor writes C++ kernels that use OpenMP to spread work across cores, usually fusing several operations into one kernel.

In the atlas: C++

TorchInductor is the default compiler backend for TorchDynamo, which translates PyTorch programs into OpenAI's Triton for GPUs and C++ for CPUs.
PyTorch 2 paper and tutorial (PyTorch blog) Project docs
OpenMP provides a work sharing parallel execution model, and enables support for CPUs.
TorchInductor design post by its lead author (PyTorch dev-discuss) Project docs

Triton kernels

kernel language

On GPU, Inductor writes kernels in Triton, a Python-embedded language for GPU programming, usually fusing several operations into one kernel.

In the atlas: Triton

For NVIDIA, AMD and Intel GPUs, it leverages OpenAI Triton as the key building block.
torch.compiler (PyTorch 2.15 documentation) Project docs
Triton is a language and compiler for parallel programming.
Triton documentation Project docs

Triton compiler

compiler

Compiles each Triton kernel through a series of MLIR passes, then hands it to LLVM for code generation.

In the atlas: Triton

Triton is a language and compiler for parallel programming.
Triton documentation Project docs
MLIR_ENABLE_DUMP=1 dumps the IR before every MLIR pass Triton runs, for all kernels.
Triton README (triton-lang/triton) Source repository

LLVM IR

intermediate representation

LLVM's intermediate representation. Triton uses LLVM to generate the device code.

In the atlas: LLVM

Triton uses LLVM to generate code for GPUs and CPUs.
Triton README (triton-lang/triton) Source repository

PTX

intermediate representation

NVIDIA's virtual instruction set, the last portable step. It is not yet the GPU's own machine code.

In the atlas: CUDA

TRITON_KERNEL_DUMP enables the dumping of the IR from each compilation stage and the final ptx/amdgcn.
Triton README (triton-lang/triton) Source repository
PTX defines a virtual machine and ISA for general purpose parallel thread execution.
PTX ISA (NVIDIA documentation) Project docs

GPU machine code

machine code

The instructions of one specific GPU architecture, produced from PTX by NVIDIA's translator and driver.

In the atlas: Machine Code

The PTX-to-GPU translator and driver enable NVIDIA GPUs to be used as programmable parallel computers.
PTX ISA (NVIDIA documentation) Project docs

CPU

hardware

Runs the compiled C++ kernels directly, across its cores.

OpenMP provides a work sharing parallel execution model, and enables support for CPUs.
TorchInductor design post by its lead author (PyTorch dev-discuss) Project docs

NVIDIA GPU

hardware

Runs kernels launched from the CPU. Launches are queued, so Python usually returns before the GPU has finished.

By default, GPU operations are asynchronous. When you call a function that uses the GPU, the operations are enqueued to the particular device, but not necessarily executed until later.
CUDA semantics (PyTorch 2.15 documentation) Project docs

Every step

From	Step	To	When	What happens	Source
Your model code	runs in	CPython interpreter	any mode · any device	Your model is ordinary Python, executed by the CPython interpreter.	Project docs
CPython interpreter	calls, one op at a time	ATen dispatch	Eager · any device	In eager mode each tensor operation is a separate call from Python into PyTorch's C++ core.	Project docs
ATen dispatch	dispatches to	ATen CPU kernels	Eager · CPU	For CPU tensors, ATen picks the CPU implementation of the operation.	Project docs
ATen dispatch	dispatches to	ATen CUDA kernels	Eager · NVIDIA GPU	For CUDA tensors, ATen picks the GPU implementation of the operation.	Project docs
ATen CPU kernels	runs on	CPU	any mode · CPU	The precompiled C++ kernel runs on the CPU.	Project docs
ATen CUDA kernels	launched on	NVIDIA GPU	any mode · NVIDIA GPU	The kernel is queued on the GPU; Python continues without waiting for it to finish.	Project docs
CPython interpreter	intercepted by	TorchDynamo	torch.compile · any device	With torch.compile, TorchDynamo hooks CPython's frame evaluation and rewrites the function's bytecode before it runs.	Project docs
TorchDynamo	captures	FX graph	torch.compile · any device	Sequences of PyTorch operations become an FX graph. Code that cannot be traced causes a graph break and runs in normal Python between graphs.	Project docs Project docs
FX graph	compiled by	TorchInductor	torch.compile · any device	The graph goes to the backend compiler; by default that is TorchInductor.	Project docs
TorchInductor	generates	C++/OpenMP kernels	torch.compile · CPU	For CPU, Inductor generates C++ kernels parallelized with OpenMP.	Project docs Project docs
TorchInductor	generates	Triton kernels	torch.compile · NVIDIA GPU	For GPUs, Inductor generates Triton kernels.	Project docs Project docs
TorchInductor	some ops fall back to	ATen CPU kernels	torch.compile · CPU	Operations Inductor does not generate code for are still run by the existing ATen kernels.	Project docs
TorchInductor	some ops stay	ATen CUDA kernels	torch.compile · NVIDIA GPU	Some operations stay ATen calls under torch.compile, for example certain matrix multiplications that remain cuBLAS calls.	Project docs Project docs
C++/OpenMP kernels	runs on	CPU	torch.compile · CPU	The compiled C++ kernels run on the CPU, split across cores by OpenMP.	Project docs
Triton kernels	compiled by	Triton compiler	torch.compile · NVIDIA GPU	Each Triton kernel is compiled by the Triton compiler, which runs a series of MLIR passes.	Source repository
Triton compiler	lowers to	LLVM IR	torch.compile · NVIDIA GPU	Triton lowers the kernel to LLVM IR and uses LLVM for code generation.	Source repository
LLVM IR	lowers to	PTX	torch.compile · NVIDIA GPU	For NVIDIA GPUs the final stage of Triton's pipeline is PTX (for AMD GPUs it is AMDGCN instead).	Source repository
PTX	translated to	GPU machine code	torch.compile · NVIDIA GPU	NVIDIA's translator and driver turn portable PTX into the machine code of the installed GPU.	Project docs
GPU machine code	launched on	NVIDIA GPU	torch.compile · NVIDIA GPU	The compiled kernel is launched on the GPU, asynchronously, like any other CUDA kernel.	Project docs

What this does not show

Illustrative, documented paths for one PyTorch program in eager mode and under torch.compile with the default TorchInductor backend, on a CPU or an NVIDIA GPU. Not every PyTorch program takes these paths: other torch.compile backends, AMD and Intel GPUs, Apple MPS, CUDA graphs and custom operators are out of scope. TorchInductor's GPU path through Triton also serves AMD and Intel GPUs, ending in a different final format (for AMD, AMDGCN instead of PTX). Other torch.compile backends replace TorchInductor entirely.

How this page is checked

Every stage and step above quotes the source that supports it. The quotes were checked against PyTorch 2.15 documentation, Triton main README, NVIDIA PTX ISA documentation on 2026-11-10. The data is a versioned JSON file under the same CC BY 4.0 license as the dataset. Found something wrong or out of date? Open an issue with the better source.

What these tools are written in is in the main atlas: Python, PyTorch, C++, CUDA, Triton, LLVM, Machine Code. More execution paths: How Does llama.cpp Run a Model on Your Hardware.

Explore the full graph →