How Does TensorFlow Run Python on a CPU or GPU?

TensorFlow starts with a Python API, but the selected execution mode changes what runs next. Eager mode dispatches operations individually. tf.function traces TensorFlow operations into a reusable graph. jit_compile=True additionally asks XLA to compile the function. Device placement then determines CPU or GPU execution; using a graph does not itself mean XLA or guarantee GPU-only work.
Execution mode
Device

Showing eager execution on the CPU: 5 stages, 4 steps.

Your Python code calls TensorFlow TensorFlow API TensorFlow API dispatches each op Eager execution TensorFlow API traces TensorFlow ops Traced tf.Graph Traced tf.Graph executes graph Graph execution Traced tf.Graph compiled with XLA XLA compilation Eager execution selects CPU kernels CPU op kernels Eager execution selects GPU kernels GPU op kernels Graph execution selects CPU kernels CPU op kernels Graph execution selects GPU kernels GPU op kernels XLA compilation targets CPU XLA CPU code CPU op kernels executes on CPU XLA CPU code executes on CPU XLA compilation targets GPU XLA GPU executable GPU op kernels executes on NVIDIA GPU XLA GPU executable executes on NVIDIA GPU Your Python code programming language TensorFlow API framework component Eager execution runtime Traced tf.Graph intermediate representation Graph execution runtime XLA compilation compiler CPU op kernels kernel GPU op kernels kernel XLA CPU code machine code XLA GPU executable machine code CPU hardware NVIDIA GPU hardware

The selected path, step by step

  1. Your Python code calls TensorFlow TensorFlow API: The program enters TensorFlow through its Python API.
  2. TensorFlow API dispatches each op Eager execution: Eager execution evaluates operations as Python calls them.
  3. Eager execution selects CPU kernels CPU op kernels: Device placement chooses the implementations for the supported operations shown here.
  4. CPU op kernels executes on CPU: Numerical work reaches the selected physical processor.

Does TensorFlow execute Python on the GPU?

Python supplies the program and launches TensorFlow operations. Numerical kernels execute on the selected device; a Python loop, a print statement and a matrix multiplication are not interchangeable units of GPU work. The device-placement guide explains how to log where operations actually run. A visible GPU is a prerequisite, not proof that the complete program uses it.

The CPU selection here shows CPU numerical work. The GPU selection shows supported NVIDIA device operations while leaving host orchestration implicit. Operations without an applicable GPU implementation may run on the CPU. Inspect placement and data movement before interpreting a low GPU utilization reading as a compiler defect.

What is the difference between eager mode and tf.function?

Eager mode is useful for inspecting values because Python dispatches each operation immediately. Wrapping a compatible function with tf.function produces a callable backed by a traced graph. Repeated compatible calls can reuse that graph instead of rebuilding it operation by operation. The graph introduction describes the difference and why graphs can be exported without the original Python program.

Tracing is not the same as running every Python statement again on every invocation. Python-side effects and changing argument signatures can behave differently from ordinary eager code. Retracing can add startup work; it is separate from the steady-state cost of executing an already constructed graph.

Is tf.function the same as XLA?

No. The graph-only choice in this page uses tf.function with jit_compile=False. The explicit-XLA choice adds compiler optimization through jit_compile=True. The API contract states that an explicitly compiled function must be supported by XLA or an error is raised. Do not assume an unsupported function will silently fall back to eager execution.

XLA can optimize and combine work for supported targets. It is also used by JAX, but sharing a compiler does not make the frameworks identical: their front ends, tracing rules and user APIs differ. See the OpenXLA overview for the supported compiler role.

Where does Keras fit?

A Keras model using the TensorFlow backend enters this stack through TensorFlow operations. Keras 3 can also select another backend, so a Keras import alone does not identify this path. This page explains TensorFlow execution rather than prescribing one framework for every workload.

How should I interpret speed differences?

Compare equivalent computations after separating initialization, tracing or compilation from repeated execution. A graph or XLA can reduce some overhead, but neither is a universal speed guarantee. Tensor sizes, supported operations, device placement and transfers all affect the resulting path. This diagram records mechanisms rather than benchmark rankings.

Every stage

Your Python code

programming language

Python invokes TensorFlow operations or calls a function previously wrapped with tf.function.

In the atlas: Python

Eager execution

runtime

Operations are dispatched one by one from Python. This path does not first trace the whole function into a graph.

In the atlas: TensorFlow

Traced tf.Graph

intermediate representation

tf.function traces TensorFlow operations into a reusable graph; Python side effects do not become ordinary graph operations.

XLA compilation

compiler

With jit_compile=True, the selected function must be supported by XLA. Compilation failure is an error, not a promised eager fallback.

In the atlas: XLA

  • XLA (Accelerated Linear Algebra) is an open-source compiler for machine learning.
    XLA overview (OpenXLA) Project docs

CPU op kernels

kernel

TensorFlow operations placed on the CPU use CPU implementations of their numerical kernels.

GPU op kernels

kernel

Supported operations placed on a visible NVIDIA GPU use GPU implementations; CPU-only operations can still remain on the host.

XLA CPU code

machine code

The successful XLA CPU selection executes code produced for its CPU target.

  • XLA (Accelerated Linear Algebra) is an open-source compiler for machine learning.
    XLA overview (OpenXLA) Project docs

XLA GPU executable

machine code

The successful XLA GPU selection executes a GPU-targeted compiled computation; this is not the ordinary per-op eager path.

  • XLA (Accelerated Linear Algebra) is an open-source compiler for machine learning.
    XLA overview (OpenXLA) Project docs

CPU

hardware

The CPU executes the selected CPU computation. Host-side control also exists for GPU workloads.

NVIDIA GPU

hardware

A configured NVIDIA GPU executes supported GPU computations. The diagram does not assert that every host operation moves to it.

Every step

FromStepToWhenWhat happensSource
Your Python codecalls TensorFlowTensorFlow APIany execution mode · any deviceThe program enters TensorFlow through its Python API.Project docs
TensorFlow APIdispatches each opEager executionEager · any deviceEager execution evaluates operations as Python calls them.Project docs
TensorFlow APItraces TensorFlow opsTraced tf.Graphtf.function or Explicit XLA · any deviceBoth graph selections first obtain a traced TensorFlow graph.Project docs
Traced tf.Graphexecutes graphGraph executiontf.function · any deviceGraph-only mode executes the graph with explicit XLA compilation disabled.Project docs
Traced tf.Graphcompiled with XLAXLA compilationExplicit XLA · any deviceExplicit jit_compile=True compiles the function rather than merely tracing it.Project docs
Eager executionselects CPU kernelsCPU op kernelsany execution mode · CPUDevice placement chooses the implementations for the supported operations shown here.Project docs
Eager executionselects GPU kernelsGPU op kernelsany execution mode · NVIDIA GPUDevice placement chooses the implementations for the supported operations shown here.Project docs
Graph executionselects CPU kernelsCPU op kernelsany execution mode · CPUDevice placement chooses the implementations for the supported operations shown here.Project docs
Graph executionselects GPU kernelsGPU op kernelsany execution mode · NVIDIA GPUDevice placement chooses the implementations for the supported operations shown here.Project docs
XLA compilationtargets CPUXLA CPU codeany execution mode · CPUXLA targets the selected supported hardware after successful compilation.Project docs
CPU op kernelsexecutes onCPUany execution mode · any deviceNumerical work reaches the selected physical processor.Project docs
XLA CPU codeexecutes onCPUany execution mode · any deviceThe compiled computation executes on its target hardware.Project docs
XLA compilationtargets GPUXLA GPU executableany execution mode · NVIDIA GPUXLA targets the selected supported hardware after successful compilation.Project docs
GPU op kernelsexecutes onNVIDIA GPUany execution mode · any deviceNumerical work reaches the selected physical processor.Project docs
XLA GPU executableexecutes onNVIDIA GPUany execution mode · any deviceThe compiled computation executes on its target hardware.Project docs

What this does not show

TensorFlow 2 numerical computation on one CPU or a configured NVIDIA GPU. The graph-only option explicitly disables XLA; the XLA option assumes successful compilation. TPU, distributed training, TensorFlow Lite, browser TensorFlow.js, custom ops and detailed optimizer/gradient execution are outside this diagram. GPU branches show supported device work; host control and CPU-only ops may coexist.

How this page is checked

Every stage and step above quotes the source that supports it. The quotes were checked against the TensorFlow 2 API, TensorFlow graph and device-placement guides, and the OpenXLA compiler overview on 2026-11-11. The data is a versioned JSON file under the same CC BY 4.0 license as the dataset. Found something wrong or out of date? Open an issue with the better source.

See this path in the graph explorer →

What these tools are written in is in the main atlas: Python, TensorFlow, XLA. More execution paths: How Does a Rust LLM Generate Tokens with Candle, What Actually Happens When You Run JAX Code, What Runs Underneath a Keras 3 Model, How Does llama.cpp Run a Model on Your Hardware, How Does MLC LLM Turn a Compiled Model into Generated Tokens, What Actually Happens When You Run MLX on a Mac, What Actually Runs When You Send Ollama a Prompt, How Does ONNX Runtime Execute a Model Across Hardware, What Actually Happens When You Run a PyTorch Model, What Actually Happens When SGLang Serves a Request, What Actually Happens When TensorRT-LLM Runs a Model, What Actually Happens When You Run a Hugging Face Pipeline, What Actually Happens When vLLM Serves a Request, or compare them all.

Explore the full graph →