What Actually Happens When You Run MLX on a Mac?

MLX operations do not compute anything when you call them: they record a lazy compute graph. Work happens when the graph is evaluated, either by calling eval() or implicitly, for example by printing an array. Evaluation runs each operation's primitive on the device you chose: Metal kernels on the Apple silicon GPU, or C++ kernels on the CPU. Arrays live in unified memory, so both devices use the same data without copies. With mx.compile, MLX traces the function once, merges and fuses operations, and generates code: a Metal kernel for the GPU, or a C++ shared library for the CPU.

Showing plain lazy evaluation on the Apple silicon GPU: 6 stages, 5 steps.

Your program
Runtime
Framework core
Kernel
Compiler
Intermediate representation
Hardware

The selected path, step by step

Your MLX code calls MLX: Your code calls MLX operations such as mx.matmul, or runs an mlx.nn model.
MLX records Lazy compute graph: Each operation adds a node to the graph and returns an array that has no data yet.
Lazy compute graph runs when you call Evaluation (eval): The graph runs at mx.eval, or implicitly when you print an array, convert it to NumPy, or call .item().
Evaluation (eval) calls eval_gpu() of Metal kernels: Operations on a GPU stream run their primitive's Metal kernel. In compiled code, operations that were not fused still run here.
Metal kernels run on Apple silicon GPU: Metal kernels run on the GPU, reading the same memory the CPU uses.

Lazy by default

Calling mx.add or a layer of an mlx.nn model only adds a node to a graph. The graph runs when something needs the values: an explicit mx.eval, printing an array, converting it to NumPy, or calling .item(). A training loop typically evaluates once per step, so each step's forward pass, gradients and optimizer update run as one graph.

That is the main difference from PyTorch in eager mode and from JAX without jit, where each operation is dispatched as Python reaches it. mx.compile plays the role that torch.compile and jax.jit play there.

One memory, two devices

On Apple silicon the CPU and GPU share one memory pool. MLX arrays do not belong to a device; you pick the device per operation (the stream argument), and MLX inserts the dependencies when an operation on one device needs the result of another. That is why the device choice above changes only the kernels, not where the data lives.

For running LLMs locally on a Mac with hand-written Metal kernels instead, compare how llama.cpp runs a model.

Every stage

Your MLX code

programming language

Python code using mlx.core arrays, and often mlx.nn models and mlx.optimizers.

In the atlas: Python

MLX has a Python API that closely follows NumPy.
MLX README (ml-explore/mlx) Source repository

MLX

framework component

Apple's array framework. Its operations are thin front ends that add nodes to a graph.

In the atlas: MLX

MLX is an array framework for machine learning on Apple silicon, brought to you by Apple machine learning research.
MLX README (ml-explore/mlx) Source repository
Operations in MLX build the computation graph. Primitives provide the rules for evaluating and transforming the graph.
Custom Extensions in MLX (MLX documentation) Project docs

Lazy compute graph

intermediate representation

The recorded operations, each a primitive with its inputs. Nothing has been computed yet.

When you perform operations in MLX, no computation actually happens. Instead a compute graph is recorded.
Lazy Evaluation (MLX documentation) Project docs

mx.compile

compiler

MLX's compiler transformation. On the first call it traces the function with placeholder inputs, shrinks the graph and fuses operations; later calls with the same shapes and types reuse the result.

Function compilation results in smaller graphs by merging common work and fusing certain operations.
Compilation (MLX documentation) Project docs
The first time you call a compiled function, MLX will build the compute graph, optimize it, and generate and compile code.
Compilation (MLX documentation) Project docs

Evaluation (eval)

runtime

When values are needed, MLX schedules the graph and runs each primitive on the device of its stream.

In the atlas: MLX

The actual computation only happens if an eval() is performed.
Lazy Evaluation (MLX documentation) Project docs
When we evaluate the output array, MLX schedules the execution of the computation graph, and calls Axpby::eval_cpu() or Axpby::eval_gpu() depending on the stream/device specified by the user.
Custom Extensions in MLX (MLX documentation) Project docs

C++ CPU kernels

kernel

Each primitive's CPU implementation (eval_cpu), written in C++.

In the atlas: C++

A Primitive has methods to run on the CPU or GPU and for function transformations such as vjp and jvp.
Custom Extensions in MLX (MLX documentation) Project docs

Fused C++ library

machine code

For the CPU, mx.compile generates C++ source for the fused operations, builds it into a shared library and loads it.

In the atlas: C++

Could not load C++ shared library
mlx/backend/cpu/compiled.cpp (ml-explore/mlx) Source repository
The first time you call a compiled function, MLX will build the compute graph, optimize it, and generate and compile code.
Compilation (MLX documentation) Project docs

Metal kernels

kernel

Each primitive's GPU implementation (eval_gpu), a kernel written in the Metal shading language and shipped prebuilt in MLX's Metal library.

Apple silicon devices address their GPUs using the Metal shading language, and GPU kernels in MLX are written using Metal.
Custom Extensions in MLX (MLX documentation) Project docs
the MLX Metal library which contains pre-built GPU kernels
Build and Install (MLX documentation) Project docs

Fused Metal kernel

kernel

For the GPU, mx.compile writes Metal source for the fused operations and compiles it into one kernel.

Too many inputs/outputs fused in the Metal Compiled
mlx/backend/metal/compiled.cpp (ml-explore/mlx) Source repository
all of the operations in the gelu are fusible into a single kernel with compile()
Compilation (MLX documentation) Project docs

Apple silicon CPU

hardware

The CPU cores, sharing one memory pool with the GPU.

Apple silicon has a unified memory architecture. The CPU and GPU have direct access to the same memory pool.
Unified Memory (MLX documentation) Project docs

Apple silicon GPU

hardware

The integrated GPU, sharing one memory pool with the CPU, so arrays need no copies between devices.

Apple silicon has a unified memory architecture. The CPU and GPU have direct access to the same memory pool.
Unified Memory (MLX documentation) Project docs

Every step

From	Step	To	When	What happens	Source
Your MLX code	calls	MLX	any mode · any device	Your code calls MLX operations such as mx.matmul, or runs an mlx.nn model.	Source repository
MLX	records	Lazy compute graph	any mode · any device	Each operation adds a node to the graph and returns an array that has no data yet.	Project docs
Lazy compute graph	runs when you call	Evaluation (eval)	Lazy (no compile) · any device	The graph runs at mx.eval, or implicitly when you print an array, convert it to NumPy, or call .item().	Project docs Project docs
Lazy compute graph	traced and optimized by	mx.compile	mx.compile · any device	On the first call, the compiled function is traced with placeholder inputs and its graph is simplified and fused. Calls with new shapes, types or numbers of inputs compile again.	Project docs Project docs
mx.compile	smaller graph runs at	Evaluation (eval)	mx.compile · any device	The compiled graph is still lazy: it runs when it is evaluated, and later calls reuse the cached compilation.	Project docs
Evaluation (eval)	calls eval_cpu() of	C++ CPU kernels	any mode · CPU	Operations on a CPU stream run their primitive's C++ CPU implementation.	Project docs
Evaluation (eval)	calls eval_gpu() of	Metal kernels	any mode · GPU (Metal)	Operations on a GPU stream run their primitive's Metal kernel. In compiled code, operations that were not fused still run here.	Project docs Project docs
mx.compile	generates and builds	Fused C++ library	mx.compile · CPU	For the CPU, the fused operations become C++ source that is compiled into a shared library and loaded.	Project docs Source repository
mx.compile	generates	Fused Metal kernel	mx.compile · GPU (Metal)	For the GPU, the fused element-wise operations become one Metal kernel.	Project docs Source repository
C++ CPU kernels	run on	Apple silicon CPU	any mode · CPU	The CPU kernels read and write arrays in unified memory.	Project docs
Fused C++ library	runs on	Apple silicon CPU	mx.compile · CPU	The compiled library's functions run on the CPU when the graph is evaluated.	Source repository
Metal kernels	run on	Apple silicon GPU	any mode · GPU (Metal)	Metal kernels run on the GPU, reading the same memory the CPU uses.	Project docs
Fused Metal kernel	runs on	Apple silicon GPU	mx.compile · GPU (Metal)	The fused kernel runs on the GPU when the graph is evaluated.	Project docs

What this does not show

Documented paths for MLX Python code on one Apple silicon Mac, run lazily or with mx.compile, on the GPU or the CPU. MLX's CUDA backend for Linux, distributed training, custom Metal kernels written with mx.fast.metal_kernel, and the C++, C and Swift APIs are out of scope.

How this page is checked

Every stage and step above quotes the source that supports it. The quotes were checked against the MLX documentation (Lazy Evaluation, Unified Memory, Compilation, Custom Extensions, Build and Install), the MLX README, and the Metal and CPU compile sources in ml-explore/mlx on 2026-11-11. The data is a versioned JSON file under the same CC BY 4.0 license as the dataset. Found something wrong or out of date? Open an issue with the better source.

What these tools are written in is in the main atlas: Python, MLX, C++. More execution paths: What Actually Happens When You Run JAX Code, How Does llama.cpp Run a Model on Your Hardware, How Does ONNX Runtime Execute a Model Across Hardware, What Actually Happens When You Run a PyTorch Model, What Actually Happens When vLLM Serves a Request, or compare them all.

Explore the full graph →