Toolchain · 2023
TensorRT-LLM
NVIDIA inference software combining a Python-facing model/executor stack, C++ scheduling and cache-management bindings, and specialized GPU kernels.
What is TensorRT-LLM written in?
Its runtime is implemented in Python, C++, CUDA, and PyTorch (documented).
Frequently Asked Questions
What language is TensorRT-LLM written in?
TensorRT-LLM is primarily implemented in Python and C++ and CUDA and PyTorch.
When was TensorRT-LLM first released?
TensorRT-LLM was first released in 2023.
Evidence Sources
- https://github.com/NVIDIA/TensorRT-LLM/blob/main/tensorrt_llm/_torch/pyexecutor/py_executor.py Source repository
- https://nvidia.github.io/TensorRT-LLM/latest/torch/arch_overview.html Project docs
- https://github.com/NVIDIA/TensorRT-LLM/tree/main/cpp/tensorrt_llm/kernels Source repository
- https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/torch/arch_overview.md Source repository
Discover More
Spot something wrong?
Every fact on this page is meant to be checkable. If one is wrong, say so and it becomes a dataset change, not a patch to this page.