Tt Metal
Tt Metal docs: TT-NN operator library and the TT-Metalium low-level kernel programming model for Tenstorrent hardware.
Last verified:
What is Tt Metal?
Tt Metal (TT-Metalium) is Tenstorrent's open source low-level programming model, and this documentation covers TT-NN — the neural network operations library built on the Tt Metal programming model. TT-NN serves as a user-friendly API for running machine learning workloads on Tenstorrent hardware, supporting both C++ and Python projects. The library is designed to feel familiar to developers experienced with PyTorch, lowering the learning curve for those transitioning to Tenstorrent's AI accelerators.
Key features include more than 200 operations covering matrix multiplication, convolution, reduction, CCL, and fused Transformer operations. TT-NN provides a specialized Tensor type that enables representing, distributing, and accessing data on device memory. It includes native support for multi-device mesh operations, tools to trace and visualize computation, utilities to cache converted parameters for faster model loading on repeated runs, and a reference mode for debugging long operation sequences.
The Tt Metal stack is intended for AI/ML developers, researchers, and engineers working with Tenstorrent hardware such as Grayskull, Wormhole, and Blackhole architectures. It is particularly useful for teams building and deploying neural networks, converting models from frameworks like PyTorch, JAX, TensorFlow, and ONNX, and those who need custom operation registration capabilities. The library is open source under Apache 2.0 License.
Tt Metal pricing
Pricing model: Freemium
TT-NN is completely free and open source under the Apache 2.0 License. There are no paid plans or subscription tiers. The library, including TT-Forge, TT-NN, and TT-Metalium, is freely available to download from GitHub. Users can install via Python wheel, Docker release image, source build, or Anaconda at no cost. Pre-built model demos and tutorials are also freely accessible.
Tt Metal pros
- Open source library under Apache 2.0 License
- PyTorch-like API familiar to ML developers
- Supports both C++ and Python projects
- More than 200 neural network operations included
- Native multi-device mesh support for scaling
- Works with Grayskull, Wormhole, and Blackhole architectures
- Custom operation registration capability
- Parameter caching speeds up repeated model loading
- Reference mode for debugging operation sequences
- TT-NN Visualizer for execution analysis
- Python tutorials for tensor operations and inference
- Docker release images for quick setup
- Pre-built model demos available
- Supports PyTorch, JAX, TensorFlow, and ONNX
- N-dimensional tensor support
Tt Metal cons
- Requires Tenstorrent hardware (Grayskull, Wormhole, or Blackhole)
- Linux only with glibc 2.34 or newer
- Complex installation requiring driver and firmware setup
- IOMMU must be enabled at host level for VMs
- Limited to specific Tenstorrent device compatibility
- Source build requires CMake and build dependencies
- CPU performance governor must be set for models
- Steep learning curve for tt-metal programming model
Frequently asked questions about Tt Metal
What is TT-NN?
TT-NN is an open source library of neural network operations built on the tt-metal programming model. It can be used in C++ and Python projects and is designed to feel familiar to developers experienced with PyTorch. It provides a user-friendly API for running ML workloads on Tenstorrent hardware.
What hardware does TT-NN support?
TT-NN supports Tenstorrent devices including Grayskull, Wormhole (wormhole_b0), and Blackhole architectures. Specific hardware requirements vary by device: Galaxy systems and Blackhole require Ubuntu 22.04, Python 3.10, driver v2.5.0 or above, and firmware fw_pack-19.2.0.fwbundle.
How do I install TT-NN?
There are four installation options: (1) Install from wheel using pip install ttnn for quick setup, (2) Use Docker release image with docker pull ghcr.io/tenstorrent/tt-metal/tt-metalium-ubuntu-22.04-release-amd64:latest-rc, (3) Build from source by cloning the repository and running build_metal.sh, or (4) Install via Anaconda with conda create -n metalium python=3.10 tt-metalium -c conda-forge.
What operations does TT-NN provide?
TT-NN provides more than 200 operations including matrix multiplication, convolution, reduction, CCL (collective communication), and fused Transformer operations. It also includes comparison operations like ge (greater than or equal), tensor operations, and supports broadcasting for various data types including Float32, BFLOAT16, BFLOAT8_B, and INT32.
Is TT-NN open source?
Yes, TT-NN and TT-Metalium are licensed under the Apache 2.0 License. The source code is available on GitHub at github.com/tenstorrent/tt-metal. The fully open-source stack includes TT-Forge, TT-Lang, TT-NN, and TT-Metal built on LLVM.
Can I use TT-NN with PyTorch?
TT-NN is designed to feel familiar to PyTorch developers with a PyTorch-like API. The Tenstorrent software stack has native PyTorch support, along with JAX, TensorFlow, and ONNX support. Developers can convert models from these frameworks to run on Tenstorrent hardware using TT-NN.
What tutorials are available for TT-NN?
TT-NN provides Python tutorials for tensor operations, model conversion, and inference. Each tutorial has an equivalent standalone Python script in the ttnn/tutorials directory of the TT-Metal repository. Tutorials can be run with Jupyter Lab using jupyter lab --no-browser --port=8888 from within the ttnn/tutorials directory.
How do I debug TT-NN operations?
TT-NN includes a reference mode that enables debugging of long operation sequences against a known reference. Additionally, TT-NN Visualizer is an interactive tool for visualizing and analyzing model execution on Tenstorrent hardware, providing detailed insights through graphs and performance metrics.
Can I register custom operations in TT-NN?
Yes, TT-NN provides the ability for developers to register custom operations. This feature allows users to extend the library with their own neural network operations beyond the 200+ built-in operations, giving flexibility for specialized model requirements.
What are the system requirements for running TT-NN?
TT-NN requires Linux with glibc 2.34 or newer. For VMs, IOMMU must be enabled at the host level (intel_iommu=on or amd_iommu=on) and the VM must have vIOMMU provisioned. For models users, CPU performance governor must be set to performance mode using cpupower frequency-set -g performance. Python 3.10 is recommended for most systems.