Dstack
dstack is an open-source tool designed for the efficient development and deployment of LLM (Large Language Models) across multiple cloud pr...
Last verified:
What is Dstack?
dstack is a unified control plane for GPU provisioning and orchestration that works with any GPU cloud, Kubernetes, or on-prem clusters. It streamlines development, training, and inference for AI/ML workloads, and is compatible with any hardware (NVIDIA, AMD, TPU, Tenstorrent), open-source tools, and frameworks. dstack automatically manages infrastructure provisioning, job scheduling, auto-scaling, port-forwarding, and ingress without requiring Kubernetes or Slurm expertise.
Key features include: Fleets for managing cloud and on-prem clusters with auto-scaling; Dev environments for interactive development with desktop IDEs like VS Code, Cursor, and Windsurf; Tasks for scheduling jobs including distributed multi-node training; Services for deploying production inference services with auto-scaling and OpenAI-compatible endpoints; Volumes for managing network storage; and Gateways for routing. It supports declarative YAML configurations in your repository and provides a simple CLI, UI, and REST API.
dstack is designed for ML engineers, AI researchers, and data science teams who need to provision GPU compute, run training jobs, deploy inference services, or develop remotely with GPU access. It's particularly valuable for teams working across multiple cloud providers, those with on-prem GPUs, or anyone wanting to avoid vendor lock-in while reducing GPU costs through smart orchestration and spot instance support.
The tool is open-source and serves as an alternative to Kubernetes and Slurm specifically built for AI workloads. It integrates with top serving frameworks like SGLang, vLLM, and TensorRT-LLM, and supports advanced inference features like Disaggregated Prefill/Decode and cache-aware routing for production-grade optimization.
Dstack pricing
Pricing model: Free
dstack is open-source and free to self-host. You can install the server on your laptop or any environment using uv, pip, or Docker with no cost. The open-source version includes full CLI, UI, and API access. dstack Sky is a managed hosted service (preview) that enables accessing GPUs from multiple providers without individual cloud accounts - users pay for GPU credits/instances through the service at competitive rates. dstack Enterprise is a self-hosted paid tier with SSO, air-gapped setup, and dedicated support. AWS Marketplace offers dstack Standard at $3,000/month fixed subscription for unlimited usage (additional AWS infrastructure costs apply). GPU costs vary by provider and instance type when using backends (e.g., RTX 4090 from $0.35/hour on Vast.ai, H100 at ~$2.95/hour on Nebius).
Dstack pros
- Open-source alternative to Kubernetes and Slurm built specifically for AI
- Works with any GPU cloud, Kubernetes clusters, or on-prem servers via SSH
- Supports NVIDIA, AMD, TPU, and Tenstorrent accelerators out of the box
- Simple YAML configuration files stored in your repository
- No Kubernetes or Slurm expertise required to get started
- Auto-scaling with fine-grained fleet controls including spot/on-demand/reserved
- Remote dev environments accessible via desktop IDEs (VS Code, Cursor, Windsurf)
- Native support for distributed multi-GPU and multi-node training jobs
- Production inference services with OpenAI-compatible endpoints
- Supports Disaggregated Prefill/Decode and cache-aware routing for inference
- Automatic port-forwarding and ingress management
- GPU utilization, memory usage, and health metrics reporting during job execution
- Integrates with SGLang, vLLM, TensorRT-LLM and other open-source frameworks
- CLI, UI, and REST API for programmatic control
- Reduces GPU costs through smart orchestration and right-sizing
- Works with any Docker image and existing code without modification
- SSH fleets provide lightweight alternative to Kubernetes for on-prem
- AI agent skills available for Claude, Codex, and Cursor integration
Dstack cons
- Relatively new project compared to battle-tested Slurm
- Requires Git and OpenSSH installed for CLI usage
- dstack Sky managed service has limited provider transparency
- Windows users need Git for Windows with OpenSSH configured
- Less mature ecosystem compared to established Kubernetes tooling
- Documentation still evolving with some gaps
- GPU marketplace access via dstack Sky adds intermediary layer
- Self-hosting requires server setup and maintenance
- Limited enterprise features in open-source version (SSO, air-gap in Enterprise only)
- Learning curve for YAML configuration format
Frequently asked questions about Dstack
How does dstack differ from Slurm?
Slurm is a battle-tested system with decades of production use in HPC environments. dstack is built for modern ML/AI workloads with cloud-native provisioning and a container-first architecture. While both support distributed training and batch jobs, dstack also natively supports development environments and production-grade inference. dstack provides cloud API integration for automatic GPU provisioning, which Slurm lacks.
How does dstack compare to Kubernetes?
Kubernetes is a general-purpose container orchestrator. dstack also orchestrates containers but provides a lightweight, streamlined interface purpose-built for ML. You declare dev environments, tasks, services, and fleets with simple YAML configuration. dstack provisions GPUs, manages clusters via fleets with fine-grained controls, and optimizes cost and utilization while keeping a simple UI and CLI. If you already use Kubernetes, you can run dstack on it via the Kubernetes backend.
Can I use dstack with Kubernetes?
Yes. You can connect existing Kubernetes clusters using the Kubernetes backend and run dev environments, tasks, and services on them. Choose the Kubernetes backend if your GPUs already run on Kubernetes and your team depends on its ecosystem and tooling. If your priority is orchestrating cloud GPUs and Kubernetes isn't required, VM-based backends offer better native cloud integration. For on-prem GPUs where Kubernetes is optional, SSH fleets provide a simpler alternative.
When should I use dstack?
dstack accelerates ML development with a simple, ML-native interface. Use it to spin up dev environments, run single-node or distributed tasks, and deploy services without infrastructure overhead. It radically reduces GPU costs via smart orchestration and fine-grained fleet controls including efficient reuse, right-sizing, and support for spot, on-demand, and reserved capacity. It's 100% interoperable with your stack and works with any open-source frameworks, tools, Docker images, and code across GPU clouds, Kubernetes, and on-prem GPUs.
What accelerators does dstack support?
dstack supports NVIDIA, AMD, TPU, and Tenstorrent accelerators out of the box. This includes various GPU models like H100, RTX 4090, L4, and others from different vendors without requiring additional configuration.
What configuration types does dstack support?
dstack supports five main configuration types: Fleets for managing cloud and on-prem clusters; Dev environments for interactive development using desktop IDEs; Tasks for scheduling jobs including distributed ones or running web apps; Services for deploying models or web apps as endpoints; and Volumes for managing network volumes to persist data. All configurations are defined as YAML files in your repository.
How do I access dev environments?
When you create a dev environment, dstack apply prints both an IDE URL and SSH command. You can access it using your desktop IDE such as VS Code, Cursor, or Windsurf by clicking the IDE URL (e.g., vscode://vscode-remote/ssh-remote+vscode/workflow) or by running ssh <run name> in your terminal.
What is dstack Sky?
dstack Sky is a managed hosted service built on top of dstack that enables users to access GPUs at competitive rates from a wide pool of providers without needing an account in each cloud provider. It streamlines the process of navigating multiple GPU providers to find the most competitive offerings. Services run with dstack Sky are available at https://<run name>.<project name>.sky.dstack.ai.
What inference frameworks does dstack integrate with?
dstack integrates with top open-source serving frameworks including SGLang, vLLM, and TensorRT-LLM. It also works with Shepherd Model Gateway (SMG) on both NVIDIA and AMD, and with NVIDIA Dynamo on NVIDIA. dstack enables Disaggregated Prefill/Decode and cache-aware routing for production-grade optimized inference.
How do I install dstack?
Install the server using uv tool install dstack[all] -U, pip install dstack[all] -U, or Docker with the dstackai/dstack image. Install the CLI using uv tool install dstack -U or pip install dstack -U. The CLI works on Linux, macOS, and Windows (via WSL 2) and requires Git and OpenSSH. After starting the server, configure the CLI with dstack project add using the server URL, token, and project name.