Dat1
We Built a Serverless GPU Platform with Fast Cold Starts
Last verified:
What is Dat1?
Dat1 is a serverless GPU platform for running custom generative AI models at scale, offering fast, cost-efficient inference, privacy compliance, and zero hardware management. The platform eliminates the complexity and expense of managing hardware for large AI models by automatically handling scaling and distributing models across its infrastructure.
Key features include serverless GPU inference where you deploy custom AI models without managing hardware or scaling logistics, pay-per-second pricing where you only pay for seconds your model is processing requests with no charges for idle time or timeouts, and ultra-low cold start times of 15-20 seconds for 70GB models. The platform is fully GDPR and CCPA-compliant, follows top cybersecurity best practices, and is part of Nvidia's Inception program for AI startups.
Dat1 is designed for machine learning engineers, AI startups, and companies running custom ML models or pipelines that require real-time inference at scale. Users upload model weights and Python code using the dat1-cli command-line interface, receive an endpoint in return, and are only billed for active inference time. The platform provides dedicated 24/7 support, reserved capacity for mission-critical workloads, and option to deploy fully on-premise for security and compliance requirements.
Dat1 pricing
Pricing model: Freemium
Pay-per-second billing: NVIDIA H100 (26 vCPU, 234GB RAM) at $0.0014/second, NVIDIA A100 (12 vCPU, 85GB RAM) at $0.0008/second, NVIDIA T4 (1 vCPU, 3.75GB RAM) at $0.0001/second. Only charged for time model actively processes requests—never for startup time, idle periods, or timeouts. Contact for other GPU models. Pricing calculator available on website.
Dat1 pros
- Pay-per-second billing with no idle or timeout charges
- Ultra-low cold starts: 15-20 seconds for 70GB models
- Zero hardware management required
- Fully GDPR and CCPA-compliant
- Part of Nvidia Inception program
- Automatic auto-scaling based on demand
- Efficient GPU sharing reduces costs
- Dedicated 24/7 support available
- Reserved capacity for mission-critical workloads
- On-premise deployment option available
- Simple dat1-cli for deployment
- Fast endpoint generation after upload
- Supports streaming responses with SSE
- Dozens of GPUs available at any time
- No charges for startup time or model loading
Dat1 cons
- Requires handler.py with FastAPI app structure
- Uses common Docker image for all models
- May need to package custom libraries with pickle
- Local serving requires Docker and CUDA-compatible GPU
- Cold start optimization may require client collaboration
- Limited to custom ML models you deploy yourself
- GPU model selection may require contacting sales
- Learning curve for CLI deployment workflow
Frequently asked questions about Dat1
What is Dat1?
Dat1 is a serverless GPU platform for running custom generative AI models at scale. You upload your model weights and Python code, receive an endpoint, and Dat1 handles all hardware management, auto-scaling, and high availability while you only pay for active inference time.
How does the pricing work?
Dat1 uses pay-per-second billing where you only pay for the seconds your model is actively processing requests. There are no charges for startup time, idle periods, or timeouts. NVIDIA H100 costs $0.0014/second, A100 costs $0.0008/second, and T4 costs $0.0001/second.
What are cold start times?
Dat1 achieves ultra-low cold starts of 15-20 seconds for 70GB models, including container launch, model weight download, GPU memory load, and inference server startup. For 40GB models, expect cold starts under 20 seconds, which is one of the fastest in the industry.
How do I deploy a model?
Install dat1-cli with pip install dat1-cli, login with dat1 login, initialize your project with dat1 init to create dat1.yaml, create handler.py with FastAPI app containing GET / for healthcheck and POST /infer for inference, then deploy with dat1 deploy to get your endpoint.
Is Dat1 compliant with privacy regulations?
Yes, Dat1 is fully GDPR and CCPA-compliant and follows top cybersecurity best practices. The platform runs workloads in fully isolated containers and doesn't store your data, ensuring privacy and security of user data and models.
What GPU options are available?
Dat1 offers NVIDIA H100 (26 vCPU, 234GB RAM), NVIDIA A100 (12 vCPU, 85GB RAM), and NVIDIA T4 (1 vCPU, 3.75GB RAM). Contact Dat1 for other GPU models. Due to low cold start times, dozens of GPUs are available at any given time.
Can I stream responses?
Yes, Dat1 supports streaming responses using Server-Sent Events (SSE). Set response_type: sse in dat1.yaml and modify handler.py to return a generator using EventSourceResponse that yields responses incrementally to the client.
What is the dat1-cli?
The dat1-cli is the official command-line interface for interacting with the Dat1 platform. It supports commands like dat1 login (authenticate with API key), dat1 init (create dat1.yaml config), dat1 deploy (upload model), and dat1 serve (test locally with Docker and CUDA GPU).
Does Dat1 offer enterprise support?
Yes, Dat1 provides dedicated 24/7 support, reserved capacity for mission-critical workloads, and the option to deploy Dat1 fully on-premise to meet security and compliance requirements for enterprise customers.
What files are required for deployment?
Deployment requires a dat1.yaml configuration file with model name and exclude patterns, and a handler.py file in the root directory containing a FastAPI app with GET / endpoint for healthchecks and POST /infer endpoint for inference requests.