Cerebrium
Show HN: A Highly Available Distributed Router for Global Realtime AI
Last verified:
What is Cerebrium?
Cerebrium is a serverless GPU infrastructure platform designed for real-time AI workloads like voice agents, video models, LLMs, and generative AI applications. The platform delivers sub-second to 2-4 second cold starts through memory and GPU snapshotting, enabling low-latency responses from the first request. It provides elastic GPU scaling that handles sudden bursts and scale-outs automatically without compromising performance or user experience.
Thalamus is Cerebrium's highly available distributed router that makes critical routing decisions for global realtime AI workloads. When requests arrive, Thalamus determines which cluster across multiple data centers, continents, and GPU providers should serve them. It balances latency, capacity, health, cost, cache locality, and customer constraints to route requests to healthy, cheaper, lower-latency clusters. The router uses probabilistic decision-making weighted by available capacity and modifiers like network latency and health checks, with automatic failover when clusters or data centers become unhealthy.
Cerebrium is built for teams deploying production-grade AI applications that need reliability at scale, including voice agent developers, video model creators, LLM practitioners, and generative AI companies. The platform supports 12+ GPU types including H100, A100, B200, and RTX PRO 6000 across multiple regions (us-east-1, eu-west-2, eu-north-1, ap-south-1). Key features include WebSocket endpoints, streaming endpoints, REST API endpoints, OpenTelemetry integration, private Docker images, concurrency and batching, asynchronous jobs, distributed storage, multi-region deployments, auto-scaling, custom Dockerfiles, CI/CD with gradual rollouts, and secrets management.
Cerebrium pricing
Pricing model: Freemium
Cerebrium uses pay-per-second pricing based on actual compute time. GPU prices range from $0.000164/s (T4) to $0.00167/s (B200), with H100 at $0.000944/s and A100 (80GB) at $0.000583/s. Memory costs $0.00000222/GB/s and storage costs $0.05/GB/month with first 100GB free. CPU-only costs $0.00000655/vCPU/s. Three plans are available: Hobby (free + compute) includes 3 user seats, up to 3 deployed apps, 500 containers, 5 concurrent GPUs, stack & intercom support, and 1 day log retention. Standard ($100 + compute/month) includes unlimited seats, unlimited apps, 1000 containers, 30 GPU concurrency, custom domains, and 30 days log retention. Enterprise offers custom pricing with volume discounts, unlimited concurrent GPUs, dedicated Slack support, white glove onboarding, and ML engineering services.
Cerebrium pros
- Sub-second to 2-4 second cold starts with GPU snapshotting
- Instant autoscaling without capacity planning
- No Kubernetes management required
- Pay-per-second pricing with no idle costs
- Access to 2500+ GPUs across multiple clouds
- Multi-region failover with 99.999% uptime
- Thalamus router enables global deployment automatically
- No reservations or lock-ins required
- Bring your own code without rewrites or custom SDKs
- End-to-end observability with real-time logs and metrics
- Native OpenTelemetry integration for existing monitoring
- SOC 2, HIPAA, GDPR, and ISO 27001 compliance
- Data residency control for specific region deployment
- gVisor-based hardened container isolation
- WebSocket and streaming endpoint support
Cerebrium cons
- Cannot use AWS or GCP cloud credits
- GPU scarcity may limit availability during peak demand
- Limited to 4 regions currently (us-east-1, eu-west-2, eu-north-1, ap-south-1)
- Hobby plan limited to 3 deployed apps and 3 user seats
- Hobby plan has only 1 day log retention
- Standard plan costs $100/month minimum plus compute
- No guaranteed capacity without minimum monthly spend commitment
- Enterprise pricing is custom and not publicly listed
Frequently asked questions about Cerebrium
What is Thalamus and what does it do?
Thalamus is Cerebrium's highly available distributed router for global realtime AI workloads. It makes routing decisions when requests arrive, determining which cluster across multiple data centers, continents, and GPU providers should serve them. Thalamus balances latency, capacity, health, cost, cache locality, and customer constraints to route requests to healthy, cheaper, lower-latency clusters with automatic failover.
How fast are cold starts on Cerebrium?
Cerebrium delivers 2-4 second cold starts, with sub-second cold starts possible using memory and GPU snapshotting for fast restores. This is significantly faster than competitors: Cerebrium with snapshots achieves 3.38s vs Provider A at 61s and EKS/GKE at 91s.
How does Cerebrium pricing compare to AWS on-demand/spot?
Cerebrium pricing shouldn't be directly compared to raw CPU/GPU instance prices on AWS. Traditional cloud compute includes minutes of provisioning, warm-up, and idle time you still pay for, plus overprovisioning costs. Cerebrium scales containers up and down in 1-3 seconds with snapshotting for faster restores, reducing billable startup overhead. Pricing includes orchestration, networking, and the serverless platform, and Cerebrium integrates across multiple cloud providers globally to route workloads to the most cost-efficient infrastructure.
Can I use my AWS or GCP credits on Cerebrium?
No. Cerebrium pricing is separate from AWS and GCP, so their cloud credits cannot be applied to usage on the Cerebrium platform.
Is there a discount for larger deployments or long-term contracts?
Yes. Cerebrium offers discounts for larger deployments and longer-term commitments. Pricing depends on expected spend, number of consecutive months maintaining that spend, and specific GPU or compute SKUs needed. Discounts vary based on current infrastructure availability. For larger or longer-term workloads, customers should reach out to the team for tailored pricing.
Do you offer guaranteed capacity without traditional reservations?
Yes. For bursty workloads, Cerebrium can guarantee access to capacity without requiring reservations and paying for infrastructure 24/7. Customers pay for compute used with a minimum monthly spend commitment. For example, Cerebrium may guarantee access to up to 50 H100s at any point for however long needed, with a $10,000 minimum monthly spend.
What GPU types are available on Cerebrium?
Cerebrium offers 12+ GPU types including B200, H200, H100, RTX PRO 6000, A100 (80GB and 40GB), L40s, A10, L4, and T4. Each GPU type has different pricing per second, with B200 at $0.00167/s being the most expensive and T4 at $0.000164/s being the most affordable GPU option.
What regions does Cerebrium support?
Cerebrium currently supports four regions: us-east-1 (N. Virginia), eu-west-2 (United Kingdom), eu-north-1, and ap-south-1 (India). Multi-region deployments are available in beta, allowing apps to be deployed across three continents for reduced latency, global scaling, and data sovereignty with region-aware infrastructure.
What security and compliance certifications does Cerebrium have?
Cerebrium is SOC 2, HIPAA, GDPR, and ISO 27001 compliant. The platform is built to meet strict security and privacy standards, providing a compliant foundation for sensitive and regulated workloads. Each workload runs on top of gVisor in a hardened, isolated environment for strong container isolation without compromising performance. Data residency controls allow deploying workloads in specific regions to meet regulatory or contractual data privacy requirements.
How do I deploy my application on Cerebrium?
Cerebrium requires no rewrites, decorators, or custom SDKs. Point to your entry point or Dockerfile and Cerebrium will run your application exactly as is - versioned, reproducible, and ready to scale. You can use the CLI command 'cerebrium run training_script.py::train --hardware HOPPER_100:8' to deploy, or configure via cerebrium.toml file specifying hardware, region, CPU, and memory settings.