Dvc
🦉 Data Versioning and ML Experiments
Last verified:
What is Dvc?
Data Version Control (DVC) is a free, open-source version control system specifically designed for Data Science and Machine Learning projects. It provides a Git-like experience to organize data, models, and experiments, bringing software engineering best practices to data science teams. DVC allows users to track large datasets and ML models alongside code without storing them in Git, using human-readable metadata files (.dvc files) as placeholders.
Key features include data versioning using MD5 hash values, ML pipeline automation through dvc.yaml files, experiment management with metrics and parameters tracking, model registry for transparency and discovery, remote storage support (Amazon S3, Azure Blob Storage, Google Cloud Storage, SSH, HDFS, HTTP, WebDAV), VS Code extension, command line interface, and Python API. DVC works on top of Git repositories, maintaining Git as the single-source-of-truth for data, experiments, and models.
DVC is designed for data scientists, machine learning engineers, and AI/ML teams who need to manage large datasets, make projects reproducible, and collaborate better. It's particularly suitable for individual data scientists working on small projects (using the open-source DVC) and enterprise AI/data engineering teams handling petabyte-scale environments (using lakeFS Enterprise). The tool is platform-agnostic, running on Linux, macOS, and Windows, and works independently of programming languages and ML libraries.
The tool integrates with major ML frameworks including PyTorch, TensorFlow, Keras, Scikit-learn, XGBoost, LightGBM, Hugging Face Transformers, Fast.ai, and PyTorch Lightning. DVC also offers DVC Studio, a web application for ML experiment tracking, visualization, and collaboration that works on top of DVC and Git.
Dvc pricing
Pricing model: Freemium
DVC open-source is 100% free and open source with no cost. DVC Studio offers a free tier that includes core experiment tracking, DVC pipeline visualization, model metrics comparison, and limited team collaboration. The free tier covers essentials for individual practitioners and small teams. For organizations needing advanced collaboration features, dedicated support, and enterprise-grade security, DVC Studio offers custom-priced enterprise plans available through direct sales conversations. Enterprise features include SSO/SAML authentication, priority support, custom integrations, SLA guarantees, on-premise deployment, audit logs, and advanced user management controls (RBAC). There is no published mid-tier plan between free and enterprise.
Dvc pros
- Free and 100% open source
- Git-like experience familiar to developers
- Works on top of existing Git repositories
- No special servers required - uses cloud storage directly
- Supports multiple remote storage types (S3, GCS, Azure, SSH, HDFS)
- Human-readable metafiles (dvc.yaml, .dvc files)
- VS Code extension available for intuitive UX
- Python API for programmatic access
- Platform agnostic (Linux, macOS, Windows)
- Language agnostic (works with Python, R, Julia, shell scripts)
- Compatible with major ML frameworks (PyTorch, TensorFlow, etc.)
- Enables reproducible ML experiments
- No additional databases or infrastructure needed
- Optimizes file hash calculation for performance
- Supports GitOps for model deployment
- Built-in experiment tracking and visualization
- Model registry for transparency and discovery
- Run cache for efficient experiment re-runs
- External dependencies and outputs support
- Works with large datasets without copying to Git
Dvc cons
- Does not replace Git - Git must be installed separately
- No built-in GUI for the open-source version
- No daemon or server runs in background
- Lacks advanced pipeline execution features like monitoring and error handling
- Purely command line tool without GUI (open-source version)
- Enterprise pricing not published - requires sales contact
- No mid-tier plan between free and enterprise
- Compute infrastructure costs separate from tool
- Storage and egress fees for large datasets accumulate independently
- Limited team collaboration features in free tier
Frequently asked questions about Dvc
What is DVC and what does it do?
DVC (Data Version Control) is a free, open-source tool for data management, ML pipeline automation, and experiment management. It helps data science and machine learning teams manage large datasets, make projects reproducible, and collaborate better by bringing software engineering best practices to data. DVC provides a Git-like experience to organize data, models, and experiments without storing large files in Git.
Does DVC replace Git?
No, DVC does not replace Git. DVC works on top of Git repositories and maintains Git as the single-source-of-truth for your data, experiments, and models. DVC metafiles like dvc.yaml and .dvc files serve as placeholders to version data and ML pipelines, which are then versioned with Git. You must have Git installed in your system to enable important features like data versioning and quick experimentation.
What storage systems does DVC support?
DVC supports many remote storage types including Amazon S3, Azure Blob Storage, Google Cloud Storage, Google Drive, Aliyun OSS, SSH & SFTP, HDFS & WebHDFS, HTTP, and WebDAV. You can also use local directories as remotes. DVC does not require special servers - any cloud storage or SSH server can be used as remote storage without additional databases or infrastructure.
How do I install DVC?
The easiest way to install DVC is with uv or pipx: uv tool install dvc or pipx install dvc. DVC can also be installed on Visual Studio Code, any system terminal, and used as a Python library. After installation, verify it's available by running dvc --help. To check your version, use dvc version.
What is DVC Studio?
DVC Studio is a web application that enables Machine Learning teams to seamlessly simplify ML experiment tracking, visualization, and collaboration on top of Git. It works on top of DVC and Git, maintaining Git as the single-source-of-truth. Features include ML experiment tracking, pipeline visualization, model metrics comparison, model registry for transparency, and GitOps support for model deployment. You can sign in using GitHub.com, GitLab.com, Bitbucket.org accounts, or email.
What ML frameworks does DVC integrate with?
DVC integrates with major ML frameworks including PyTorch, PyTorch Lightning, PyTorch Lightning Fabric, TensorFlow, Keras, Scikit-learn, XGBoost, LightGBM, MMCV, Fast.ai, Hugging Face Accelerate, Hugging Face Transformers, Optuna, and Ultralytics YOLO v8. It also integrates with Amazon SageMaker, Databricks, Evidently, and Hugging Face.
How does DVC track data versions?
DVC tracks files based on their MD5 hash values instead of timestamps, which avoids heavy processes like model retraining when checking out previous versions. When you run dvc add, DVC stores information about the file in a .dvc metadata file that acts as a placeholder. The actual data is moved to the project's cache and linked back to the workspace. The hash value determines the cache path, enabling fast version switching.
What is the difference between DVC and lakeFS?
DVC focuses on lightweight, file-based data versioning for data science workflows using Git repositories, ideal for individual data scientists and small projects. lakeFS is a Git-like data version control system for large-scale data lakes and object stores (S3, GCS, Azure), designed for data engineering and production AI workloads with petabyte-scale multimodal object stores. lakeFS operates as a control plane over centralized storage enabling branching, commits, and rollbacks without data duplication.
Can DVC work without Git?
DVC can work stand-alone without Git, but without versioning capabilities. DVC is not fundamentally bound to Git and can work without it except for versioning-related features. However, Git is recommended to enable important features such as data versioning and quick experimentation. DVC works best on top of Git repositories with similar feel and flows to regular Git workflows.
How do I share and collaborate on DVC projects?
To share DVC projects, upload DVC-tracked data to remote storage using dvc push after configuring a remote with dvc remote add. Team members can retrieve data with dvc pull after git pull or git clone. For collaboration, DVC Studio provides web-based experiment tracking, visualization, and team collaboration features. You control access to all aspects of your project and share them with chosen people and teams through secure collaboration.