EPOCH VCS - Distributed Version Control System
A distributed Version Control System built in Rust using Merkle DAGs for history tracking, optimized for large binary assets with delta compression and cryptographic verification for commit integrity.
Visit website
The problem
Existing version control systems like Git were originally designed for source code — relatively small text files with line-based diffing. When repositories contain large binary assets (game assets, ML model weights, scientific datasets), Git's performance degrades significantly. Clone times balloon, storage requirements explode, and the lack of efficient binary diffing means every revision stores a full copy.
The solution
EPOCH is a distributed Version Control System engineered from the ground up in Rust, using Merkle DAGs for history tracking with a focus on large binary assets and high-throughput repositories. Unlike Git, EPOCH implements intelligent delta compression algorithms that work efficiently on binary data, dramatically reducing storage requirements.
The system integrates cryptographic verification at every level — from individual chunks to complete commit histories — ensuring data integrity without sacrificing performance. EPOCH's architecture leverages Rust's zero-cost abstractions and ownership model to achieve memory safety without garbage collection overhead.
Technical implementation
EPOCH's core is built on a content-addressable storage layer using Merkle DAGs, where each node represents a content-hashed data chunk. The system implements advanced distributed systems concepts with a focus on performance and reliability.
Key technical features include:
- Merkle DAG-based history with content-addressable storage
- Delta compression algorithms optimized for binary assets
- Cryptographic verification (SHA-256) for commit integrity
- Async I/O with zero-copy optimizations in Rust
- Chunked file storage with deduplication
- Efficient branching and merging with conflict resolution
- Network protocol for distributed repository synchronization
Outcomes & impact
EPOCH rivals Git in specific large-file benchmarks, demonstrating significant improvements in storage efficiency and clone times for repositories with large binary assets. The delta compression algorithms reduce storage requirements by up to 60% compared to naive full-copy approaches.
The project demonstrates deep expertise in distributed systems, cryptography, and systems programming in Rust. The implementation showcases advanced concepts including content-addressable storage, Merkle tree verification, and efficient binary diffing algorithms — core infrastructure engineering typically found in senior/staff-level systems work.
Architecture & future
EPOCH's modular architecture separates the storage layer, DAG operations, network protocol, and CLI into independent crates, allowing each component to be tested and optimized independently. Future work includes implementing a distributed lock-free merge algorithm, adding support for partial repository cloning (sparse checkout), and building integrations with CI/CD pipelines for automated binary asset management.