rsync, reimagined.

Measured speedups over rsync 3.2.7

Docker containers, tc netem traffic shaping, LAN profile (1 Gbps, 0.5 ms RTT). Reproducible: bash bench/docker_network_benchmark.sh

rsync vs resync

Architectural differences that produce the speedup.

Nine reasons for the speedup

Each optimization compounds. Together they produce 4–56× over rsync.

1. Parallel everything

   Rayon work-stealing thread pool across directory scanning, BLAKE3 hashing, delta computation, and file application. rsync uses exactly one core.

2. Zero-copy I/O

   mmap with madvise(SEQUENTIAL|HUGEPAGE), copy_file_range(2) for kernel-space copies. Bytes never enter userspace buffers.

3. BLAKE3 SIMD hashing

   10–15× faster than MD5. Auto-vectorizes with AVX-512, AVX2, NEON, SSE4.1. Batched parallelism: 128 chunks per rayon task.

4. Content-defined chunking

   FastCDC with Gear rolling hash. An insert at byte 0 no longer shifts every chunk boundary. 95%+ hash preservation on prepend edits.

5. Zstd compression

   10× faster than zlib at equivalent ratios. Network protocol uses zstd frame compression on all wire messages.

6. No fsync by default

   rsync calls fsync() per file, flushing NVMe write cache. resync trusts the OS page cache. Use --fsync when durability matters.

7. Manifest cache

   Binary manifest + tiny TOC for dir-mtime checks. Skip destination re-scan entirely. With --trust-mtime, skip source stat() too: sub-5ms at 100K files.

8. Inode-sorted processing

   Files sorted by inode number before I/O. Minimizes seeks on rotational media, optimizes readahead on SSDs.

9. Pre-allocated output

   fallocate(2) reserves space before writing. Zero fragmentation on ext4/xfs/btrfs. Eliminates metadata updates during write.

Architecture

Two-phase pipeline: sequential classification, then parallel I/O. Files sorted by inode.

Scanner ──▸ FilterEngine ──▸ Phase 1: Sequential Decision
                                  │
                    ┌──────────── ┼ ────────────┐
                    │             │              │
                  New file     Skip         Update
                    │                          │
              copy_file_range         Hasher (BLAKE3 + mmap)
              (zero-copy)                      │
                                          CdcChunker (FastCDC)
                                               │
                                          DeltaEngine (hash-map)
                                               │
                                          Applier (atomic write)

              Phase 2: Parallel I/O (rayon work-stealing)
              Files sorted by inode for optimal disk access

Security

Defense in depth at every layer.

• BLAKE3256-bit. No known collisions. Immune to length-extension attacks.
• TLS 1.3Memory-safe rustls. No OpenSSL. Forward secrecy. Certificate verification.
• Atomic writesWrite to temp, rename(2). Crash never corrupts the destination file.
• Path traversalRejects ".." components and absolute paths. Server blocks /etc, /proc, /sys, /dev.
• DecompressionZstd messages capped at 256 MiB. Oversized payloads rejected.
• setuid stripNon-root users automatically strip setuid and setgid bits from synced files.
• Streaming I/OPull mode streams to disk via temp files. No OOM on large transfers.