Assets, Unpacking & On-Demand Pull

Assets and Traditional Farm Bottlenecks

In 3D animation and VFX pipelines, asset files (textures, HDRIs, cache caches, audio, and linked libraries) usually account for over 90% of the project's total size.

Artists often enable "Pack Resources" in Blender, embedding hundreds of textures into a .blend file, or repeatedly link shared character and environment libraries across multiple shots. In traditional render farms, this causes severe bottlenecks:

• Bloated files & explosive versioning: Tweak a single keyframe or light intensity, and the newly saved file still carries gigabytes of embedded textures. Every submission requires uploading gigabytes over again, wasting disk space and bandwidth.
• Redundant transfers: When 5 render nodes collaborate on a shot, traditional workflows force each machine to sync all project assets in full, even if a node only renders a distant background element.
• Cross-platform path breakage: Textures referenced on Windows (such as Tex\Wood.PNG) fail on Linux cloud instances due to case mismatches (where the disk file is tex\wood.png) or backslash separators, resulting in pink missing textures.

danio solves these challenges through automatic asset unpacking, content-addressable deduplication, and on-demand Range pulling.

Automatic Unpacking & Streamlined Projects

When you submit via the Blender extension or drag a file into danio, the background analyzer traces all external and internal dependencies:

1. Automatic unpacking: Textures, fonts, and sounds packed into the main .blend or linked libraries are extracted into danio's local cache, named after their content hash without altering your original files.
2. Lightweight snapshots & reference rewriting: Extracted assets are mapped as standardized relative references in the snapshot. The submitted entry .blend copy strips embedded textures, slimming down to a lightweight project structure (often just a few megabytes).
3. Cache acceleration: Resubmitting the same library or .blend reuses prior extraction and rewriting results if contents have not changed, reducing scanning time for large scenes to seconds.

Upload Once per Texture (Global Deduplication)

danio manages assets by content-addressable storage (CAS) using SHA-256 hashes instead of file names or folder paths:

• Cross-version deduplication: When updating a task to Version 2 or 3, unchanged textures keep identical hashes. Danio queries the host's index before upload, and already-uploaded textures are completely skipped, transmitting only the modified .blend.
• Cross-project & cross-shot deduplication: Deduplication spans across the entire workspace. If Shot 01, Shot 02, or a completely different project shares the same 4K HDRI or character model, that asset is stored only once and uploaded only once in the workspace.
• In-place ingestion on shared NAS: When projects reference shared assets on a studio NAS, if the host runs directly on the NAS (such as in Docker), it reads and packs assets directly on the NAS disk. Artist workstations never need to round-trip heavy assets over the local network.

Pack Volumes & On-Demand Range Pulling

To balance network efficiency and object storage costs, danio avoids scattering millions of loose files across storage (which incurs heavy API fees) while avoiding monolithic single-archive downloads:

• ~256 MB Pack volumes: Files smaller than 64 MB (textures, shader scripts) are bundled into ~256 MB immutable tar packs grouped by directory proximity. Entries use fast zstd compression (pre-compressed formats like PNG/JPEG/MP4 are stored as-is). Files 64 MB and larger, as well as the main .blend, get their own dedicated packs.
• On-demand Range pulling: When a worker takes a chunk, it compares its local cache and identifies only the files needed for that chunk. In object storage (such as Cloudflare R2), the worker uses HTTP Range headers (bytes=offset-length) to read only the exact byte range of the target file, without downloading the entire 256 MB pack.
• Zero-copy hardlink mounting: Pulled files are verified against their SHA-256 hashes and cached locally. When preparing a job environment, danio links files into jobs/ via filesystem hard links or block cloning, creating the workspace instantly without duplicating disk consumption.

Linked Libraries & Cross-Platform Reliability

Large productions rely extensively on Blender's Linked Libraries. danio safeguards these assets automatically:

• Missing library alerts & one-click relinking: If a referenced .blend library path is missing during submission, danio flags it with a yellow warning and lets you select the new location directly in the submission dialog, rewriting the snapshot without modifying your source files.
• Case sensitivity correction: Windows and macOS filesystems are case-insensitive, so referenced texture paths frequently mismatch the exact casing on disk. Linux render nodes reject mismatched casing. danio checks actual disk names and rewrites paths to match disk casing, ensuring textures load cleanly on Linux cloud nodes.
• Long path protection: Windows imposes a 260-character path limit. For deeply nested project directories, danio automatically applies \\?\UNC path prefixes to rendering copies, preventing Blender from failing to load long asset paths.