Version Control for Binary Files

TL;DR

Although standard Git is designed for text, game development and VFX require version control for binary files, such as 3D models and 4K textures. Legacy systems like Perforce and SVN use "exclusive checkout" to prevent conflicts. Unity Version Control offers a hybrid approach. In contrast, Anchorpoint provides a modern, Git-based solution which automates Git LFS and can handle massive file locking (1,000+ files in under a second), allowing artists to work safely without the complexity of traditional centralized systems.

‍

Traditional version control systems were designed for lines of code. When you change a word in a script, for example, Git only saves that specific change. However, Git is "opaque" to binary files, such as .psd, .fbx, and .blend files. Changing a single pixel in a texture causes Git to try to save the entire file again, which results in an explosion of repository sizes.

A robust system for version control for binary files is mandatory for game engines like Unreal Engine or Unity to prevent "merge hell." Unlike code, it is not possible to "merge" two different versions of a 3D character model. If two artists work on the same file simultaneously, one person's work will inevitably be overwritten.

Scaling Git with Git LFS and Anchorpoint

‍
The industry standard for handling massive assets in Git is Git LFS. It replaces large binaries in your main repository with small text pointers, while storing the actual files on a separate server, such as GitHub or a self-hosted solution.

Although Git LFS solves the storage problem, it doesn't address the "human" problem for non-technical artists. Tools like Anchorpoint function as dedicated version control solutions by automating the technical setup of LFS, allowing artists to avoid the command line and focus on their creative tasks.

1. Lock Policies: Preventing File Corruption

Since binary files cannot be merged, "exclusive locking" is the only way to ensure data integrity. Anchorpoint’s file locking system is built on a cloud-based metadata layer that extends Git’s capabilities.

• Speed: It can lock 1,000 files in under a second.
• Visual Feedback: Artists see a lock icon directly in the UI, signaling that a teammate is currently working on that asset.
• Prevention: It prevents users from pushing changes if they haven't acquired the lock, stopping accidental overwrites before they happen.

2. Mixing Git with Shared Storage

Many studios attempt to mix Git for code with Dropbox or a NAS for binaries. Without a unified interface, this often leads to "out of sync" projects where the code expects a texture that hasn't been uploaded to the server yet.

Anchorpoint acts as a single source of truth. Since it is fully compatible with Git, you can maintain your project's original structure. It facilitates communication between your local repository and your Git server, such as GitHub, ensuring that each "checkpoint," or Git commit, includes the correct versions of your code and binary assets.

3. Storage Tiers and Deduplication

Managing version control binary files requires smart storage.

• Deduplication: Git LFS ensures that if the same 50MB texture is used in two different branches without changes, it is only stored once on the server.
• Local Cleanup: Anchorpoint allows users to prune old binary data from their local machines to save disk space, while the full history remains safe on the server.

4. Previews and Review Workflows

For a lead artist or a CTO, scrolling through a list of file names is useless. Anchorpoint provides high-speed previews for videos, 3D files, and textures.

• Visual Discovery: Use tags and metadata to find assets across complex project folders.
• Simplified Review: A Git commit is like making a checkpoint for your project. Leads can review these checkpoints visually, seeing exactly what changed in a texture or model before it is integrated into the main build.

The Landscape: Options for Binary Assets

When it comes to version control for binary files, the industry has traditionally split into two camps: the "code-centric" distributed world (Git) and the "asset-centric" centralized world (Perforce, SVN). However, modern workflows are increasingly looking for a middle ground that combines the flexibility of Git with the safety of centralized locking.

Perforce: The Industry Standard

Perforce is the heavyweight in AAA game dev. It handles binary files by using a centralized server and a "lock-modify-unlock" workflow.

• Mechanism: It uses exclusive checkout. When an artist wants to edit a .uasset or .fbx, they "check out" the file, which tells the server to lock it. No one else can submit changes to that file until the first person "submits" (checks in) their work.
• ‍Why it’s used: It handles petabytes of data and millions of files efficiently because you don't have to download the entire project history to your local machine.

Anchorpoint: Modern Git-Based Locking

Anchorpoint is a version control solution for game development that is 100% compatible with Git. It is designed specifically for non-technical users like artists who need to manage project assets without learning complex DevOps commands.

• Mechanism: It uses a cloud-based metadata system to add missing features to Git, such as centralized file locking that requires no prior configuration. This system is high-performance, capable of locking 1,000 files in under a second.
• Why it’s used: It automates the asset pipeline and handles version control for binary files using Git LFS under the hood, but without the manual setup. Because it’s Git-compatible, teams can push their work to providers like GitHub while artists benefit from visual previews and an easy-to-use desktop interface.

SVN (Subversion): The Legacy Linear Option

SVN is a simpler centralized system. Like Perforce, it uses a central repository, but it’s often seen as slower and less flexible for modern branching.

• Mechanism: It relies on the svn:needs-lock property. If this is set, files are read-only on your computer until you explicitly request a lock from the server.
• ‍Why it’s used: It’s free and open-source, making it a common choice for smaller teams or modding communities that need a straightforward, linear history.

Unity Version Control (formerly Plastic SCM)

Unity’s solution attempts to bridge the gap between Git and Perforce, offering a "Gluon" mode specifically designed for artists.

• Mechanism: It uses smart locks. You can set rules (e.g., all .unity scenes or .png textures) to be locked automatically upon checkout.
• Why it’s used: It is deeply integrated into the Unity ecosystem, allowing users to manage locks directly from the Unity Dashboard or Hub.

FAQ

Why can’t I just commit large binaries to Git?

Git copies the entire history of every file. If you commit without Git LFS a 500MB file ten times, your .git folder grows by 5GB. This makes cloning and pulling unbearable for your team.

How does Anchorpoint handle 3D files that Git usually struggles with? 

Anchorpoint uses Git LFS to manage large files. It automatically configures the .gitattributes file to ensure that binary formats (like .obj, .fbx, or .exr) are tracked correctly as LFS pointers rather than raw Git objects.

What happens if two people try to edit the same binary file? 

Anchorpoint utilizes a centralized file locking system. When one user starts working, they "lock" the file. Other team members will see that the file is in use and will be unable to push changes to that specific file until it is unlocked, preventing merge conflicts.

Where to learn more

• Version Control for Game Development: https://www.anchorpoint.app/blog/version-control-for-game-development
• Git with Unreal 5: https://www.anchorpoint.app/blog/git-with-unreal-engine-5
• Git vs Perforce: https://www.anchorpoint.app/blog/git-vs-perforce-for-game-development
• File locking in Git: https://www.anchorpoint.app/blog/using-file-locking-in-git‍