Scaling Git to 1TB of files with GitLab and Anchorpoint using Git LFS
Git has a reputation for not scaling well for large binary projects. But with new features like sparse checkout and Git LFS, it performs at a decent speed even on large projects.
tl;dr
The setup should take you less than 10 minutes.
Create an account on GitHub, then create a repository, and add a collaborator.
Download Anchorpoint (a desktop application that pushes and pulls files from GitHub), create a new project as a Git repository, and pick your current Unity project folder.
Add the URL from GitHub and add the same collaborators to your Anchorpoint project. In the Timeline of Anchorpoint, just push all the files to GitHub.
Your team member needs to accept the project invite of GitHub and Anchorpoint, so she can join the Git repository and download all the files.
Now you can collaborate by pushing and pulling files through GitHub.
If you want to collaborate on a Unity project, you need version control. Git is the most popular solution among developers and is also well-suited for Unity projects. It’s a great alternative to Perforce, Plastic SCM, or SVN and has the biggest ecosystem of tools and cloud services. This article will help you set up a repository on GitHub and bring your team together to work on a Unity project.
To avoid confusion, Git is the version control system, and GitHub is a cloud service provider that builds upon this system. Git was developed by Linus Torvalds to manage the open-source development of the Linux kernel. The first version was released in 2005, and since then, it has been actively developed and improved by the community with around 1,600 contributors.
GitHub was launched in April 2008 and has around 83 million users (August 2022). Since 2018, it’s owned by Microsoft. GitHub is not the only way to host your Unity project in the cloud. There are plenty of alternatives like GitLab, Bitbucket, or Azure DevOps. It’s also possible to self-host a Git server if you want more control of your data.
Both are development projects that involve writing code, versioning, working in sprints, and any other agile practices. However, game development has two major differences.
In software development, UI designers have no clue about the codebase. They usually pass their designs in a handover process to the developer, who is rebuilding that in code.
In game development, artists usually work with the codebase (the Unity project) but without writing any code. They have to import their models, adjust shaders, lighting, animation, etc. This means they have to understand version control. Git, by its nature, is complicated. The command-line interface is impossible to use for artists, and most GUIs are made for developers.
A tool like Anchorpoint can bridge that gap by providing the world’s first Git GUI for artists that simplifies Git into a two-button solution for them.
While code is stored in light text files (and Git was optimized for that), textures, geometry, etc. are stored in binary files. They can reach heavy sizes that can quickly blow up a Git repository. In 2014, Git got an extension called Git LFS (large file storage) that solves that problem and is supported by nearly all GUIs and cloud services.
However, storing heavy files on GitHub can get pretty costly. The free storage and bandwidth amount on GitHub is limited to 1 GB per month. If you need more, you have to buy data packs.
When your game project requires multiple GBs of storage, it’s recommended to look at Azure DevOps or AWS code commit. They are way more generous when it comes to storage costs.
Setting up GitHub for Unity is actually pretty easy and involves just a few steps.
You can use an existing Unity project. If you are using Unity 2021 or newer, you don’t need to do any configuration. But just to double-check, make sure that asset serialization is set to force text and that meta files are set to visible. To do that, go to Edit / Project Settings…
By default, Unity does not respect the read only property of a file. Anchorpoint places a script in the Assets/Anchorpoint folder so that Unity does not overwrite read only files. You can put this script in your Editor Scripts folder. If you can still save to read-only (locked files), just place this C# script in your Assets folder. Preferably under Scripts/Editor.
1. Go to GitHub.com and create an account
2. Click on Create repository
3. Just enter a name and press Create repository
4. Invite your team members
5. Copy the repository URL to the clipboard
In this case, we will use Anchorpoint because it is the easiest one for artists and can deal very well with binary files.
1. Download and install Anchorpoint and create an account.
2. Create a new project
3. Configure the project
4. Upload your files to GitHub.
Once your team members are invited by you, they simply need to accept the invite and create a GitHub account. That’s all on GitHub.
Your team members will also receive an invite email from Anchorpoint. They have to download and install the application as well as create an account. Once they open Anchorpoint, they first have to accept the invite, and the corresponding project will open automatically.
After clicking on Join Git Repository, Anchorpoint will ask for the place where the files should be downloaded. In some cases, a popup appears that requires to install Git first, before Anchorpoint can proceed with the download.
During the download process, a popup from GitHub will also appear, where your member has to authenticate to GitHub so that Anchorpoint can download the files.
When working with Git using Anchorpoint, it is basically a two-button solution. The process is as follows:
This workflow is like a file sync with more control. The recommendation is to push changes pretty often so that your team is in sync. It’s about developing a good habit of commenting on pieces of your work so that your whole project process is more transparent.
Sometimes multiple people work on the same asset or scene, which can lead to conflicts that need to be resolved when pulling the files from GitHub.
When you both work on the same scene but on different objects, your changes can be merged. Git can merge text files, but it is not intelligent enough to merge Unity scenes. For that, you have to use their UnityYAMLMerge tool. To use it, you have to modify the .gitconfig file.
'path to UnityYAMLMerge' should be replaced by the real path to the UnityYAMLMerge.exe on your computer. Usually, you can find it under: “C:/Program Files/Unity/Hub/Editor/2021.3.21f1/Editor/Data/Tools/UnityYAMLMerge.exe”. Make sure that you use forward slashes (/) and not backslashes (\) like Windows is doing.
This is especially true for binary files. Let’s say you modified an asset and worked on the same FBX file at the same time. These changes cannot be merged, and you have to decide which file to take. Anchorpoint resolves conflicts in a very simple way. Your change wins and is taken as the latest. This does not mean that the work from your team member is lost. It’s still in the history and can be restored.
Git is a version control solution that was developed for building the Linux kernel and was initially released in 2005. Git is a decentralized version control system. This means that you have a copy of the whole repository on your computer. A repository is basically your Unity project folder on a remote server, e.g., GitHub.
The benefit of working decentralized means that you can fully work offline, you have an automated backup system, and file operations are way faster because you don’t need to wait for a server. To sync with your team, you push (upload) your files to the remote repository, and you pull (download) new file changes from the repository to your local computer. These steps are done manually, in contrast to Dropbox, which is syncing files instantly.
Git was made to deal with code, which is basically a bunch of text files. Games require a lot of binary files like 3D models, textures, sound files, etc. To keep the efficiency and to not download the whole history of your 3D models and textures, Git introduces an add-on in 2014 called Git LFS (Large File Storage). This add-on is now part of almost every Git product and service out there.
It helps to deal with binary files by storing them on a separate location on the remote server and replacing the original file with a 1kb sized pointer. This pointer only knows where the original file is so it can be downloaded to your local computer on demand. This allows you to save storage space.
When you decide on a Git hosting provider (GitHub, GitLab, Bitbucket, or Azure DevOps), you have to look at their Git LFS costs.
Unity produces a lot of overhead of files that store caches, build data, or anything else that you don’t want to upload to the Git repository. It will just take away space and slow down your push and pull procedures.
A .gitignore file is basically a filter that tells Git which files, file types, and folders should never be committed to the Git version history. Here is an example gitignore for Unity.
When a project becomes more complex or the team gets bigger, friction arises. Work in progress commits from e.g. developers can disturb the work of the artist. To prevent this, you can split the project into certain zones. Each zone can then be used by a developer or artist without getting in each other’s way. When the work is finished, these zones are dissolved and merged into the project. Git calls such a thing branches. A short video in our documentation explains them pretty well.
The recommendation is to keep the number of branches very small. The reason is, when you work on your own branch for a long time, you may never run into conflicts. But, once you merge that branch to the main line (imagine this like a giant commit), all conflicts that you ignored have to be solved now. It’s better to run into conflicts early and solve them quickly by working only on one branch.
If you need branches, Anchorpoint allows you to create and switch them in the dropdown in the Timeline next to the Pull button.
GitHub can work great if your project size is pretty small. As soon as you are in the double-digit gigabyte range, it’s smarter to pick another Git hosting provider such as Azure DevOps or AWS code commit. We made a comparison so you don’t need to spend so much time on research.