You can, but you shouldn't. The reason is that the size of your repository will increase drastically. Depending on your Git hosting provider, this could result in higher storage costs.
Furthermore, operations such as checking out a branch or switching branches will take longer, as all the binaries will have to be fetched and downloaded again.
As a general rule, you should keep generative files, such as binaries or render exports, out of your repository, since these can be created from any project version.
The Anchorpoint binary sync feature will allow you to keep these binaries on an external storage solution such as Google Drive or an S3 bucket.