Stage 2 · Tools
Remote Collaboration
Large Repos & Shallow Clones
sparse-checkout, partial clone, LFS — working with big repos efficiently.
The Problem with Large Repos
As repositories grow — millions of files, gigabytes of history, large binary assets — Git operations slow down. Clone times increase, disk usage balloons, and commands like git status and git log become sluggish. This is common in monorepos, game projects with assets, and machine learning repos with datasets.
Git provides several mechanisms to work with large repositories without downloading the entire history and all blobs. These techniques reduce clone time, disk usage, and network transfer while keeping the full Git workflow intact.
All these techniques sacrifice some aspect of completeness — full history, all file versions, or all blobs — to improve performance. Choose the right trade-off for your use case.
Shallow Clones
A shallow clone downloads only the most recent N commits instead of the full history. This dramatically reduces clone time and disk usage for repositories with deep history. You get a working repository with the latest code — you just can't see or traverse older history.
git clone --depth 1 https://github.com/large-org/monorepo.gitDownloads only the latest commit. The clone is instant regardless of repository size. You can work with the code, make commits, and push — but git log only shows one commit.
git clone --depth 10 https://github.com/large-org/monorepo.gitDownloads the 10 most recent commits. Enough to see recent context without the full history. Adjust the depth based on how far back you need to look.
git fetch --unshallowDownloads the full history, converting a shallow clone into a regular clone. Useful when you need git blame or bisect on older commits after initially doing a shallow clone.
CI systems rarely need full history. Use --depth 1 or --depth 10 in CI pipelines to cut clone times from minutes to seconds. Many CI providers do this automatically.
Partial Clones
Partial cloning lets you skip downloading file content (blobs) while keeping the full commit history and tree structure. You get the metadata — commit messages, file names, directory structure — without the actual file contents until you need them.
git clone --filter=blob:none https://github.com/large-org/monorepo.gitDownloads commits and trees but no file content. When you check out a file, Git fetches its blob on demand. First checkout of each file is slower, but the initial clone is fast.
git clone --filter=blob:limit=1m https://github.com/large-org/monorepo.gitDownloads small blobs immediately but skips blobs larger than 1MB. Large files are fetched on demand. This is useful when most files are small but the repo has some large assets.
git fetch --filter=blob:none origin mainFetches only blobs needed for the current working tree. After a partial clone, this ensures you have the content for the files you're editing.
Shallow clones limit history depth. Partial clones limit blob download. You can combine both: git clone --depth 1 --filter=blob:none gives you minimal history and minimal blobs — the fastest possible clone.
Sparse Checkout
Sparse checkout lets you clone a repository but only check out a subset of directories. In a monorepo with hundreds of services, you might only need one service's directory. Sparse checkout avoids downloading the entire working tree.
git clone --sparse https://github.com/large-org/monorepo.git
cd monorepo
git sparse-checkout set services/auth services/apiClones the repo with an empty working tree, then enables sparse-checkout for only the specified directories. Only files under services/auth and services/api appear in your working tree.
git sparse-checkout add services/payments shared/libExpands the sparse-checkout to include additional directories. The new directories are populated immediately from the existing git objects.
git sparse-checkout disableRestores the full working tree. All directories and files appear. Useful when you need to search across the entire codebase or run cross-cutting tools.
Cone mode (git sparse-checkout init --cone) is faster for simple directory patterns. It uses a optimized algorithm for matching top-level directories. Use it unless you need complex glob patterns.
Git LFS
Git Large File Storage (LFS) replaces large files with text pointers in the Git repository while storing the actual file content on a separate LFS server. This keeps the Git repository small while making large files available when needed.
git lfs install
git lfs track "*.psd"
git lfs track "*.zip"
git lfs track "assets/**"Installs LFS hooks and configures which files to track. The git lfs track commands update .gitattributes. LFS intercepts these files during git add and stores them on the LFS server.
git clone https://github.com/team/game-assets.git
cd game-assets
git lfs pullNormal clone gets the LFS pointers. git lfs pull downloads the actual large files. Some Git clients and CI systems do this automatically.
# Images
*.psd filter=lfs diff=lfs merge=lfs -text
*.png filter=lfs diff=lfs merge=lfs -text
*.jpg filter=lfs diff=lfs merge=lfs -text
# Archives
*.zip filter=lfs diff=lfs merge=lfs -text
*.tar.gz filter=lfs diff=lfs merge=lfs -text
# Media
*.mp4 filter=lfs diff=lfs merge=lfs -text
*.wav filter=lfs diff=lfs merge=lfs -textThe .gitattributes file maps file patterns to LFS tracking. Every matched file is stored via LFS. Commit this file to the repository so all contributors use the same LFS configuration.
LFS requires a server that supports it (GitHub, GitLab, Bitbucket all do). Self-hosted Git servers need the LFS server component installed. Without it, LFS pointers are stored as plain text files.
Performance Comparison
The right technique depends on your bottleneck. Here's how each approach compares for a repository with 500K commits, 100K files, and 50GB of history.
| Technique | Clone time | Disk usage | Best for |
|---|---|---|---|
| Full clone | 10+ minutes | 50GB+ | Full history, bisect, blame |
| Shallow (depth 1) | < 10 seconds | 100MB | CI, quick checks |
| Partial clone | < 30 seconds | 200MB | Browse history, light editing |
| Sparse checkout | < 30 seconds | Varies | Work on one service in a monorepo |
| LFS | Varies | Varies | Large binary assets |
These techniques are composable. git clone --filter=blob:none --sparse gives you a partial clone with sparse-checkout — maximum speed with full history metadata. Add --depth 1 for even faster clones.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.