Git Reference ยท Objects, Refs and Internals
Packfile format
Reference for Git packfiles and pack indexes: the PACK header, object type numbers, OFS and REF deltas, the .idx lookup file, and how to inspect packs.
A packfile stores many Git objects in one compressed file, often with deltas so that similar objects share bytes. Git creates packs when you run git gc or git repack, when fetch and clone download objects, and when push uploads them. Loose objects (one zlib file each) are convenient for new data; packs are what keeps a repository with a long history small and fast.
A pack lives in .git/objects/pack/ as a pair of files: pack-<name>.pack holds the objects and pack-<name>.idx is a sorted index that maps object IDs to offsets in the pack so a single object can be found without scanning. Newer repositories may also have .rev (reverse index), .bitmap and .mtimes files next to them.
git repack -a -d # put everything into one pack git verify-pack -v pack.idx # list objects, sizes and delta chains git count-objects -v # loose versus packed totals
Try it
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.
Press Run (or Ctrl + Enter) to see the result here.
Program input ยท what you'd type while the program runs
Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.
Format of the .pack file
"PACK" 4 bytes version 4 bytes, big-endian (2 or 3; Git writes 2) object count 4 bytes, big-endian object entries ... pack checksum 20 bytes (SHA-1) or 32 bytes (SHA-256) over everything before it
Each entry starts with a variable-length header. The first byte holds a "more bytes follow" flag in its top bit, a 3-bit type in the next bits and the low 4 bits of the size; each following byte adds 7 more size bits. The size is the size of the object after decompression. The data that follows is compressed with zlib.
| Type number | Meaning |
|---|---|
| 1 | Commit. |
| 2 | Tree. |
| 3 | Blob. |
| 4 | Tag. |
| 6 | OFS delta: a delta against an object earlier in the same pack, named by a negative offset. |
| 7 | REF delta: a delta against an object named by its full ID. It may be outside the pack (thin packs). |
(Type 5 is reserved.)
Deltas
A delta is a list of instructions that rebuild a target object from a base object. The delta data starts with the sizes of the base and the target, then contains two kinds of instruction: copy a range of bytes from the base, or insert literal bytes. Deltas can be chained, with a limit set by pack.depth (default 50) and repack --depth. Git chooses bases by sorting objects by type, name and size and trying neighbours within --window (default 10). A newer version of a file is often stored whole and the older ones as deltas against it.
Format of the .idx file (version 2)
| Section | Description |
|---|---|
| Header | Magic bytes ff 74 4f 63 and version number 2. |
| Fan-out table | 256 four-byte counters; entry N holds how many objects have a first ID byte of N or less. It narrows a lookup to a small range. |
| Object IDs | All IDs, sorted. |
| CRC32 values | One per object, over the packed bytes, to copy data between packs without recompressing. |
| 4-byte offsets | One per object; if the top bit is set, the rest is an index into the 8-byte table. |
| 8-byte offsets | Only for packs larger than 2 GiB. |
| Trailer | The checksum of the pack and the checksum of the index. |
A lookup does a binary search inside the range given by the fan-out table, then reads the offset and jumps into the pack.
Usage notes
- Packs are immutable. Git never edits a pack; it writes a new one and deletes the old one.
git gcandgit repack -a -ddo this safely. - Thin packs. On the wire,
pushandfetchmay send REF deltas against objects the other side already has. The receiver completes them withindex-pack --fix-thin. - Reachability bitmaps (
.bitmap) speed upcloneandfetchby answering "which objects are reachable from this commit" without walking history. - Multi-pack-index (
multi-pack-index) lets Git treat many packs as one for lookups. - Pack name. The file name is a hash that identifies the pack; do not rely on its value and never rename packs by hand.
- Cruft packs hold unreachable objects together with their modification times so
gccan expire them later without creating thousands of loose files. - Limits.
pack.packSizeLimitcaps pack size,core.bigFileThreshold(default 512 MiB) stops Git from trying deltas on huge blobs.