Skip to content
Tutorials Center
My learning

Topics

Lesson 51 of 78 ยท Customizing and Automating Git

.gitattributes, Line Endings and Large Files

Learn .gitattributes: fix CRLF/LF line endings, mark binary files, export-ignore, custom diff and merge drivers, clean/smudge filters, and how Git LFS stores large files.

Git treats every file as bytes and stores them exactly as they are. That is simple, and it is also a problem: Windows editors end lines with two characters (CRLF), Linux and macOS with one (LF), some files are not text at all, and a few files need special handling when they are compared, merged or exported. The file .gitattributes is where you tell Git, per path pattern, how to treat each kind of file. Because it is committed with the project, every contributor gets the same rules.

In this lesson you will learn how attributes work, fix line endings properly, mark binary and generated files, trim release archives with export-ignore, teach Git to diff and merge special formats, and see how filters and Git LFS keep large files out of your history. Everything runs in the sandbox except LFS itself, which needs a server.

Attributes: patterns and what they do

A .gitattributes file has one rule per line: a pattern (the same glob style as .gitignore) followed by one or more attributes.

TEXT
*.txt        text
*.png        binary
*.bat        text eol=crlf
docs/**      export-ignore
poetry.lock  merge=keepmine -diff

An attribute can be set (text), unset (-text), set to a value (eol=crlf) or unspecified (not mentioned, so Git decides). The file can live in any directory and applies to that subtree; the one at the repository root is the most common. For rules that should not be committed, use .git/info/attributes, the equivalent of .git/info/exclude.

To see what Git concluded for a path, ask it with git check-attr:

CONSOLE
git check-attr text eol diff -- notes.txt

Line endings: LF, CRLF and text=auto

On Windows a line ends with \r\n, on Linux and macOS with \n. If one developer commits CRLF files and another LF files, every line of a file can show as changed and merges turn into noise. The cure is to store LF in the repository and convert at the edges if needed. The text attribute says "this is a text file; normalize its line endings when adding to the index". text=auto lets Git guess: it treats a file as text unless it looks binary.

git ls-files --eol shows the truth. For each file it prints the line ending in the index (i/), in the working tree (w/) and the attribute that applies (attr/).

Bash Normalizing line endings with .gitattributes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Without attributes win.txt was stored as i/crlf: the repository itself contained carriage returns. After adding .gitattributes and running git add --renormalize . the index entry became i/lf, while the file on disk still has w/crlf until it is checked out again. The PNG stayed -text (binary: untouched) because the binary attribute is a shortcut for -text -diff -merge, as the check-attr output shows: text: unset, diff: unset.

Add .gitattributes to the project before the first commit, or run git add --renormalize . and commit once when you introduce it. Otherwise old commits keep CRLF and new ones have LF, and git blame blames the normalization commit for every line. You can hide that commit later with .git-blame-ignore-revs (blame, grep and Searching History).

Choosing the working-tree ending: eol

Storing LF inside the repository does not decide what you see on disk. The eol attribute does: eol=lf checks out LF everywhere, eol=crlf checks out CRLF on every platform. Use eol=crlf for files that really must have CRLF on all machines, such as Windows batch files, and eol=lf for shell scripts, which break when they contain \r (bad interpreter: /bin/sh^M).

Bash eol=crlf and eol=lf at checkout

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

On disk run.bat has \r \n after every line, run.sh only \n, and the committed blob of run.bat contains just \n. That is the whole idea: one canonical form in the repository, the right form per file on disk.

core.autocrlf and why attributes are better

Before .gitattributes was common, the advice was a per-user setting: core.autocrlf=true on Windows (convert to LF on add, to CRLF on checkout) and core.autocrlf=input on macOS and Linux (convert to LF on add only). It still works, and the Git for Windows installer offers it as a default. But it lives in each person's config, so rules differ between machines. A committed .gitattributes beats it, because it travels with the repository and overrides the personal setting.

core.safecrlf adds a safety net: true makes Git refuse a conversion that would not be reversible, warn only complains.

Bash core.autocrlf and core.safecrlf (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

With core.autocrlf=true the CRLF file went into the index as i/lf and came back out with \r\n, even though this sandbox runs Linux. The file mixed.txt has one CRLF and one LF line, so converting it would change bytes in a way Git could not undo; core.safecrlf=true stopped it with fatal: LF would be replaced by CRLF in mixed.txt and exit code 128.

Do not use core.autocrlf as your only protection in a team. One developer without it can still commit CRLF files. Commit a .gitattributes with * text=auto (or explicit text eol= rules) and treat the personal setting as a convenience.

Binary files and generated files

Git decides whether a file is binary by looking for NUL bytes, but you can say it explicitly. binary means no line-ending conversion, no textual diff and no textual merge. For generated files that are text but huge and unreadable (lock files, minified bundles, snapshots), -diff keeps them out of git diff output, and linguist-generated or linguist-vendored makes GitHub collapse them in pull requests and exclude them from language statistics (read-only: the sandbox has no GitHub).

TEXT
*.png                 binary
*.min.js              -diff
package-lock.json     -diff linguist-generated
vendor/**             linguist-vendored

Trimming releases with export-ignore

git archive builds a tar or zip from a commit, without the .git directory. Files with export-ignore are left out of that archive, so developer-only things such as tests and CI files do not ship in release tarballs. export-subst additionally expands placeholders like $Format:%H$ in marked files with the commit hash.

Bash export-ignore and git archive

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The repository tracks five files (including .gitattributes itself), but the archive holds only docs/guide.txt and src/app.txt (and their directories). Attributes are read from the commit you archive, which makes releases reproducible.

Custom diff and merge behaviour

Diff drivers and textconv

Git can show readable diffs for formats that are not plain text if you give it a textconv command that turns the file into text for display. The classic uses are word processor documents, PDFs, images (show metadata) and spreadsheets. The stored file is untouched; only git diff, git log -p and git show use the converted view.

Bash A textconv diff driver

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The diff lines are upper case because the driver upper converted both versions before comparing, which proves that Git used the converter rather than the raw bytes. The pattern for real tools is the same: git config diff.pdf.textconv "pdftotext" plus *.pdf diff=pdf (read-only here; those tools are not installed in the sandbox).

Merge drivers

When both branches change the same file, Git runs its built-in text merge and may create conflict markers (Resolving Merge Conflicts). The merge attribute selects a different strategy for matching files. merge=binary always conflicts instead of mixing bytes, merge=union keeps the lines of both sides, and merge=<name> calls a driver you define in config. The tiny driver below is the command true, which exits successfully without touching the file, so our version is kept and there is never a conflict.

Bash A merge driver that keeps our version

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Without the driver definition, the attribute is ignored and the merge conflicts in a.txt. With the driver defined, Git reports a normal merge and the file still says A. A real driver is a script that receives the base, ours and theirs files (%O %A %B) and writes the merged result into %A; use it for formats where a line-based merge makes no sense, such as lock files you regenerate or version files you always bump.

Important: merge.<name>.driver lives in config, not in .gitattributes, so every clone needs the setting. Git cannot let a repository run programs on your machine just because you cloned it.

Filters: clean and smudge

A filter transforms content in both directions. The clean command runs when a file goes into the index (git add), the smudge command when it comes out to the working tree (git checkout). Typical uses are keeping secrets or machine-specific values out of commits, expanding keywords, and Git LFS.

Bash A clean/smudge filter that hides a host name

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The index contains db_host=@@HOST@@, so the internal host name never enters history, while the working tree gets localhost after checkout. git status shows nothing, because Git runs the clean filter before comparing. filter.<name>.required true makes Git fail instead of silently skipping a missing filter. Filters are powerful and easy to get wrong (they must be fast, deterministic and reversible), so prefer a plain config template plus a local, ignored file for settings, and keep filters for cases like LFS.

Keyword ident

The ident attribute is the one built-in filter: on checkout $Id$ becomes $Id: <blob hash> $, and on add it is shrunk back. It is rarely used, but it is a good example of what a filter does.

Large files and Git LFS

Git stores every version of every file forever, and every clone downloads all of them. A 50 MB design file edited ten times adds about 500 MB to every clone, and deleting it later does not shrink history. Git compresses well for text, but not for already compressed binaries such as videos, archives, PSD files and ML models.

Git LFS (Large File Storage) is an extension that keeps big files outside the repository. In history Git stores a tiny pointer file, and LFS keeps the real content on a separate server and downloads it only for the commit you check out. It uses exactly the mechanisms from this lesson: a filter (to swap content for a pointer) driven by .gitattributes. The git lfs command is not installed in the sandbox, so the commands below are read-only:

CONSOLE
$ git lfs install                  # once per machine: registers the filters and hooks
$ git lfs track "*.psd" "*.mp4"    # writes lines to .gitattributes
$ cat .gitattributes
*.psd filter=lfs diff=lfs merge=lfs -text
*.mp4 filter=lfs diff=lfs merge=lfs -text
$ git add .gitattributes design.psd
$ git commit -m "Track design files with LFS"
$ git push origin main             # uploads the pointer commit and the large files to the LFS store
$ git lfs ls-files                 # which files are stored in LFS

What lands in the repository is a pointer of three lines. You can build one yourself with sha256sum, which shows why the repository stays small:

Bash What an LFS pointer file looks like

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The 2,000-byte file would be replaced by a 129-byte pointer. With a real 500 MB file the saving is the same 129 bytes. The oid is the SHA-256 of the content, so LFS can find and verify the file on its server.

Things to know before adopting LFS:

  • Decide early. Files committed normally before you tracked them stay in history. Moving them needs git lfs migrate import, which rewrites history (see Amending, Splitting and Rewriting Commits).
  • A server is needed. GitHub, GitLab and Bitbucket support LFS, usually with storage and bandwidth quotas. Self-hosted servers need an LFS endpoint.
  • Clones need LFS installed. Without it, people see pointer files instead of real content.
  • Alternatives: keep binaries out of Git altogether (a package registry or object storage such as S3 with a manifest committed to Git), use partial clones to avoid downloading old blobs (Large Repositories: Shallow, Partial and Sparse), or git-annex for huge datasets.
  • Do not track what you can regenerate. Build outputs belong in .gitignore, not in LFS.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which line in .gitattributes makes Git store LF in the repository for every file it considers text?

    text=auto lets Git detect text files and normalize their line endings to LF when adding them. binary does the opposite: it disables conversion.
  2. 2. What does git ls-files --eol show for a file?

    The output has i/ (index), w/ (working tree) and attr/ columns, for example i/lf w/crlf attr/text eol=crlf.
  3. 3. A release tarball built with git archive should not include the tests/ directory. What do you add?

    export-ignore excludes matching paths from archives made by git archive. The files stay in the repository.
  4. 4. Why must merge.<name>.driver be set in config instead of .gitattributes?

    Executing arbitrary commands from a freshly cloned repository would be a security hole. The attribute only names the driver; the definition is local.
  5. 5. What does Git actually store in a commit for a file tracked by Git LFS?

    The LFS clean filter replaces the content by a pointer (version, oid, size) and uploads the real file to the LFS store when you push.

Summary

  • .gitattributes assigns behaviour to path patterns and is committed, so the whole team shares it; git check-attr shows the result for a path.
  • Store LF in the repository with * text=auto or explicit text eol=...; use eol=crlf only for files that need CRLF everywhere, and git ls-files --eol to inspect. git add --renormalize . applies new rules to existing files.
  • core.autocrlf is a per-user alternative; core.safecrlf blocks lossy conversions. A committed .gitattributes is more reliable.
  • binary and -diff keep binary or generated files out of text diffs; export-ignore removes paths from git archive output.
  • diff=<driver> with textconv shows readable diffs, merge=<driver> customises merging, and filter=<name> runs clean/smudge commands. Driver and filter definitions live in config, not in the attributes file.
  • Git LFS replaces big files by tiny pointers in history and stores the content on a server; adopt it before large files are committed, and remember that it needs installation and server support.

Next: Signing, Credentials and Security