Text Diff

Two texts go in. What comes out is which lines changed, and which of the changed lines do not look changed at all. CRLF against LF, a non-breaking space, a curly quote, an accented letter stored as two code points. Every diff tool shows those lines as modified and none of them says why. This one does.

The lines look identical, the diff disagrees

The symptom is familiar. A diff marks a line as changed, red on one side and green on the other, and you stare at both sides and they are the same. A colleague's editor changed the file, a config came back from Word, a snippet was pasted out of Slack, and now the review contains 40 changed lines and one actual change.

The cause is always a byte that no font renders differently. The list below is the one the findings panel above checks for, in the order you are likely to meet them.

The tool above runs every changed line pair through those normalizations and reports which one makes the two sides equal. For a single text rather than a pair, the invisible character detector lists every hidden code point with its position.

When every changed line falls into that list, the summary line says so outright instead of leaving you to compare glyphs.

How to read the result

Paste or drop the original into 01 and the changed text into 02. The diff under it updates while you type. Side by side puts the original left and the changed text right with both sets of line numbers, and a changed line shows the words that differ highlighted inside the line, so a one-word edit in a 200-character line is found without reading the line. Unified is the single-column form with - and +, the one git prints, and it shows the @@ hunk headers once --context is set.

The options are the ones people reach for on the command line, named after the flags they mirror:

• --ignore-whitespace compares lines with every whitespace difference removed, the git diff -w reading: indentation, trailing spaces, tabs against spaces, doubled spaces and line endings all stop counting. In words mode it switches to a word diff that ignores whitespace between words.
• --ignore-case treats Foo and foo as equal.
• --ignore-eol treats CRLF and LF as the same line ending and forgives a missing newline after the last line. The CRLF half is git diff --ignore-cr-at-eol, without touching any other whitespace.
• --context N shows only the changed hunks with N lines around each, the -U N of diff and git. Empty shows every line, which is the better view for short texts.
• lines, words, chars set the unit. Lines for code and configs. Words for prose, where a reflowed paragraph would otherwise read as deleted and rewritten. Chars for keys, hashes and short strings.

The findings panel works on the raw text regardless of the options, so switching --ignore-whitespace on hides the trailing spaces from the diff and the panel still tells you they are there. Copy patch puts a unified diff on the clipboard with a/ and b/ headers, which git apply and patch -p1 take as they are. If you dropped a file, its name stands in both headers, so git apply knows which file to patch. The patch carries the --context setting with a floor of one line, because git apply refuses zero-context hunks unless it is run with --unidiff-zero. We use that button mostly for the last metre of a review: fix the text in the browser, copy the patch, git apply it in the checkout. It costs a round trip through the clipboard, and for more than a handful of hunks an editor with git integration is the better tool.

Large inputs are fine. 30,000 lines on each side diff in well under a second. Word-level highlighting runs on the changed line pairs only, and the display stops at 4,000 rows unless --context narrows it. Two texts that have nothing in common are the one slow case, because the algorithm then has to prove there is no shared line, and the tool gives up after two seconds with a message rather than freezing the tab.

How a line diff works

A diff does not compare line 5 with line 5. It looks for the longest sequence of lines that appear in both texts in the same order, and everything that is not part of that sequence is a deletion or an insertion. The algorithm that finds it, Myers' 1986 shortest-edit-script algorithm, is what diff, git and the diff npm package behind this tool run. Its running time grows with the length of the files times the size of the change, which is why a one-line edit in a huge file is instant and two unrelated files of the same length are slow.

That model explains the two things people find odd about diffs. There is no such thing as a "changed" line in the output, only a removed line next to an added one, and it is the viewer that pairs them and highlights the words inside. And when a block of lines moves, it is shown as removed in one place and added in the other, because the algorithm has no concept of movement. git diff --color-moved paints such blocks in a different colour but does not change the output.

The unified format that carries the result is small enough to read in a minute:

--- a/config.toml
+++ b/config.toml
@@ -1,4 +1,5 @@
 [server]
-port = 8080
+port = 8443
 workers = 4
+tls = true
 timeout = 30

--- names the old file, +++ the new one. The hunk header says where the block sits: line 1 of the old file, four lines long, and line 1 of the new file, five lines long. Then context with a leading space, removals with -, additions with +. The context is not decoration. It is what lets patch find the right spot after the surrounding lines have moved, and three lines of it is the default everywhere because that is usually enough to be unambiguous and rarely enough to collide with a second change.

Why git shows the whole file as changed

You change one line, git diff lists every line of the file, and git diff --stat reports a rewrite. Nine times out of ten the editor saved the file with CRLF where the repository has LF, or the other way round. Each line now differs in its last byte, which is exactly the kind of change a diff reports and a screen does not render.

$ git diff --stat
 config.toml | 214 ++++++++++++++++--------------
 1 file changed, 107 insertions(+), 107 deletions(-)

$ git diff --ignore-cr-at-eol --stat
 config.toml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

$ printf "* text=auto\n" > .gitattributes
$ git add --renormalize .
$ git diff --cached --stat
 config.toml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

The diagnosis is one flag: git diff --ignore-cr-at-eol shows the real change if line endings were the cause, and git diff -w shows it if an auto-formatter reindented everything. file config.toml prints "with CRLF line terminators" when that is the state on disk. The fix that lasts is a .gitattributes with * text=auto, committed, followed by git add --renormalize . once, so that git stores LF in the repository and every checkout gets its platform's ending. core.autocrlf does a similar job per machine, and that is its weakness. It has to be set on every clone, and the one developer who did not set it reintroduces the problem with their next commit.

Two more flags worth knowing. git diff --word-diff shows changes inside lines as [-old-]{+new+} instead of whole lines, the better view for prose and Markdown. And git diff --no-index a.txt b.txt diffs two arbitrary files with git's engine, colours and flags, whether or not they are tracked or even inside a repository.

Comparing files in the terminal

For anything that already lives on disk, the command line is faster than any website, and the flags map one to one onto the options above.

cmp is the one people forget. It answers "are these two files the same" in a byte comparison and, with -l, lists every differing byte offset, which is the quickest route to a stray NBSP when the diff output itself looks identical on both sides. The process substitution in the sort and xxd rows is bash and zsh syntax. In plain sh, write the intermediate files out.

JSON and YAML: text diff or semantic diff

A text diff of two JSON files reports a reordered key as one deletion and one insertion, and a reindented file as changed from top to bottom, while neither changes the data. For configuration that is usually what you want to know anyway, because the file is what gets deployed. For an API response or an export, what you want is the semantic answer: which values differ.

The cheap route is to normalize first and diff second. jq -S . sorts keys recursively and pretty-prints with two spaces, so diff <(jq -S . a.json) <(jq -S . b.json) shows only real differences. The JSON formatter on this site does the same in the browser with its sort-keys option: format both files with it, paste the results into 01 and 02 above, and the key order stops being a difference. For YAML, yq -P 'sort_keys(..)' does the equivalent.

A semantic diff is not always the better one. Two YAML files that load to the same data can still behave differently, because one quotes NO and the other does not (the Norway problem), and a textual diff is the one that shows you that line.

diff, patch and merge

diff lists the changes between two versions. patch replays that list onto a file, using the context lines to find the right spot after line numbers have shifted. merge is the three-way case, two versions with a common ancestor, conflicting wherever both touched the same lines. git apply is patch, git merge is merge.

Diff and patch questions