Comparing two documents, and why the diff looks wrong
A diff that marks a whole paragraph when you swapped one word is not broken. It compares lines, and prose is not made of lines. Here is the mechanism, and the fixes.
You put two versions of a document into a comparison tool. You changed one word. The tool marks the whole paragraph, on both sides, and now you are reading a page of colour looking for an edit you already know is there. Nothing has gone wrong. The tool is doing what a line-based comparison does, and a line is not the unit you changed.
Both causes look identical on screen: the unit the tool may mark, and the wrapping.
Why a one-word edit marks a paragraph
A diff was invented for source code, and that explains its behaviour on prose. In code a line is a meaningful unit: a statement, a function signature, a closing brace. Edits in code tend to fill whole lines, because you change an argument and the line holding it changes too. Comparing code line by line is a good approximation of "here is what changed", and good enough that people who work on code stop calling it one.
The algorithm compares lines for equality: it finds the longest run of lines the two files share, in order, and marks everything outside that run. There is no partial credit inside a line, so the smallest thing a line-based comparison can report is one whole line.
Over prose that unit fits badly. A paragraph is what a reader thinks in, and it bears no relation to a line in a file. A plain text paragraph might be one enormous line the editor wraps on screen without inserting anything; exported from a word processor it might be eight lines of seventy characters. Change one word near the start and the line holding it no longer matches its counterpart, so the tool reports the line and you see a paragraph flagged. It is not failing to notice the word. It has no way to see a word.
When both versions were wrapped at different columns
The other cause produces a far worse report from the same mechanism. Copy a paragraph out of a word processor and it arrives wrapped at that program's margin. Copy the same paragraph out of an email and it arrives wrapped at seventy-two or seventy-six characters, or not wrapped at all. Now compare the two.
Almost no line matches another, because no wrap point sits in the same place, and the words between the wrap points are identical. The comparison reports nearly every line as removed and nearly every line as added. The edit you were looking for is one word wide; the report is the size of the document. A rewrite changes the words, a reflow changes only where the lines break, and the giveaway is that both versions hold the same words in the same order.
The fixes, in the order worth trying
None of these needs a different document, only a setting or a few seconds of preparation.
Compare by word, not by line
Most comparison tools have a mode that marks differences word by word rather than line by line. The tool still aligns the two texts; what changes is the unit it is allowed to mark. Swapping committee for board mid-paragraph then produces two small marks and leaves the rest of the paragraph plain, which is the report the reader wanted. This is the mode for prose, and the wrong mode for source code, where a reformatted line or a renamed variable turns the report into a dust of tiny marks with the real change somewhere inside. Code wants lines. Documents want words.
Normalise the text before you compare
Normalising means removing differences that carry no meaning: trim the spaces exporters leave at the end of lines, collapse runs of spaces to one, and unify the line endings.
The endings matter most. Windows text files end a line with two characters, a carriage return and a line feed, written CRLF. macOS and Linux end a line with the line feed alone, LF. A comparison that treats the line break as part of the line it terminates sees every line as different when the only difference is which characters sit at the end of it. That result, every line marked in a document you are sure you did not edit, nearly always means a line-ending or whitespace difference rather than a real one, so fix the endings before you read anything into the report.
The rest of normalising is a decision: whether case and punctuation matter here. Checking wording, they do not. Checking a quotation or a contract clause, they do.
Split on sentences for prose
The most reliable preparation is to put one sentence on each line before comparing. The unit of comparison becomes the unit a reader thinks in, and reflow stops mattering, because a rewrapped paragraph still holds the same sentences in the same order.
The catch is the splitter: abbreviations ending in a full stop split badly, and so do decimals and
version strings. One that breaks a sentence at Dr. hands the comparison two fragments
where the document has one sentence, and the report shows changes on both sides of a sentence nobody
edited.
Ignore whitespace when the wording is all that matters
Most tools carry a switch that ignores whitespace differences, and its correct setting depends on what you are asking. Checking what a draft says: on. Checking how a draft is laid out: off, because then the whitespace is the subject. Ignoring whitespace and ignoring line endings are sometimes one switch and sometimes two, so if every line is still marked, the endings are what is left.
What each mode reports
The same situations through a line-based comparison and a word-based one. The patterns in the right-hand column tell you which mode produced the report in front of you.
| Change | Example | A line diff reports | A word diff reports |
|---|---|---|---|
| Swapped word | committee becomes board, mid-paragraph | The whole paragraph, on both sides | The two words, and nothing else |
| Reflow | same words, lines broken at different columns | Every line of the paragraph, on both sides | Nothing, once the wrapping is normalised |
| CRLF against LF | one file saved on Windows, one on Linux | Every line in the file, on both sides | Every line too, unless the endings were folded first |
| Added sentence | one new sentence inside an existing paragraph | That paragraph, on both sides | The new sentence, and nothing else |
| Renumbered heading | Section 4 becomes Section 5 | That heading and every heading after it | The number inside each heading, one line at a time |
The right-hand column is sharper, and only when the comparison was normalised first. Find the row matching the shape of your report, and it names the mode that made it.
Here is that one-word edit written out both ways. The paragraph is wrapped over two lines, and the edit is the month. Square brackets mark the word removed, curly braces the word added.
line diff
- Renewal falls at the end of March, and the
- invoice goes out a week before that date.
+ Renewal falls at the end of April, and the
+ invoice goes out a week before that date.
word diff
Renewal falls at the end of [March] {April}, and the
invoice goes out a week before that date.
How to read a diff once you have one
Every tool shows a removal and an addition for a single edit, because that is what it found: text present in the first version and absent from the second, and the reverse. Read the pair as one edit, not two, or you will believe you made twice the change you did.
The case that catches people out is the moved paragraph. Cut a section from halfway down a document and paste it near the top, and the comparison has no concept of a move. It reports the paragraph deleted where it was and inserted where it now is, which looks exactly like a paragraph removed and a different one written from scratch. To tell them apart, search for a distinctive phrase from the added block. If it appears again, the text moved. If it appears nowhere else, it is new.
A diff also says nothing about correctness. Two documents that agree line for line are reported as identical whether or not both are right, and a paragraph that is wrong in the first version and still wrong in the second is never marked.
The heading renumbering cascade
Numbered headings have a failure mode of their own. Insert a new section after "Section 4" and it becomes Section 5, the old Section 5 becomes 6, and every heading below moves up by one. Each is a one-character edit, and each is a line the comparison has to mark, so one structural edit near the top makes every heading after it look different. Cross-references move with them, so "see Section 5 for the method" becomes "see Section 6", and those lines are marked too.
A document that has been reorganised therefore produces a diff whose size follows the length of the document rather than the size of the edit. Compare it in stages: the body first with the numbers stripped out, so you see only the wording that changed, then the headings alone as a list, where the renumbering shows up as one visible shift.
When a count beats a diff
If you are comparing two documents to check a claim rather than to see what changed, the word count of each side is a faster first check. Suppose a report is meant to have kept its paragraph on costs. Run a count on both versions. If the older one is around ninety words longer, a paragraph-sized block has gone, and you can find it knowing how big it is. A dropped sentence is usually between fifteen and thirty words, so a difference in that range points somewhere specific.
The count is a screen, not a proof: two versions can hold the same number of words and say different things. But it is cheap, and a difference of any size tells you whether to expect a small edit or a missing block before you start reading.
Where the tools fit. The Diff Checker takes the two versions and marks what changed between them, and it has a word-level mode for the prose case this article is about. The Word Counter gives the count for each side on its own, which is the check to run before you read a line of the comparison, when what you want to know first is whether anything large went missing.
The habit worth keeping
Before you read a comparison, decide which unit you are comparing. Lines for code, where an edit fills a line and the line is the honest unit. Words or sentences for prose, where a line is an accident of wrapping. Then normalise: line endings, trailing spaces, runs of spaces. The order matters less than doing both, but a comparison of one CRLF file against one LF file defeats every other setting you choose.
Then read the report expecting a move to look like a deletion plus an insertion, expecting the first marks you see to be the least interesting ones, and expecting the tool to stay silent about whether either version is right.