Git Can Break Automated Tests
Did you know that
git will sometimes silently change the line endings on
files? I learned this yesterday, and now you can learn from my confusion.
I recently contributed to a go tool that will format long lines. I was happy with my contribution (more on that in another post maybe), but unfortunately I broke the build on Windows. Someone else quickly contributed a fix for my mistake, and I asked the maintainer whether they would welcome a pull request to do cross-platform testing using GitHub Actions. That’s where the real story begins for us.
When I set up the GitHub action, I discovered that the current tests failed on Windows. To be clear, I don’t mean an edge-case failure. The test failures suggested that the tool shouldn’t work at all on Windows. This confused me since the tool clearly works on Windows. (People are using it there—you can tell from the immediate bug report that I just mentioned. Yet nobody had reported this complete failure.)
When I dug into the log of the failed tests, I quickly saw that the failures were about line endings. The tool uses LF for line endings, and the tests expected CRLF—but only on Windows. At first, however, I couldn’t figure out why the tests on Windows expected those line endings.
A bit more background: the failing tests compared the output of the tool to golden files. I had the golden files in my repository (on macOS), and I could confirm that they had LF rather than CRLF line endings. Nevertheless, the tests were failing, and the output of the failed tests made clear that the line endings were to blame. Why were the line endings in the testing environment different from my line endings but only on Windows?
As my title suggests,
git was the culprit.
git has a configuration
core.autocrlf that controls how line endings are
handled in a repository. (Actually,
git has several configuration settings
that affect line endings. There’s also
core.autocrlf was to blame here.) On Windows, the default installation
true, but for macOS and Linux, the default is
false. When you test on Windows using GitHub Actions, the test environment
core.autocrlf set to
true. As a result, when files are checked out by
git will automatically change LF line endings into CRLF line endings.
If some of those files are golden files, tests may fail.
For the record, GitHub is aware of this situation, but they feel that they are doing the right thing. See this discussion and this discussion. (To be clear, they may be absolutely right. The result was a pain for me, but that doesn’t mean GitHub is wrong.)
Happily, you can fix this problem pretty easily. All you need to do is create
.gitattributes file and tell
git not to tamper with the line endings of
the test files. (You can also globally change how
git handles line endings,
but that may not be very kind to people who work on your project on Windows.
That should probably be a group decision.)
In my case, the fix was as simple as this
.gitattributes file. The tests
pass, and all is well.
_fixtures/* text eol=lf
One More Warning
If you add a
.gitattributes file, people who work with the repository on
Windows may have one other problem.
git may claim that there have been
changes to files that the developers have not changed. That is simply
telling them that the line endings have changed on some files. There’s a fix
for that too. Check out this link for details.