Git Can Break Automated Tests

2022-03-21

Did you know that git will sometimes silently change the line endings on files? I learned this yesterday, and now you can learn from my confusion.

Background

I recently contributed to a go tool that will format long lines. I was happy with my contribution (more on that in another post maybe), but unfortunately I broke the build on Windows. Someone else quickly contributed a fix for my mistake, and I asked the maintainer whether they would welcome a pull request to do cross-platform testing using GitHub Actions. That’s where the real story begins for us.

A Mystery

When I set up the GitHub action, I discovered that the current tests failed on Windows. To be clear, I don’t mean an edge-case failure. The test failures suggested that the tool shouldn’t work at all on Windows. This confused me since the tool clearly works on Windows. (People are using it there—you can tell from the immediate bug report that I just mentioned. Yet nobody had reported this complete failure.)

When I dug into the log of the failed tests, I quickly saw that the failures were about line endings. The tool uses LF for line endings, and the tests expected CRLF—but only on Windows. At first, however, I couldn’t figure out why the tests on Windows expected those line endings.

A bit more background: the failing tests compared the output of the tool to golden files. I had the golden files in my repository (on macOS), and I could confirm that they had LF rather than CRLF line endings. Nevertheless, the tests were failing, and the output of the failed tests made clear that the line endings were to blame. Why were the line endings in the testing environment different from my line endings but only on Windows?

Blame Git

As my title suggests, git was the culprit. git has a configuration setting core.autocrlf that controls how line endings are handled in a repository. (Actually, git has several configuration settings that affect line endings. There’s also core.eol and core.safecrlf. But core.autocrlf was to blame here.) On Windows, the default installation option for core.autocrlf is true, but for macOS and Linux, the default is false. When you test on Windows using GitHub Actions, the test environment has core.autocrlf set to true. As a result, when files are checked out by git, git will automatically change LF line endings into CRLF line endings. If some of those files are golden files, tests may fail.

For the record, GitHub is aware of this situation, but they feel that they are doing the right thing. See this discussion. (To be clear, they may be absolutely right. The result was a pain for me, but that doesn’t mean GitHub is wrong.)

A Solution

Happily, you can fix this problem pretty easily. All you need to do is create a .gitattributes file and tell git not to tamper with the line endings of the test files. (You can also globally change how git handles line endings, but that may not be very kind to people who work on your project on Windows. That should probably be a group decision.)

In my case, the fix was as simple as this .gitattributes file. The tests pass, and all is well.

_fixtures/* text eol=lf

One More Warning