Your API needs a lockfile

Aidan Cunniffe 2023-05-11

APIs are a contract. When you publish a new endpoint, you are making a new promise to consumers. Every time we change the code we must be careful not to break that promise. With all the abstractions in a modern backend codebase it is easy to break a promise without even realizing it — changing one line of code can easily change a dozen API endpoints. We can’t keep our promises if we don’t know what they are, and for most APIs, they have not been written down.

Think about how you would write a test that checks for breaking API changes, like renaming a field in an API response.

Very quickly you realize that you need to encode information about the API’s previous behavior in your project somewhere. You could write verbose tests that assert each endpoint works as designed, or you might write a snapshot test that compares the current response for each endpoint to the last saved one. These sorts of tests are tedious to write, fragile, and can only be used to compare each version of your code to the version directly before it. You can’t arbitrarily test for changes between two branches or two git tags ie production Δ v2.0.0.

What we need is something like a lockfile for our API’s behavior. That would allow us to:

• Review API Diffs (the behavior changes) at the same time as Code Review
• Test for breaking API changes in CI and when releasing
• Refactor the backend safely. Lots of code can change, but the lockfile should stay the same

Using OpenAPI as your API lockfile

Most developers think of OpenAPI as a documentation artifact; a large YAML file that you have to write by hand and struggle to keep in sync with your code. That’s how it used to be, but with so many companies moving from monoliths to microservices, coordinating API changes across teams has become a huge challenge. Large enterprises have invested considerable resources into making accurate OpenAPI specifications an artifact of their builds. I’ve gotten to see some of this impressive engineering up-close. In the open source world, API frameworks like FastAPI have set the standard with amazing first-class support for generating specs. The world is moving this way, fast.

Once an accurate OpenAPI spec is checked into your repo, you start to see API diffs during code review. Just seeing the API changes spelled out helps developers ship better APIs. Accidents go away, and teams start giving each other better API reviews. Better designs shake out once everyone can see the API and talk about how it’s changing.

The same API diffs that power those changelogs can be used to write CI tests that constrain how the API is allowed to change. These tests are very effective at preventing breaking changes, making sure a consistent versioning scheme is followed each release, and improving the design with a style guide.

I built an open source tool that can run tests on your API changes. It’s called Optic — I’d also suggest looking at openapi-diff (opens in a new tab) or oasdiff (opens in a new tab).

First, we load the specifications from Git, then compute a semantic diff between the two versions structural elements like order, indentation, and other styles do not affect the diff. You end up with a stream of diffs that our test fixtures can hook into.

# compare the spec to the version of it on 'main' branch
optic diff openapi.yml --base main

Each test is triggered by a specific kind of change, like response.property.changed or operation.parameter.added :

responseAssertions.property.changed((before, after) => {
  if (before.value.required && !after.value.required) {
    throw new RuleError({
      message: `cannot make required response property '${after.value.key}' optional. This is a breaking change.`,
    });
  }
});

When tools like this are run in CI — developers find and fix breaking changes before they ship and save themselves and their consumers a lot of time and headaches.

With APIs becoming the main dependency of many projects, I think we’re going to see more and more teams writing tests about how those APIs change over time. It’s not enough to test how each build of our software behaves, we need to understand its history and the promises we’ve made to our colleagues. Even if you use OpenAPI for nothing else — you should use it as a lockfile, so that you change your APIs with intention. APIs will continue to be the joint where all our software comes together. I expect we’ll see similar test practices for protocols like GraphQL, gRPC, and whatever comes next.