How Optic's CLI renders API diffs in the browser without a server

Nicholas Lim 2023-10-04

At Optic we help teams track, test and review API changes before they ship. When a developer opens a PR that changes the API, Optic visualizes the diff and runs breaking change detection and design checks. For most developers, there’s a high bar for trying out a new tool in CI, so we wanted to build a way to discover and to try Optic before adding anything to CI.

The first thing we considered was bundling our UI rendering code into the CLI so that it can generate valid HTML and serve a static page with changes. A downside to this approach is the CLI needs to know how to render the changelog. This means including all the react libraries, component libraries and rendering code in our CLI bundle which would greatly increase the size of our CLI. This would also mean the CLI and our app would be running different versions of the rendering code, meaning users would get out of date experiences on their first use of the app.

We settled on sending the diff data from our CLI and rendering it with our web app on the client’s browser. This avoids ever sending the data to Optic’s backend servers.

Passing data to the browser

For Optic to render a diff, we need the before and after specs and the results of our breaking change and design checks. Usually, you would store the data on the server and the browser would then request that data. This is how Optic Cloud works, but requires an account, adding friction to the first user experience.

We could temporarily persist the data in a key-value store on our servers. However we didn’t want to persist this data because we value our user’s privacy. Instead of storing the data, we instead pass the data directly into the browser via the URL. We do this by:

• stringifying the JSON data (before, after specs and rule results)
• compressing the string (we’re using brotli (opens in a new tab)
• encoding the string to base64

import zlib from 'node:zlib';
 
// Data that we compress for a diff
const diffData = {...};
const stringified = JSON.stringify(diffData);
const compressed = zlib.brotliCompressSync(Buffer.from(stringified));
const encoded = Buffer.from(compressed).toString('base64');

On the client, we reverse the process:

// Since we're in the browser context, we need to import a client side brotli implementation
import decompress from 'brotli/decompress';
 
// Data that we compress for a diff
const hash = window.location.hash;
const compressed = Buffer.from(hash, 'base64');
const decompressed = decompress(compressed);
// Ready to use in the browser!
const data = JSON.parse(decompressed);

The result sets can be large (we’ve seen them contain 100,000 characters), even after compression. Web browsers implement different limits when to comes to the length of pasted URLs, leading to confusing situations where a URL seemingly doesn’t work.

Fortunately, we can still open these URLs directly using the command line. Here we’re using the NPM package open (opens in a new tab)) to open the URL in a browser.

import open from 'open';
import zlib from 'node:zlib';
 
// Data that we compress for a diff
const diffData = {...};
const stringified = JSON.stringify(diffData);
const compressed = zlib.brotliCompressSync(Buffer.from(stringified));
const encoded = Buffer.from(compressed).toString('base64');
 
// Opens up the url using the npm open package
await open(`https://app.useoptic.com/cli/diff#${compressedData}`);

This doesn’t work quite as expected on windows, instead we get an error ENAMETOOLONG - indicating the name is too long to open! We can work around this by creating a local html file in a tmp directory with a redirect to our web app and then call open on that locally created file instead.

import fs from 'node:fs/promises';
import open from 'open';
import os from 'os';
import path from 'path';
 
const tmpDirectory = os.tmpdir();
 
// A open function that handles ENAMETOOLONG errors from windows
export const openUrl = async (url: string) => {
  try {
    await open(url);
  } catch (e) {
    if (e instanceof Error && /ENAMETOOLONG/i.test(e.message)) {
      const tmpHtmlPath = path.join(tmpDirectory, 'optic', 'tmp-web.html');
      await fs.mkdir(path.dirname(tmpHtmlPath), { recursive: true });
 
      await fs.writeFile(
        tmpHtmlPath,
        `<!DOCTYPE html><html><body><script type="text/javascript">window.location.replace("${url}")</script></body></html>`
      );
 
      await open(tmpHtmlPath);
    }
  }
};

Finally, you’ll notice that the generated URL places the data behind the fragment identifier (opens in a new tab), #. The fragment identifier is interpreted by the client — the compressed data is never sent to the server. This greatly reduces the amount of network traffic that is needed to serve this page since none of the OpenAPI data and rule data leave your computer.

Try it out

Now that we understand how it works, let’s see it in action! The following command runs Optic against our example repo:

npx @useoptic/optic diff --web --check \
https://raw.githubusercontent.com/opticdev/bookstore-example/dba6a34e6d5473a89de0b64dc49791affe1ceb7f/openapi.yml \
https://raw.githubusercontent.com/opticdev/bookstore-example/ac5f3c55a6f7f27c482a557563686d0328dafb55/openapi.yml

Optic will generate the diffs between the OpenAPI spec at version dba6a3 (opens in a new tab) and version ac5f3c (opens in a new tab), opening the results in a web browser. By default Optic will check for breaking changes, and can be configured to run additional design rules.

Learn how to set up Optic in CI by following our instructions here (opens in a new tab). Optic also lets you set design standards and comes with Spectral (opens in a new tab) compatibility out of the box, learn more here (opens in a new tab).