Document existing APIs

Writing an OpenAPI specification for a large existing API is tedious, exhausting and error-prone. The up-front costs prevent many teams from getting the benefits of adopting OpenAPI and working API-first. Optic can help you document an existing API in less than an hour -- just show the tool traffic, and it will write the boilerplate OpenAPI for you.

Documenting an API from traffic

1. Make an empty OpenAPI spec

First create an empty OpenAPI specification. Optic has a helper command that can output a valid specification to start from, but there is no reason you can not write this yourself or start with an OpenAPI specification you have written by hand.

optic oas new > openapi.yaml

2. Capturing traffic

Optic learns how your API works by capturing and analysing real-traffic. Point Optic at a version of your API running locally, a development environment, or even production. Optic starts a local transparent proxy on localhost:8000 that intercepts and analyses all the traffic to your API that originates from your local machine. None of this traffic leaves your machine because the CLI does its work locally.

First time HTTPs captures

If you want to capture traffic over HTTPs you will need to run this command to give Optic the ability to decode TLS traffic sent through its proxy.

optic oas setup-tls

Capturing traffic to localhost:

optic oas capture openapi.yaml localhost:4000

Capturing traffic to your development environment:

optic oas capture openapi.yaml https://developer123.api.example.dev

Capturing traffic to a third-party API (just for fun):

optic oas capture openapi.yaml https://api.twitter.com

Client Compatibility

Most clients work out of the box (✅) or after minor tweaks to their settings (🟡)

HTTP/s Client	Compatibility	Steps to capture traffic
Chrome	✅
Safari	✅
Postman	✅ (defaults)	Tell Postman to use System proxy
Insomnia	✅ (defaults)	Tell Insomnia to use System proxy
Paw	✅ (defaults)	Tell Paw to use System proxy
Httppie	🟡	Tell httppie to use the Optic Proxy
curl	🟡	set `HTTP_PROXY` and `HTTPS_PROXY` in the environment before running curl commands
newman	🟡	set `HTTP_PROXY` and `HTTPS_PROXY` in the environment before running curl commands
Firefox	🟡	Firefox detects the MITM proxy and blocks it by default. 1) Navigate Firefox to the API host you want to capture from and "Accept" the Security Risk. For `localhost` captures set the netowrk option `network.proxy.allow_hijacking_localhost` to `true`

Common issue: TLS traffic is not being captured?

The Optic CLI will not be able to learn from TLS encrypted traffic unless you trust its certificate. The command will guide you through the instructions for trusting the certificate on your operating system:

This allows the Optic proxy to man-in-the-middle your TLS traffic when the capture command is running. An intruder would need pretty deep access to your machine to exploit this. Our code is open source and we encourage people working with sensitive APIs to use development environments.

optic oas setup-tls

Capturing test traffic

Optic can intercept traffic from your tests, provided they are sent over the network. When you run capture with --command flag the proxy will start and then run the command provided with HTTP_PROXY and HTTPS_PROXY injected into the environment:

optic oas capture openapi.yaml <target-url> --command <your-test-command>

If your tests do not run over the network, many frameworks have HAR logging libraries you can use to capture a HAR and pass it into Optic directly verify --har logs.har.

3. Document your API Operations

Run Optic verify to compare the captured traffic to your OpenAPI specification. The first time you run verify you should see some "Undocumented" errors. This happens because your API is responding to API requests that have not been documented.

optic oas verify openapi.yaml

 » Verifying API Behavior...

 [ Undocumented ]    get  /todos/{todoId}/status
 [ Undocumented ]  patch  /todos/{todoId}/status
 [ Undocumented ]   post  /todos/{todoId}
 [ Undocumented ]  patch  /user/profile/settings

   (use "--document all" to to document these paths)
   (use "--document "[method path], ..."" to to document one or more paths)

You could resolve these "Undocumented" errors by manually documenting these operations with OpenAPI, but Optic can do it faster and more accurately.

Document a single operation by adding the --document "[method] /path " flag

Pick one of the operations on the list of Undocumented paths and run this command to add it:

optic oas verify openapi.yaml --document "get /todos/{todoId}/status"

Now open the OpenAPI file in your editor. You will see Optic documented the operation.

Document all the operations at once --document all flag

You can rapidly all the observed operations by passing --document all, but make sure you want everything Optic observed documented before you run this command. You may need to add some operations one-at-a-time first, or ignore certain paths to get the results you are looking for:

optic oas verify openapi.yaml --document all

Do the paths for 'Undocumented' operations look wrong? Optic tries its best to guess which path parameters are constants and which are IDs/variables. If your API is mostly-restful it should be able to guess most of them correctly. You should not run --document all until the list of undocumented operations you see when you run optic oas verify looks correct to you. If Optic guessed something wrong, just add that path pattern manually. As you provide Optic with more priors, it is able to guess other path components with more accuracy.

Should Optic being ignoring some of the paths? Many servers have endpoints you would not wish to document i.e. health checks, images, javascript bundles, internal endpoints, etc. Verify will skip any paths or globs you list in x-optic-path-ignore

openapi: 3.1.0
x-optic-path-ignore:
  - "**/*.+(ico|png|jpeg|jpg|gif)"
  - "/health-check"
info:
  title: Our Service
  version: 1.0.0

4. Edit and repeat

Unlike every other OpenAPI generator, Optic is not one-shot. That means you can use the generator over and over again -- whenever you add new API operations or want to track an API change. You can even make major changes to the structure of the document and the tool will continue to work.

Many teams use Optic to keep very complicated OpenAPI specs up-to-date. Optic's core is robust and flexible, allowing you to organize your OpenAPI files however you like:

you can change anything Optic wrote and it will not lose those changes when you generate new sections of OpenAPI.
manually written description and summary fields will never be lost when Optic documents new operations
you can use $ref references and Optic will work properly (updating the correct component if the API changes)
split your OpenAPI spec into multiple files (one team has 1200 files). Optic will always update in the right place.

Need help?

Hopefully the tools just-work, but we are around to help if you have trouble. We want to make these tools easy to use and powerful -- please ask us for help if you have trouble so we can improve the tools for everyone.

Documenting an API from traffic

1. Make an empty OpenAPI spec​

2. Capturing traffic​

Client Compatibility​

Capturing test traffic​

3. Document your API Operations​

4. Edit and repeat​

Need help?​

Next up: Tracking API changes with OpenAPI