With previous posts we have learned to produce an OpenAPI specification containing all OpenAPI specification subtleties. Some specification files may become quite large or may contain elements which could be reused in other APIs. Splitting a specification file will help to keep it maintainable by creating smaller files and also help to ensure consistency throughout APIs by sharing common elements.

Writing OpenAPI (fka Swagger) Specification tutorial

This tutorial is composed of several posts:

In previous parts we’ve learned to create highly accurate API description which can become quite large or may contain elements that can be reused, in this eighth part we’ll learn how to split an OpenAPI specification file into smaller and reusable elements.

All tutorial’s files are available on gist.

If you’re a bit lost in the specification, take a look at my [visual documentation:

JSON Pointers

In part 3 - Simplifying spefication file we have learned how to simplify the specification by creating reusable elements. In the example below, the Person definition is defined once in definitions and used as

  • A body parameter in POST /persons
  • A response schema in GET /persons/{username}
  • A sub-elements in Persons definition

To use the Person definition in these different places, we use a JSON Pointer (defined by RFC6901):

This pointer describes a path in the document, pointing to Person in definitions which is as the root (#) of the current document.

But JSON pointers are not only meant to point something within the current document, they can be used to reference something in another document.

Basic splitting

Let’s see how we can split the file we created in part 3

Referencing a local file

We can create a person.yaml file containing the Person definition:

Then we can remove the Person definition in definitions and replace all existing references to person #definitions/Person by a reference to Person in the person.yaml file:

Editing splitted local files with the online editor

Tools will look for the referenced file (person.yaml) in the same directory as the file containing the reference. But when you use the online editor it does not make sense, such local reference cannot be resolved.

Fortunately the editor propose a configuration allowing to set a server to resolve these local references. This configuration is accessible in the Preferences->Preferences menu:

The Pointer Resolution Base Path configuration is on the bottom of the configuration screen:

All you need to do is put the referenced yaml files into a web server (with CORS activated) and modify the editor’s configuration to point this server.

http-server a lightweight web server

You can use http-server a lightweight node.js web server:

npm install --global http-server

You now can start a web service on any folder:

http-server --cors path/to/yaml/folder

or

cd path/to/yaml/folder
http-server --cors .

By default, http-server listens on 8080 port, files will be accessble through http://localhost:8080/<path to file within folder>.

The --cors flag is used to activate CORS directive and allow XHR request to this local webserver from the online editor page. Without CORS activated, the editor will not be able to download files.

Modifying online editor configuration

Once the web server is started you can modifiy the editor to set the URL to http://localhost:8080/:

Files referenced with $ref: <filename>#/example will be downloaded from http://localhost:8080/<filename>.

Once this is done, you may need to do a force refresh (clean cache) of the editor’s the page to get the green bar.

Cache

When you add a reference to a new file (that was not already reference), this new file may not be downloaded automatically resulting in errors (Reference could not be resolved: newfile.yaml). You may need to refresh the editor’s page with cache cleaning to solve this error.

Folders

You’re under no obligation to put all sub-files on a “root” level, you can store them in differents sub-folders.

Reference to a file in a folder

Let’s move the person.yaml file into a sub-folder folder, the new reference will be like this:

File referencing a file from an upper folder

You can also reference a file outside the current folder. Let’s create a persons.yaml file into a another-folder folder:

This file reference the person.yaml file using .. to get to the upper level.

To use persons.yaml, we proceed just like with person.yaml. We remove the Persons definition from the main file and we replace its reference #/definitions/Persons by another-folder/persons.yaml#Persons.

Here’s the full file with all definitions externalized (the definitions section has been removed):

Referencing a remote files

As you may have guess while modifying the references in the specification and editor configuration, it is also possible to reference a remote file.

Remote files

All we need to do is to put the full file’s URL in the reference:

$ref: https://myserver.com/mypath/myfile.yaml#/example

Remember that the server MUST have CORS activated to allow the editor to download the file.

We have launched a web server on 8080 port, so all we have to do is add http://localhost:8080/ to the references to Person:

Remote files containing local references

What happen if the remote file reference a local file? (Main file -> Remote file -> Local file).

The parser will seek this “local” file on the remote server providing the remote file.

If we replace the local to Persons by a remote reference…

… the person.yaml file referenced “locally” in the persons.yaml file will be loaded from http://localhost:8080 which has served the persons.yaml.

Remote files containing remote references

If a remote file contains a remote reference (Main file -> Remote file -> Remote file ), it will be resolved like in 2.4.1 Remote files.

We can replace the person.yaml file local reference by a remote reference in persons.yaml:

Multiple items in a single sub-file

We have put Person and Persons definitions in separate files: this is not an obligation. You can put more than one item in a single file, you only need to use the right JSON Pointer.

Person and Person in a single file

If we concatenate person.yaml and persons.yaml into a single file called definitions.yaml:

Note that in Persons the reference to Person is now #/Person.

In the main file we only need to change the filename when reference these two definitions:

Organizing content in a sub-file

Within the sub-file you can organize the content as you wish. Here Person is in SomeDefinitions and Persons is in OtherDefinitions:

Note that in Persons the reference to Person is now #/SomeDefinitions/Person.

In the main file we have to modify the path to get the item in the sub-file for these two definitions:

Definitions, Responses, Parameters

What we have done with Person and Persons definitions can be done with any reusable items (i.e. using $ref) such as responses and parameters in the Open API Specification file.

OpenAPI chainsaw massacre

Thanks to Mohsen Azimi’s post I’ve discovered that using sub-files for definitions, responses and parameters which obviously use $ref JSON Pointers is not the only way of using sub-files. You can use sub-files for almost anything in the specification.

We will split the huge file created in previous post about documentation.

Let the chainsaw massacre begin

Let’s start with the info section:

We can put its whole content in a file called info.yaml:

And reference it just like this in info:

The wizard of resolution

We can do the same thing with paths, definitions, responses and parameters: copy the section content in a sub-file (paths.yaml, definitions.yaml, responses.yaml and parameters.yaml) and reference it in the main file:

If you remember previous posts, these sections references each other in many ways, for example in paths.yaml:

The username parameter is no longer in the main file and is not in the paths.yaml file but in the parameters.yaml file:

How could the main file be considered valid in the editor (or in any tool parsing the file)? It’s simply because the sub-files are loaded and then the whole content (file + sub-files) is validated.

A less rough split

We can use JSON pointers for almost anything in the specification as as long as the $ref JSON pointer reference something corresponding to the expected object (or value) in the OpenAPI specification.

Referencing object in custom structure

And as seen earlier in this post we can put many items in a sub-file.

The documentation.yaml file contains the data for externalDocs and tags (note the custom structure on line 1 and 6):

These data are referenced this way in the main file:

The security.yaml file contains the data for securityDefinitions and security (note the custom structure on line 20):

Referencing a string or a simple list

It also work for simpler value like a string or a list of string. The schema, host and basepath values can be moved into a single file security.yaml (note the custom structure on line 3 and 4):

And referenced like this

Reusing a value

As long as the API consumes and produces the same media types, we define a single value in the mediatypes.yaml file:

And then we reference this single value in both produces and consumes:

A smarter split

The API we built with the previous parts could be divided in four parts:

  • Common items like headers, media types, security, definitions, parameters and responses
  • Persons operations, parameters, responses and definitions
  • Legacy operations
  • Images operations

We can create 4 sub-files and reference them from a main file.

commons.yaml

In the commons.yaml file we put every items that can be reused across other files:

images.yaml

In the images.yaml file we put both /images and /images/{id} paths informations:

Note that all references to common items point to commons.yaml.

legacy.yaml

Same for the /js-less-consumer-persons path we put in legacy.yaml file in js-less-consumer-persons:

persons.yaml

All persons items go in the persons.yaml:

  • each path go in its own named section (like persons-username for /persons/{username})
  • all persons specific definitions goes in definitions (and can be referenced with #definition/name)
  • all persons specific responses goes in responses (and can be referenced with #responses/name)
  • all persons specific parameters goes in parameters (and can be referenced with #parameters/name)

full main file

Here’s the full main file where we reference the 4 sub-files:

Valid sub-files

If we put the last persons.yaml file content in the editor, it ends with these errors:

Splitting a huge Open API Specification file to keep it maintainable is a great idea but if you cannot validate sub-files against the specification, you gain almost nothing.

Making commons.yaml valid

If we put the commons.yaml file created earlier in the editor we get these errors:

  • Missing required property: swagger
  • Missing required property: info
  • Missing required property: paths
  • Additional properties not allowed: defaultHeaders,defaultMediatypes,defaultSecurity

Let’s see how we can fix these errors.

Missing swagger property

We just need to add this line on file’s top:

Missing info property

We add an short info section after swagger:

Missing paths property

As we do not define any path in this file we just need to add an empty paths section:

Property defaultSecurity not allowed

As the defaultSecurity correspond to the OpenAPI security section we just need to rename it:

Invalid commons.yaml file:

Valid commons.yaml file:

Property defaultMediatypes not allowed

defaultMediaType is not a valid OpenAPI property:

We will use produces and consumes to define the default media types. But as we want to define a single set of media type we will use this trick:

In the future if we want to define different sets of media types for produces and consumes, we’ll be ready.

Property defaultHeaders not allowed

defaultHeaders is not a valid OpenAPI property:

We will use a dummy response definition DefaultHeaders to declare these default headers:

Of course we need to update common responses to use these headers:

Making persons.yaml valid

Just like with commons.yaml, if we put persons.yaml content in the editor we get some errors:

  • Missing required property: swagger
  • Missing required property: info
  • Reference could not be resolved: commons.yaml#/defaultHeaders
  • Missing required property: paths
  • Additional properties not allowed: persons-username-collecting-items,persons-username-friends,persons-username,persons

Missing swagger and info properties

We just add swagger and info like for commons.yaml:

commons.yaml#/defaultHeaders reference could not be resolved

As the commons.yaml structure has been modified concerning default headers we need to replace these references:

by this one:

Missing paths and additional properties not allowed

We have defined custom properties for each path relative to persons operations, therefore there is no paths section and our custom properties (like persons) are not allowed by the OpenAPI specification.

We need to add a paths section and put all our path in it using the correct path value as key:

New errors: Security definition could not be resolved

Once this is done, 2 new errors appear:

  • Security definition could not be resolved: OauthSecurity
  • Security definition could not be resolved: LegacySecurity

Now that we have a valid structure, the paths have been parsed and the parser detected missing security definitions. To solve this error, we add these definitions by referencing the commons.yaml file:

images.yaml and legacy.yaml

For images.yaml and legacy.yaml we do exactly the same things as we’ve done with persons.yaml.

Here the images.yaml file modified (partial view):

Here the legacy.yaml file modified (partial view):

Updating the main file

Now that we have modified all sub-files, if we try to edit the main file we get these errors

  • Reference could not be resolved: commons.yaml#/defaultMediatypes
  • Reference could not be resolved: commons.yaml#/defaultSecurity
  • Reference could not be resolved: persons.yaml#/persons
  • Reference could not be resolved: images.yaml#/images-imageId
  • Reference could not be resolved: persons.yaml#/persons-username
  • Reference could not be resolved: persons.yaml#/persons-username-friends
  • Reference could not be resolved: persons.yaml#/persons-username-collecting-items
  • Reference could not be resolved: images.yaml#/images
  • Reference could not be resolved: legacy.yaml#/js-less-consumer-persons

These errors are of 2 types:

  • those due to the commons.yaml structure modification
  • and those due to the persons.yaml, images.yaml and legacy.yaml paths structure modifications

Modifiying commons.yaml references

Before:

After:

Modifying paths references

This is the trickyest part of this tutorial. To reference the /persons path in persons.yaml we used persons.yaml#/persons:

But persons.yaml structure has changed from :

to:

The new reference should be something like persons.yaml#/paths//persons but if we try it, the editor shows an error Reference could not be resolved: persons.yaml#/paths//persons. The / in /persons needs to be escaped, how can we do that?

Evaluation of each reference token begins by decoding any escaped character sequence. This is performed by first transforming any occurrence of the sequence ‘~1’ to ‘/’ RFC6901

Before modification:

After modification (all / in reference name have been replaced by ~1):

And now the main file and all its sub-files are considered valid by the editor.

Conclusion

You are now ready to split any OpenAPI specification file into valid sub-files in any possible ways. In this 8th part you’ve learned how to use JSON pointers in almost any places in an OpenAPI specification to references items from other files and how to create valid sub-files containing partial informations. In the next final part (at last) we will learn how to extend the OpenAPI specification (coming soon).