JQ and OpenAPI Series - Part 1

Using JQ to extract data from OpenAPI files

By Arnaud Lauret, January 15, 2020

Ever wanted to quickly find, extract or modify data coming from some JSON documents on the command line? JQ is the tool you’re looking for. In this 4 parts post series, you’ll discover why and how I use JQ with OpenAPI Specification files. But more important, you’ll get some basic and more advanced example of how to use JQ on any JSON document to get and modify JSON data as you want. In this first part we’ll focus on what is JQ, why I use it with OpenAPI files and we’ll learn how to invoke JQ and discover some of the many JQ filters that can be used to extract data from JSON.

This 4 parts post is the first one of a new API Toolbox category in which I’ll talk about the tools I use when doing API related stuff, why I use them and how. This post is also my first one using Asciinema, an awesome tool allowing to record and share terminal sessions.

JQ and OpenAPI Series

JQ’s documentation is quite complete and there are many tutorials and Stackoverflow answers, so why bother writing this series? First reason, I regularly meet people working with APIs and/or JSON files who actually don’t know JQ exists and how it could save their life (or at least their time). Second reason, I often use it with OpenAPI specification files and I found that showing how JQ can be used on such a widely adopted and familiar JSON based format could help to learn how to use it (and also writing this post actually helped me to improve my JQ skills!).

1 - Using JQ to extract data from OpenAPI files
2 - Using JQ command line arguments, functions and modules
3 - Modifying OpenAPI files with JQ
4 - Bonus: Coloring JQ's raw output

What is JQ and why I use it (on OpenAPI files)

According to JQ’s website, jq can mangle the data format that you have into the one that you want with and also jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.

I have been using JQ to transform JSON data when making API calls on the command line for quite a while, but lately I have been using it to manipulate OpenAPI Specification files. This is the use case I will focus on in this post (I’ll keep the API calls use case for another post).

The OpenAPI Specification (or OAS) is a standard and programming-language agnostic REST API description format. It can be used during the design of an API to formally describe the API’s contract. It can also be used to generate documentation, generate code or to configure tools such as API gateways. An OpenAPI file can be in YAML or JSON format. If you want to learn more about this format, read What is the OpenAPI Specification. In order to have a good understanding of an OpenAPI document structure, you should check my OpenAPI Map.

In my daily job I have to work with OpenAPI files when doing API design reviews. Tools such as SwaggerUI or ReDoc easily provide a user friendly view of OpenAPI files, but when it comes to have a more specific view to check various design concerns, you need to use something else. I can use JQ when I want to know which operations can be used with a given Oauth Scope, where a reusable schema is used or checking if an API or multiples APIs are consistent. I also have to deal with OpenAPI files when working on my company’s API catalog. I had to generate API calls body to load OpenAPI files into it, I had to extract some data from them with JQ to do so. I also had to modify them to remove deprecated elements in order to avoid showing them in their documentation.

The examples shown in this post are based on my regular use of JQ+OpenAPI but I expanded my original JQ scripts set with other ones in order to show more of JQ’s features.

Installation

If you want to play with JQ and OpenAPI as you read this post, you’ll need to install JQ and download this post’s related content(JQ scripts, OpenAPI demo files and Asciinema sessions and their underlying scripts).

Install JQ

JQ is a portable command line tool that’s very easy to install. Its website states that jq is written in portable C, and it has zero runtime dependencies. You can download a single binary, scp it to a far away machine of the same type, and expect it to work (scp is a file transfer tool). This is actually true, I have tested it on Linux servers, Windows CMD terminal, Windows Gitbash (standalone and inside VS Code) and MacOS terminal: never had a problem with it.

To install JQ on my personnal MacBook, I used brew install jq. On my professional Windows laptop, I simply downloaded the binary and added it to my PATH environment variable. Check JQ’s download page to see all available versions and ways to install it.

Once installed, open a terminal and run jq --help to check if everything is OK.

Get post’s content

All examples shown in this post are based on JQ 1.6 and OpenAPI 3. All examples can be copied using the button and downloaded using the one on code snippets. All source code can be retrieved from the JQ and OpenAPI post series’ github repository.

git clone https://github.com/arno-di-loreto/jq-and-openapi/
cd jq-and-openapi
git checkout part-1

[apihandyman.io]$ git clone https://github.com/arno-di-loreto/jq-and-openapi/
[apihandyman.io]$ cd jq-and-openapi
[apihandyman.io]$ git checkout part-1

Most of this post’s examples are run against the same OpenAPI file (demo-api-openapi.json) which is a slightly modified version of an example coming from my book The Design of Web APIs, I added a few elements here and there, convert it from YAML to JSON and uglify it.

demo-api-openapi.json (uglyfied OpenAPI 3.0)

{"openapi":"3.0.0","info":{"title":"Banking API","version":"1.0.0-snapshot","description":"The Banking API provides access to the [Banking Company](http://www.bankingcompany.com) services, which include bank account information, beneficiaries, and money transfer management.<!--more-->\n\n# Authentication\n\n## How to \n- Register\n- Create an APP\n- Request credentials\n\n# Use cases\n\n## Transferring money to an account or preexisting beneficiary\n\nThe _transfer money_ operation allows one to transfer an `amount` of money from a `source` account to a `destination` account or beneficiary.\nIn order to use an appropriate `source` and `destination`, we recommend to use _list sources_ and _list source's destinations_ as shown in the figure below (instead of using _list accounts_ and _list beneficiaries_).\n\n![Diagram](http://localhost:9090/12.2-operation-manual-diagram.svg)\n\n## Cancelling a delayed or recurring money transfer\n\n- List money transfers: To list existing money transfers and select the one to delete\n- Cancel a money transfer: To cancel the selected money transfer\n","contact":{"name":"The Banking API team","email":"[email protected]","url":"developer.bankingcompany.com"}},"tags":[{"name":"Transfers","description":"Everything you need to manage money transfers. A money transfer consists in transferring money from a source account to a destination account."},{"name":"Beneficiaries","description":"Everything you need to manage money transfer beneficiaries. Beneficiaries are pre-registred external accounts that can be used as destinations for money transfers."}],"paths":{"/accounts":{"get":{"tags":["Accounts"],"summary":"List accounts","responses":{"200":{"description":"User's accounts","content":{"application/json":{"schema":{"required":["properties"],"properties":{"items":{"type":"array","items":{"$ref":"#/components/schemas/Account"}}}}}}}}}},"/accounts/{id}":{"get":{"tags":["Accounts"],"summary":"Get an account","parameters":[{"name":"id","in":"path","description":"Account's id","required":true,"schema":{"type":"string"}}],"responses":{"200":{"description":"The account","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Account"}}}}},"x-implementation":{"security":{"description":"Only accounts belonging to user referenced in security data;\nreturn a 404 if this is not the case\n","source":{"system":"security","location":"jwt.sub"},"fail":404}}}},"/beneficiaries":{"post":{"tags":["Beneficiaries"],"summary":"Register a beneficiary","security":[{"BankingAPIScopes":["beneficiary:create","beneficiary:admin"]}],"responses":{"201":{"description":"Beneficiary added"}}},"get":{"tags":["Beneficiaries"],"summary":"List beneficiaries","security":[{"BankingAPIScopes":["beneficiary:read","beneficiary:admin"]}],"responses":{"200":{"description":"The beneficiaries list"}}}},"/beneficiaries/{id}":{"parameters":[{"name":"id","in":"path","description":"Beneficiary's id","required":true,"schema":{"type":"string"}}],"delete":{"deprecated":true,"tags":["Beneficiaries"],"summary":"Delete a beneficiary","security":[{"BankingAPIScopes":["beneficiary:delete","beneficiary:admin"]}],"responses":{"204":{"description":"Beneficiary deleted"}}},"patch":{"deprecated":true,"tags":["Beneficiaries"],"summary":"Updates a beneficiary","security":[{"BankingAPIScopes":["beneficiary:admin"]}],"requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/BeneficiaryUpdate"}}}},"responses":{"200":{"description":"The updated beneficiary"}}},"get":{"tags":["Beneficiaries"],"summary":"Get a beneficiary","security":[{"BankingAPIScopes":["beneficiary:read","beneficiary:admin"]}],"responses":{"200":{"description":"The beneficiary"}}}},"/sources":{"get":{"summary":"List transfer sources","tags":["Transfers"],"description":"Not all bank accounts can be used as a source\nfor a money transfers. This operation returns\nonly the accounts elligible as a money transfer\nsource.\n","responses":{"200":{"description":"The transfer sources"}}}},"/sources/{id}/destinations":{"parameters":[{"name":"id","in":"path","description":"Source's id","required":true,"schema":{"type":"string"}}],"get":{"summary":"List transfer source's destinations","tags":["Transfers"],"description":"Depending on the source account, only specific\nbeneficiaries or accounts can be used as a money\ntransfer destination.\nThis operation returns them.\n","responses":{"200":{"description":"The transfer destination"}}}},"/transfers":{"post":{"summary":"Transfer money","security":[{"BankingAPIScopes":["transfer:create","transfer:admin"]}],"tags":["Transfers"],"description":"This operation allows one to transfer an `amount` of money from a `source` account to a `destination` account.\nThere are three different types of money transfer:\n  - Immediate -- these are executed as soon as the request is received \n  - Delayed -- these are executed upon a given future `date`\n  - Recurring -- these are executed a given `occurrences` number of times at a given `frequency` -- the first occurrence being executed immediately or at a given `date`\n","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/TransferRequest"},"examples":{"immediate":{"summary":"Immediate transfer","description":"The money transfer is executed immediately","value":{"source":"000534115776675","destination":"000567689879878","amount":456.2}},"delayed":{"summary":"Delayed transfer","description":"The money transfer is executed at a given date","value":{"source":"000534115776675","destination":"000567689879878","amount":456.2,"date":"2019-03-19"}},"recurring":{"summary":"Recurring transfer","description":"The money transfer is executed at a given date reurringly","value":{"source":"000534115776675","destination":"000567689879878","amount":456.2,"date":"2019-03-19","occurrences":1,"frequency":"MONTHLY"}}}}}},"responses":{"201":{"description":"Immediate or recurring transfer executed","content":{"application/json":{"schema":{"$ref":"#/components/schemas/TransferResponse"},"examples":{"immediate":{"summary":"Immediate transfer","description":"The money transfer is executed immediately","value":{"source":"000534115776675","destination":"000567689879878","amount":456.2}},"recurring":{"summary":"Recurring transfer","description":"The first occurence is executed immediately","value":{"source":"000534115776675","destination":"000567689879878","amount":456.2,"date":"2019-03-19","occurrences":1,"frequency":"MONTHLY"}}}}}},"202":{"description":"Delayed or recurring delayed transfer accepted","content":{"application/json":{"schema":{"$ref":"#/components/schemas/TransferResponse"},"examples":{"delayed":{"summary":"Delayed transfer","description":"The money transfer is executed at a given date","value":{"source":"000534115776675","destination":"000567689879878","amount":456.2,"date":"2019-03-19"}},"recurring":{"summary":"Recurring transfer","description":"The money transfer is executed at a given date reurringly","value":{"source":"000534115776675","destination":"000567689879878","amount":456.2,"date":"2019-03-19","occurrences":1,"frequency":"MONTHLY"}}}}}},"400":{"description":"The transfer is rejected due to an error in the request properties or an insufficient balance.\nEach error provides the property `source` of the error along with a human-readable `message` and its `type`:\n\n- MANDATORY_PROPERTY: The property indicated in `source` is missing\n- INVALID_FORMAT: The format of the property indicated in `source` is invalid\n- INVALID_VALUE: The value of the property indicated in `source` is invalid\n- INSUFFICIENT_BALANCE: The `amount` property is higher than the `source` account balance\n","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ConsumerError"}}}}}},"get":{"summary":"List money transfers","tags":["Transfers"],"security":[{"BankingAPIScopes":["transfer:read","transfer:admin"]}],"responses":{"200":{"description":"Transfers list","content":{"application/json":{"schema":{"$ref":"#/components/schemas/TransferList"}}}}}}},"/transfers/{id}":{"parameters":[{"name":"id","in":"path","description":"Transfer's id","required":true,"schema":{"type":"string"}}],"get":{"summary":"Get a money transfer","tags":["Transfers"],"security":[{"BankingAPIScopes":["transfer:read","transfer:admin"]}],"responses":{"200":{"description":"The money transfer"},"404":{"description":"The money transfer does not exist"}}},"x-tension-example":{"some":"value"},"patch":{"tags":["Transfers"],"responses":{"200":{"description":"The money transfer has been update"}}},"delete":{"summary":"Cancel a money transfer","tags":["Transfers"],"security":[{"BankingAPIScopes":["transfer:delete","transfer:admin"]}],"description":"Only delayed or recurring money transfer can be canceled","responses":{"204":{"description":"The money transfer has been deleted"},"404":{"description":"The money transfer does not exist"}}}}},"components":{"securitySchemes":{"BankingAPIScopes":{"type":"oauth2","flows":{"implicit":{"authorizationUrl":"https://auth.bankingcompany.com/authorize","scopes":{"transfer:create":"Create transfers","transfer:read":"Read transfers","transfer:delete":"Delete transfers","transfer:admin":"Create, read, and delete transfers","beneficiary:create":"Create beneficiaries","beneficiary:read":"List beneficiaries","beneficiary:delete":"delete beneficiaries","beneficiary:admin":"Create, read, and delete beneficiaries","account:read":"Read accounts","account:admin":"Read accounts"}}}}},"schemas":{"BeneficiaryUpdate":{"description":"A beneficiary update parameter","properties":{"name":{"type":"string"}}},"UselessSchema":{"description":"An unused useless schema","type":"string"},"TransferRequest":{"description":"A money transfer request","required":["source","destination","amount"],"properties":{"deprecatedPropertyExample":{"deprecated":true,"type":"string","description":"An example of a deprecated property"},"source":{"type":"string","description":"Source account number","minLength":15,"maxLength":15,"pattern":"^\\d{15}$","example":"000534115776675"},"destination":{"type":"string","description":"Destination account number","minLength":15,"maxLength":15,"pattern":"^\\d{15}$","example":"000567689879878"},"amount":{"type":"number","example":456.2,"minimum":0,"exclusiveMinimum":true},"date":{"type":"string","format":"date","description":"Execution date for a delayed transfer\nor first execution date for a recurring one\n","example":"2019-03-19"},"occurrences":{"type":"integer","description":"Number of times a recurring transfer will be executed\n","example":2,"minimum":2,"maximum":100},"frequency":{"type":"string","description":"Frequency of recurring transfer's execution","example":"MONTHLY","enum":["WEEKLY","MONTHLY","QUARTERLY","YEARLY"]}}},"TransferResponse":{"allOf":[{"required":["id","type","status"],"properties":{"id":{"type":"string","example":"1611e71f-1bb2-412f-8c43-92b275a5c321"},"type":{"type":"string","enum":["IMMEDIATE","DELAYED","RECURRING"],"example":"RECURRING"},"status":{"type":"string","description":"An immediate transfer is always `EXECUTED`, a delayed transfer can be `EXECUTED` or `PENDING` and a recurring one is always `PENDING`\n","enum":["EXECUTED","PENDING"],"example":"PENDING"},"requestDate":{"type":"string","example":"2019-09-19"}}},{"$ref":"#/components/schemas/TransferRequest"}]},"TransferList":{"properties":{"items":{"type":"array","items":{"$ref":"#/components/schemas/TransferResponse"}}}},"ConsumerError":{"required":["errors"],"properties":{"errors":{"description":"A list of errors providing detailed information about the problem","type":"array","minItems":1,"items":{"required":["source","type","message"],"properties":{"source":{"description":"the property source of the error","type":"string","example":"amount","enum":["source","destination","amount","date","occurrences","frequency"]},"type":{"type":"string","example":"MANDATORY_PROPERTY","enum":["MANDATORY_PROPERTY","INVALID_FORMAT","INVALID_VALUE","INSUFFICIENT_BALANCE"]},"message":{"description":"a human-readable error message","type":"string","example":"The money transfer's amount must be provided"}}}}}},"ProviderError":{"properties":{"errors":{"type":"array","minItems":1,"maxItems":1,"items":{"properties":{"message":{"type":"string"}}}}}},"Account":{"properties":{"balance":{"description":"The balance in the account's default currency","type":"object","title":"Amount","required":["value","currency"],"properties":{"value":{"description":"Balance's value using the number of decimal places defined by ISO 4217","externalDocs":{"description":"Decimal places table","url":"https://www.currency-iso.org/en/home/tables/table-a1.html"},"type":"number","x-implementation":{"description":"The real time balance (not the daily one!)","source":{"system":"Core Banking","data":"ZBAL0.RTBAL"}}},"currency":{"description":"An ISO 4217 code","externalDocs":{"url":"https://www.iso.org/iso-4217-currency-codes.html"},"type":"string","example":"USD","x-implementation":{"source":{"system":"Core Banking","data":"ZBAL0.RTCUR"}}}}}}}}}}

There are also two other almost empty examples used when working on multiple files.

Invoke JQ

In this first section, we’ll learn how to invoke JQ and its basic principles. The whole content of this section has been recorded with Asciinema (but is available as regular text right after the player).

Invoking JQ

Beautify and color JSON

As shown in the following listing, the demo-api-openapi.json file is quite complex to read when printed on a terminal when using cat demo-api-openapi.json.

cat demo-api-openapi.json

Let's see what's inside the demo-api-openapi.json OpenAPI file

[apihandyman.io]$ cat demo-api-openapi.json
{"openapi":"3.0.0","info":{"title":"Banking API", ...}
# The whole document is printed one a single line
# That's totally unreadable 😱

Of course we could open our favorite code editor and beautify it. But this can also be done on the command line thanks to JQ. All we need to do is piping (with |) the file content to JQ like this cat api-openapi.json | jq '.'. Icing on the cake, the output is colored. Note that you can also simply call JQ with the JSON’s filename like this: jq '.' demo-api-openapi.json.

cat demo-api-openapi.json | jq '.'

Let's pipe this into JQ to see if it's better

[apihandyman.io]$ cat demo-api-openapi.json | jq '.'
{
  "openapi": "3.0.0",
  "info": {
    "title": "Banking API",
    ...
}
# JSON is beautified and colored 😍

jq '.' demo-api-openapi.json

JQ can also be called with a file parameter

[apihandyman.io]$ jq '.' demo-api-openapi.json
{
  "openapi": "3.0.0",
  "info": {
    "title": "Banking API",
    ...
}
# JSON is beautified and colored 😍

The first parameter of a JQ command, here '.', is the JQ filter that will be used to process the provided JSON. This . filter, named identity, is the most simple one, it only returns what it gets. Obviously, I wouldn’t write such a huge blog post to talk about a tool that only beautifies and colors JSON. Let’s see some basic JQ filtering in action.

Extract data from JSON

Even beautified and colored, the file is still quite complex to read. Indeed, the beautified JSON file is around 750 lines long. What if we only want to see the info section? It’s dead simple, we only need to use the .info JQ filter on the file with jq '.info' demo-api-openapi.json as shown below. And you probably already guessed how to get only the API’s name (called title in OpenAPI): .info.title.

jq '.' demo-api-openapi.json | wc -l
jq '.info' demo-api-openapi.json
jq '.info.title' demo-api-openapi.json

Only showing the info section or the API's name (title)

[apihandyman.io]$ jq '.' demo-api-openapi.json | wc -l
     753
# Beautified JSON is 750 lines long 
[apihandyman.io]$ jq '.info' demo-api-openapi.json
{
  "title": "Banking API",
  "version": "1.0.0-snapshot",
  "description": "The Banking API provides access ...",
  "contact": {
    "name": "The Banking API team",
    "email": "[email protected]",
    "url": "developer.bankingcompany.com"
  }
}

[apihandyman.io]$ jq '.info.title' demo-api-openapi.json
"Banking API"

The most simple JQ filters simply consist in describing the paths of the element you want to get.

Being able to simply extract a value from a JSON is quite interesting, but that’s only the tip of the tip of the tip the iceberg.

Generate tailor made JSON

With a JQ filter, you can generate tailor made JSON containing exactly what you want, how you want it. To do so, use the {} object constructor and describe what you want in it almost just like you would write a JSON object. The following listing show how to create an object containing the API name, its version and the contact’s name. Each value is the result of a JQ filter applied to the JSON provided to the filter.

jq '{name: .info.title, version: .info.version, contact: .info.contact.name}' demo-api-openapi.json

JQ can totally transform the provided JSON

[apihandyman.io]$ jq '{name: .info.title, version: .info.version, contact: .info.contact.name}' demo-api-openapi.json
{
  "name": "Banking API",
  "version": "1.0.0-snapshot",
  "contact": "The Banking API team"
}

Generate raw text

JQ is also able to output raw text instead of JSON. To do so, a filter just need to return a value. The following listing shows three attempts of printing text. The first example (line 1) simply prints the API name (.info.title) as we already have done before. The output contains no JSON structure, only the requested value as a quoted string ("Banking API"). The second one (line 4) tries to outputs tab separated API’s name, version and contact’s name. Note that the + operator is used to concatenate the different values which can come from the provided JSON (.info.title for example) but can also be static ones ("\t", the tab separator). Unfortunately, the result is not what is expected, the tabs (\t) are not interpreted. In order to actually get raw text that will be fully interpreted by the terminal, the -r flag must be provided to JQ. This is what is shown in the last example (line 10): there is no more quotes, and the value are separated by tabs.

jq '.info.title' demo-api-openapi.json
jq '.info.title + "\t" + .info.version + "\t" + .info.contact.name' demo-api-openapi.json
jq -r '.info.title + "\t" + .info.version + "\t" + .info.contact.name' demo-api-openapi.json

JQ can generate raw text (don't forget -r flag)

[apihandyman.io]$ jq '.info.title' demo-api-openapi.json
"Banking API"

[apihandyman.io]$ jq '.info.title + "\t" + .info.version + "\t" + .info.contact.name' demo-api-openapi.json
"Banking API\t1.0.0-snapshot\tThe Banking API team"

[apihandyman.io]$ jq -r '.info.title + "\t" + .info.version + "\t" + .info.contact.name' demo-api-openapi.json
Banking API     1.0.0-snapshot  The Banking API team

Pipe JQ commands and filters

Piping is a powerful command line concept: the result of a first command can be forwarded to another one using a pipe (|) . This is what we have done on our first JQ command: we took the result of a cat command (which outputs the content of a file) to provided it to JQ and the output of JQ can be forwarded to another command which could be, for example, another JQ one, as shown in the following listing.

cat demo-api-openapi.json | jq '{name: .info.title, version: .info.version, contact: .info.contact.name}' | jq -r '.name + "\t" + .version'

JQ commands can be chained with pipe (like many other command line ones)

[apihandyman.io]$ cat demo-api-openapi.json | \
jq '{name: .info.title, version: .info.version, contact: .info.contact.name}' | \
jq -r '.name + "\t" + .version'
Banking API     1.0.0-snapshot

JQ also takes advantage of this piping concept. Indeed, JQ filters can be chained using pipe as shown in the following listing. The full JSON document is (implicitly) provided to the first filter which creates an object containing a name, version and title and its result is forwarded, using |, to another filter which return a string containing tab separated name and version.

jq -r '{name: .info.title, version: .info.version, contact: .info.contact.name} | .name + "\t" + .version' demo-api-openapi.json

More interesting, JQ filters can be chained too with (also with pipe)

[apihandyman.io]$ jq -r '{name: .info.title, version: .info.version, contact: .info.contact.name} | .name + "\t" + .version' demo-api-openapi.json
Banking API     1.0.0-snapshot

Use JQ files

As a JQ filter chain becomes complex, writing it on the command line can become cumbersome and error prone. Fortunately, JQ comes with a useful -f file flag allowing to load filters from a file as shown in the following listing. The new command line gives the same result as the one before, the only difference is that the filters are now loaded from the basics.jq file (files containing JQ filters usually have a .jq extension).

jq -r -f basics.jq demo-api-openapi.json

When JQ scripts become complex, better use a JQ file (-f file.jq)

[apihandyman.io]$ jq -r -f basics.jq demo-api-openapi.json
Banking API     1.0.0-snapshot

basics.jq

# Files are easier to read and can be commented
# Creates an object
{
  name: .info.title, 
  version: .info.version, 
  contact: .info.contact.name
} |
# Outputs tab separated name and version
# + can be used to concatene almost everything
# (as you will see in later examples)
# Don't forget the -r flag
.name + "\t" + .version

Now that we know the basics of JQ, let’s try more complex stuff on OpenAPI JSON files.

Use JQ filters on an OpenAPI file

In this section, we’ll learn to use some of the many JQ filters by extracting data from an OpenAPI file. For each example, you get:

A fully detailed, step by step asciinema bash session explaining how the result is achieved
An OpenAPI structure figure and description (based on the OpenAPI Map)
A list of (new) JQ filters used
A summarized explanation of how the result is achieved (⚠️ far less details than in the asciinema bash session)
A fully commented JQ file

List paths

Let’s start by extracting the API’s paths:

List API's paths

jq -r -f list-paths.jq demo-api-openapi.json

List API's paths

[apihandyman.io]$ jq -r -f list-paths.jq demo-api-openapi.json 
/accounts
/accounts/{id}
/beneficiaries
/beneficiaries/{id}
/sources
/sources/{id}/destinations
/transfers
/transfers/{id}

In an OpenAPI file, the available paths are the keys of the paths object.

To get these paths, we’ll use the following JQ filters:

JQ Filters
`.foo` `.foo.bar`	Returns element targeted by period separated path
`keys`	Returns object's keys (properties names)
`.[]` `.[index]`	Returns all or some elements of an object or array identified by an index

To extract the paths, we only need to use the keys filter on the paths object identified by .paths. This keys filter returns an array containing the keys (property names, hence the paths in our case) of an object. Then we use [] on the result to flatten the array in order to get raw text.

list-paths.jq

# 1 - Selects the paths object
#-----------------------------
.paths
# 2 - Keeps only the keys in paths (/whatever)
#---------------------------------------------
| keys
# 3 - Flattens the array (for raw output)
#---------------------------------------- 
[]

List HTTP methods

Let’s go a level deeper to list all HTTP methods used in an API:

List HTTP methods

jq -r -f list-http-methods.jq demo-api-openapi.json

List used HTTP methods

[apihandyman.io]$ jq -r -f list-http-methods.jq demo-api-openapi.json
delete
get
patch
post

In an OpenAPI file, HTTP methods are keys inside a path object. Unfortunately, path objects may have other properties than HTTP methods ones, like summary, description, parameters or x- custom properties (we take for granted that there is no $ref properties). So we’ll need to clean this array to get rid of all other properties than HTTP methods.

To list all of these HTTP methods, we’ll use 4 new JQ filters:

JQ Filters
`[]` `[1, 2, 3]` `[.foo, .bar]` `[.items[].name]`	Creates an array
`map(filter)`	Applies a filter to each element of an array
`select(filter returning boolean)`	Returns element for which filter returns true
`IN(value1, value2, value3)`	Returns true if element is one of the values

The JQ file that follows can be roughly split in 4 steps:

(Line 1) To create the array of path objects properties, we use the array constructor [filter] and inside it do the necessary with various filters to get all keys of all path objects. Note how [] is used on .paths to only keep its properties content without caring about the actual paths names.
(Line 11) Then to clean the array of unwanted values, we use the map filter which allows to apply a filter to each element of a list. The filter executed by map consists in select(. | IN("value 1", ..., "value N")). The select filter let pass values for which its parameter filter returns true. In our case, the select parameter filter use IN which returns true if the provided value is one of its parameter (here, all possible HTTP methods). Note that inside select . represents the current element of the array being processed.
(Line 23) Then, we apply the unique filter to the array of all HTTP methods of all paths in order to keep a single occurrence of each.
(Line 27) And eventually the resulting array is flatten with [] for raw output.

list-http-methods.jq

# 1 - Creates an array of all HTTP methods
#     inside paths["/whatever"]
#-----------------------------------------
# It returns ["get", "post","summary","x-example", "post"]
[
  .paths[] # Selects the paths["/whatever"] properties content
           # to keeps only the operations
  | keys[] # Keeps only the keys (HTTP methods and few other things)
           # and flattens array
]
# 2 - Cleans keys to keep only HTTP method
#-----------------------------------------
# It returns ["get", "post", "post"]
| map( # Applies a filter to each element
  select( # Keeps only elements for which the following is true
   # With IN, which returns true if the value is one of its
   # parameters, we can get rid of x- , parameters
   # description and summary properties
   IN("get", "put", "post", "delete", 
      "options", "head", "patch", "trace")
  )
)
# 3 - Keeps an occurrence of each HTTP method
#--------------------------------------------
# It returns ["get", "post"]
| unique # Keeps only an occurence of each element
# 4 - Generates raw string
#-------------------------
[] # Flattens array for raw output

Count HTTP status codes usage

Now we take another step deeper into the OpenAPI file by listing all HTTP status codes and sorting them by how many times they are used.

Count HTTP status codes usage

jq -r -f list-http-status-codes.jq demo-api-openapi.json

Count how many times HTTP status codes are used

[apihandyman.io]$ jq -r -f list-http-status-codes.jq demo-api-openapi.json
200     10
201     2
204     2
404     2
202     1
400     1

In an OpenAPI files, HTTP status codes used to signify how went the API call are located in the responses properties of all operations (identified by an HTTP method) which are located inside all paths (identified by a path like /whatever) in the paths property. In the responses object, each response object is identified by its HTTP status code or by “default”. Note that the response object can also contains x- custom properties that we’ll need to get rid of.

To list HTTP status codes and how many times they are used, we’ll learn how to use the following new JQ filters:

JQ Filters
`.foo?`	Does not return an error if targeted element does not exist
`test(regex)` `test(regex;flags)`	Returns true if element matches regex
`not`	Returns opposite boolean element
`group_by(path_expression)` `group_by(.foo)`	Groups array elements according to provided path (returns array of array)
`{}` `{name1: filter, name2: another_filter}`	Creates an object
`length`	Returns the length of element (array, object, string, null)
`sort_by`	Returns element sort by value or path_expression
`tostring`	Turns any element into a string

The JQ file that follows is split in 5 steps:

(Line 1) The first step looks like previous example’s, but this time we go 2 levels deeper. We also use ? when getting responses property content because not all properties inside a path object are actual operations. Indeed some of them can be simple string (summary, description), object (servers) or array (parameters) and therefore not have a responses properties. Without ?, using .responses would return an error when used on properties such as summary or description. With ?, no error but a null value is returned. The same goes for keys? which may be fed with a null values having no keys at all.
(Line 12) On the second step, we need to get rid of possible x- properties. This is done like in previous example with a map(select(filter)). In this case, the select’s filter checks if the value does not start by x- using the test filter which return true if the value matches the regex parameter and then not to negate this result.
(Line 21) Now we have an array containing all HTTP status codes of all operations, we can count how many times each one is used. This is done using group_by which group equal values together. It takes an array of something and returns an array of array of something. Each internal array containing equal values. Once that is done we can create on object for each internal array using map. It contains the HTTP status code which is the first value in the array (which contains the same value multiple times) and a count which is the length of the array.
(Line 32) Then we can sort this array of {code, count} by descending count using sort_by(-.count).
(Line 37) And eventually we generate the tab separated raw text output with map and []. Note how count is converted into a string before being concatenated.

list-http-status-codes.jq

# 1 - Selects all properties of all responses
#--------------------------------------------
# It returns ["404", "200", "200", "x-example"]
[
  .paths[][].responses? # ? avoid getting an error if
                        # responses does not exist
  | keys? # ? avoid getting an error if . is not an
          # object and has no keys
  | .[] # [ ["404", "200"], ["200", "x-example"] ] ⤵️
      #                     ["404", "200", "200", "x-example"]
]
# 2 - Removes x- properties
#--------------------------
# It returns ["200", "404", "200"]
| map( # Applies a filter to each element
  select( # Keep elements for which what follows return true
    test("^x-") # Returns true if value match the regex parameter
    | not # Returns the opposite of a boolean value
  )
) 
# 3 - Counts how many times each code is used
#--------------------------------------------
# It returns [ {"code": "404", "count": 1}, 
#              {"code": "200", "count": 2} ]
| group_by(.) # ["404", "200", "200"] ➡️ [["400"],["200", "200"]]
| map( # Applies a filter to each element
  { # Creates an object
    code: .[0], # ["200", "200"] ➡️ ["200"] ➡️ "200"
    count: length # ["200", "200"] ➡️ 2
  }
)
# 4 - Sorts by descending count
#------------------------------
# It returns [ {"code": "200", "count": 2}, 
#              {"code": "404", "count": 1} ]
| sort_by(-.count) # Sort array by parameter value
# 5 - Generates tab separated string output
#------------------------------------------
| map( # Applies a filter to each element
  .code + 
  "\t" + 
  (.count | tostring) # count is a number
                      # it must be converted to string
                      # to be concatenated to other string
)[] # Flattens array for raw output

List operations

Now, let’s try something more interesting: extracting the API’s operation list.

List operations

As the following listing shows, we will extract for each operation, its HTTP method, path, summary and indicate if the operation is deprecated.

jq -r -f list-operations.jq demo-api-openapi.json

List operations

[apihandyman.io]$ jq -r -f list-operations.jq demo-api-openapi.json 
get     /accounts       List accounts
get     /accounts/{id}  Get an account
post    /beneficiaries  Register a beneficiary
get     /beneficiaries  List beneficiaries
delete  /beneficiaries/{id}     Delete a beneficiary (deprecated)
patch   /beneficiaries/{id}     Updates a beneficiary (deprecated)
get     /beneficiaries/{id}     Get a beneficiary
get     /sources        List transfer sources
get     /sources/{id}/destinations      List transfer source's destinations
post    /transfers      Transfer money
get     /transfers      List money transfers
get     /transfers/{id} Get a money transfer
patch   /transfers/{id}
delete  /transfers/{id} Cancel a money transfer

Thanks to previous examples, we start to know an OpenAPI file structure. The operation’s paths come first, then their HTTP method and a level below, we can access to summary and deprecated properties which are both optional.

To generate the operations list, we’ll learn how to use the following new JQ filters:

JQ Filters
`to_entries`	Takes an object and returns an array of key and value objects
`.foo as $some_variable`	Defines a reusable variable
`if condition then filter else another_filter end`	Conditions filter execution

The following JQ script is split in 3 steps:

(Line 1) We start by creating an array of {key: /path, value: path content} using to_entries on .paths. Then we filter this array to get rid of possible x-tensions checking the key value does not start by “x-“ using map, select, test and not as already seen in a previous example.
(Line 11) Then we create an array of {path, method, summary, deprecated} objects. To get rid of possible extensions we reuse the IN filter seen previously. The interesting thing in this step is how we define (line 15) and use (line 35) the $path variable. Such variable definition is very useful to keep some values for later use without impacting the data flow. Indeed the data coming out of .key as $ path is the exactly the same as the one that came in.
(Line 42) And to finish, we output tab separated raw text (adding deprecated mention when necessary). See line 48 how an if then else can be used.

list-operations.jq

# 1 - Selects paths objects
#--------------------------
# returns [{key: path, value: path value}]
.paths # Selects the paths property content
| to_entries # Transforms
             # { "/resources": { "get": {operation data}}} 
             # to 
             # [ { "key": "/resources", 
             #     "value": { "get": {operation data}} ]
| map(select(.key | test("^x-") | not)) # Gets rid of x-tensions
# 2 - Creates an array of operations
#-----------------------------------
# returns [{path, method, summary, deprecated}]
| map ( # Applies a transformation to each element
  .key as $path # Stores the path value (.key) 
                  # in a variable ($path) for later use
  | .value # Keeps only the path's content 
           # { "get": {operation data}}
  | to_entries # Transforms 
               # { "get": {operation data}}
               # to
               # [ { "key": "get", 
               #     "value": {operation data}} ]
  | map( # Applies a transformation to each element
    select( # Keeps only elements for which the following is true
      # With IN, which returns true if the value is one of its
      # parameters, we can get rid of x- , parameters
      # description and summary properties
      .key | IN("get", "put", "post", "delete", 
         "options", "head", "patch", "trace")
    )
    | # Creates a new JSON object
    {
      method: .key,
      path: $path, # Using the variable defined on line 4
      summary: .value.summary?,
      deprecated: .value.deprecated?
    }
  )[] # Flattens array to avoid having an array 
      # of array of {path, method, summary, deprecated}
) # Now we have an array of {path, method, summary, deprecated}
# 3 - Outputs tab separated raw text
#-----------------------------------
| map( # Applies a transformation to each element
  .method + "\t" + 
  .path + "\t" + 
  .summary + 
  (if .deprecated then " (deprecated)" else "" end)
)
[] # Flattens array for raw output

List x-tensions

It can be of interest to know which extensions are used in an OpenAPI document, where they are located and what are their values.

List x-tensions

jq -r -f list-xtensions.jq demo-api-openapi.json

Listing extensions, their locations and values

[apihandyman.io]$ jq -r -f list-xtensions.jq demo-api-openapi.json
[
  {
    "name": "x-implementation",
    "path": [
      "paths",
      "/accounts/{id}",
      "get",
      "x-implementation"
    ],
    "ref": "#/paths/~1accounts~1{id}/get/x-implementation",
    "value": {
      "security": {
        "description": "Only accounts belonging to user referenced in security data;\nreturn a 404 if this is not the case\n",
        "source": {
          "system": "security",
          "location": "jwt.sub"
        },
        "fail": 404
      }
    }
  },
  {
    "name": "x-tension-example",
    "path": [
      "paths",
      "/transfers/{id}",
      "x-tension-example"
    ],
    "ref": "#/paths/~1transfers~1{id}/x-tension-example",
    "value": {
      "some": "value"
    }
  },
  {
    "name": "x-implementation",
    "path": [
      "components",
      "schemas",
      "Account",
      "properties",
      "balance",
      "properties",
      "value",
      "x-implementation"
    ],
    "ref": "#/components/schemas/Account/properties/balance/properties/value/x-implementation",
    "value": {
      "description": "The real time balance (not the daily one!)",
      "source": {
        "system": "Core Banking",
        "data": "ZBAL0.RTBAL"
      }
    }
  },
  {
    "name": "x-implementation",
    "path": [
      "components",
      "schemas",
      "Account",
      "properties",
      "balance",
      "properties",
      "currency",
      "x-implementation"
    ],
    "ref": "#/components/schemas/Account/properties/balance/properties/currency/x-implementation",
    "value": {
      "source": {
        "system": "Core Banking",
        "data": "ZBAL0.RTCUR"
      }
    }
  }
]

The OpenAPI Specification is extensible, it means that custom data can be added to it for various purpose. The custom data structures can either be called extensions, x-tensions or vendor extension. In order to allow standard parsers to not raise an error, such custom data structure must be added using a specific key name starting by “x-“ in order to identify them. The tricky part with extensions in our case, is that they can be located (almost) anywhere in a document, the only sure thing is that they have a key name starting by “x-“. To learn more about this, check OpenAPI (Swagger) 2.0 Tutorial - Part 9 - Extending the OpenAPI Specification (note: extension management did not change between version 2.0 and 3).

To list all extensions as shown in the terminal listing above, we’ll learn how to use the following JQ filters:

JQ Filters
`paths`	Lists all possible paths in documents, each path is represented as an array
`+` `value1 + value2`	Adds/concatenates/merges numbers/strings/arrays or objects
`map_values(filter)`	Updates all provided values with filter
`gsub(regex;replacement)` `gsub("toreplace";"replaced)`	Replaces a string within a string
`join(separator)` `join(",")`	Joins string with a separator
`getpath(path)` `getpath(["a","path"])`	Returns value corresponding to a path (defined as an array), see also paths

The following JQ script consist in 3 steps:

(Line 1) First, we need to store the full document for later use (to get the extensions value)
(Line 5) Then we list all extensions paths by using paths which returns all possible paths and removing the ones that do not have a leaf starting with a “x-“.
(Line 18) And last step, we build an object containing data for each found extension. This data consists in a name, a path, a JSON pointer named ref and the value.
1. (Line 26) A in-file JSON pointer starts with “#/” and then each level is separated from its neighbour by a “/”. This is easily achieved by using + and join. But a JSON pointer cannot contains “/”. That’s why we use map_values in order to replace them by ~1 with gsub. The map_values works like map but do not return result in an array and therefore allows to do in place modification.
2. (Line 42) In order to get the extension value we use getpath on the saved document. Note how we have to define a $path variable to use it in getpath.

list-xtensions.jq

# 1 - Stores document for later use
#----------------------------------
. as $document # Variable used on line 23 to get 
               # extension value from its path
# 2 - Lists extensions paths
#---------------------------
| [
  paths # Lists ALL possible paths in documents 
        # (each path is represented as an array)
  | select( # Keeps only the values for which 
            # what follows return true
    .[-1] # Gets the path leaf (last item in array)
          # Equivalent to .[.|length-1]
    | tostring # Converts to string for next step 
    | test("^x-") # Checks if leaf name starts with x-
  )
]
# 3 - Sets all data for each extension occurence
#-----------------------------------------------
# returns an array of {name, path, ref, value}
| map( # Applies a filter to each element
  {
    name: .[-1], # Gets the path leaf (last item in array)
                 # Equivalent to .[.|length-1]
    path: .,
    # 3.1 - Creates a JSON pointer to extension
    #------------------------------------------
    ref: (
      "#/" + # adds numbers, strings, arrays or objects
      (
        . 
        | map_values( # Applies a filter on each value
                      # (in place modification)
          gsub("/";"~1" ) # replaces a value in a string
                          # / must be replace by ~1
                          # in a JSON pointer
        )
        | join("/") # concatenates string with 
                  # a separator
      )
    ), 
    # 3.2 - Gets extension value from original document
    #--------------------------------------------------
    value: (
      . as $path # storing value path in 
                 # a variable for next step
      | $document | getpath($path) # extracting value 
                                   # from original document
                                   # variable defined on line 3
    )
  }
)

Process multiple OpenAPI files

So, we have learned to use JQ filters on a single OpenAPI file, but what if we need to work on multiple files? In this section we’ll learn how to invoke JQ on multiple files and see it in action on two OpenAPI files.

Invoke JQ on multiple files

JQ’s filename parameter can contain wildcards, allowing to work on multiple files at once.

Invoke JQ on multiple files

We can, for example, extract the API name of each OpenAPI file using the following command as shown in the following listing (the github repository contains two OpenAPI demo files, both having the .json extension).

jq -r '.info.title' *.json

Processing multiple files with JQ

[apihandyman.io]$ jq -r '.info.title' *.json 
Banking API
Another Example API

That’ looks good, but if the filter outputs JSON, the result is a concatenation of the JSONs returned for each file, which is not a valid JSON document, as shown on line 1 of the following listing. In order to get something valid, like an array containing all results, you can pipe this result to a jq -s command (line 11) which will magically creates a valid JSON array. The -s flag (or --slurp) reads the entire input stream into a large array and run the filter just once instead of running the filter for each JSON object in the input. Not also that we didn’t provide any filter to the second JQ command. The ‘.’ filter is actually optional (either you use the -s flag or not).

jq '{name: .info.title, file: .info.version}' *.json
jq '{name: .info.title, file: .info.version}' *.json | jq -s

Pipe to jq -s (or --slurp) to create arrays

[apihandyman.io]$ jq '{name: .info.title, file: .info.version}' *.json
{
  "name": "Another API",
  "file": "1.2"
}
{
  "name": "Banking API",
  "file": "1.0.0-snapshot"
}

[apihandyman.io]$ jq '{name: .info.title, file: .info.version}' *.json \
                | jq -s
[
  {
    "name": "Another API",
    "file": "1.2"
  },
  {
    "name": "Banking API",
    "file": "1.0.0-snapshot"
  }
]

Obviously, when it comes to work with multiple elements on the command line, you can use your favorite commands such as xargs and/or find as shown below.

ls *.json | xargs jq -r '.info.title'
find . -type f -name "*.json" -exec jq -r '.info.title' {} \;
find . -type f -name "*.json" | xargs jq -r '.info.title'

Use JQ with xargs and find

[apihandyman.io]$ ls *.json | \
                  xargs jq -r '.info.title'
Banking API
Another Example API

[apihandyman.io]$ find . -type f -name "*.json" -exec \
                  jq -r '.info.title' {} \;
Banking API
Another API

[apihandyman.io]$ find . -type f -name "*.json" | \
                xargs jq -r '.info.title'
Banking API
Another API
# Note that find -exec is far less faster than
# find | xargs when working a large number of files

List basic API information from multiple files

For this last (but not least) example, we’ll gather basic information from different OpenAPI files.

List APIs

We’ll build an array of objects containing for each file:

Information about the file itself (its type, version and name),
A subset of the info section (API’s name, version and a shorten description)
The number of operations

Note that now jq is used on *.json files and its results is piped into another jq with -s flag in order to generate an array (as seen in previous section).

jq -f list-apis.jq *.json | jq -s

Getting some basic information about different APIs

[apihandyman.io]$ jq -f list-apis.jq *.json | jq -s
[
  {
    "specification": {
      "type": "swagger",
      "version": "2.0",
      "file": "demo-another-api-swagger.json"
    },
    "name": "Another API",
    "version": "1.2",
    "summary": "Does almost nothing",
    "operations": 1
  },
  {
    "specification": {
      "type": "openapi",
      "version": "3.0.0",
      "file": "demo-api-openapi.json"
    },
    "name": "Banking API",
    "version": "1.0.0-snapshot",
    "summary": "The Banking API provides access to the [Banking Company](http://www.bankingcompany.com) services, which include bank account information, beneficiaries, and money transfer management",
    "operations": 14
  }
]

To get that result, we need to use the following new JQ filters:

JQ Filters
`input_filename`	Returns the name of the file being processed
`indices(string)` `indices("foo")` `indices(.foo)`	Returns the list indices (in array of string) where a string is found
`.[start:end]` `.[10:15]`	Returns a subset of an array or a substring

The following JQ scripts which generates an array of objects containing information about the specification file itself, the API and its number of operations is composed of 3 parts:

(Line 2) Part 1 deals with file information. When working on multiple files, it can be very interesting to know from which file comes the data. It’s the case here, hopefully, the input_filename returns the name of the file being processed (line 8).
(Line 11) Part 2 deals with data coming from the info section. The summary is a shorter version of .info.description. If it contains a  tag (found using indices) we split right before it using .[0:tag position]. If not we take the first hundred characters (or the whole string if shorter). Note how elif is used to have more conditions.
(Line 28) Part 3 concerns counting operations, it is done almost like counting HTTP status codes.

list-apis.jq

{ 
  # 1 - Information about the file itself
  #--------------------------------------
  specification: {
    # Determines the type of specification and the version used
    type: (if has("openapi") then "openapi" else "swagger" end),
    version: (if has("openapi") then .openapi else .swagger end),
    file: input_filename # The file name because we work
                         # on multiple files
  },
  # 2 - Information about the API (.info)
  #--------------------------------------
  name: .info.title,
  version: .info.version,
  summary: (
    # indices returns an array containing all indices of the
    # provided string found in the input value
    (.info.description | indices("<!--more-->")[0]) as $more |
    if $more != null then 
      .info.description[0:$more]
      # summary cannot be longer than 100 characters
    elif (.info.description | length) <= 100 then
      .info.description
    else
      .info.description[0:100] + "[...]"
    end
  ),
  # 3 - Number of operations (an operation is get /path for example)
  #-----------------------------------------------------------------
  operations: (
    [ # Creates an array containing all HTTP methods
      # to count the number of operations
      .paths[] | # Returns the content of eah path object
      keys[] | # Returns the keys of the returned object
               # and flattens the array
      select( # Keeps only the value for which what follows is true
         IN("get", "put", "patch", "post", 
            "delete", "head", "options")
      )
    ] | length # Return the length of the array,
               # hence the number of operations
  )
}

Summary

That’s it for this first JQ and OpenAPI post. You know now how to invoke JQ on one or more files and you know how to use the 30ish following JQ filters. These are only a subset of all available filters, check JQ’s documentation to discover them all.

JQ Filters
`.`	Returns what is provided
`.foo` `.foo.bar`	Returns element targeted by period separated path
`.foo?`	Does not return an error if targeted element does not exist
`keys`	Returns object's keys (properties names)
`{}` `{name1: filter, name2: another_filter}`	Creates an object
`[]` `[1, 2, 3]` `[.foo, .bar]` `[.items[].name]`	Creates an array
`map(filter)`	Applies a filter to each element of an array
`select(filter returning boolean)`	Returns element for which filter returns true
`IN(value1, value2, value3)`	Returns true if element is one of the values
`test(regex)` `test(regex;flags)`	Returns true if element matches regex
`not`	Returns opposite boolean element
`group_by(path_expression)` `group_by(.foo)`	Groups array elements according to provided path (returns array of array)
`sort_by`	Returns element sort by value or path_expression
`length`	Returns the length of element (array, object, string, null)
`tostring`	Turns any element into a string
`to_entries`	Takes an object and returns an array of key and value objects
`.foo as $some_variable`	Defines a reusable variable
`if condition then filter else another_filter end`	Conditions filter execution
`paths`	Lists all possible paths in documents, each path is represented as an array
`getpath(path)` `getpath(["a","path"])`	Returns value corresponding to a path (defined as an array), see also paths
`map_values(filter)`	Updates all provided values with filter
`+` `value1 + value2`	Adds/concatenates/merges numbers/strings/arrays or objects
`gsub(regex;replacement)` `gsub("toreplace";"replaced)`	Replaces a string within a string
`join(separator)` `join(",")`	Joins string with a separator
`indices(string)` `indices("foo")` `indices(.foo)`	Returns the list indices (in array of string) where a string is found
`.[]` `.[index]`	Returns all or some elements of an object or array identified by an index
`.[start:end]` `.[10:15]`	Returns a subset of an array or a substring
`input_filename`	Returns the name of the file being processed

You may also have learn a few things about an OpenAPI document structure. If you want to fully master it, look at the OpenAPI Map.

What’s next

In next post, we’ll learn to search into OpenAPI files and simplify JQ code by using command line arguments, functions and modules.