JQ and OpenAPI Series - Part 2

Using JQ command line arguments, functions and modules

By Arnaud Lauret, February 3, 2020

Ever wanted to quickly find, extract or modify data coming from some JSON documents on the command line? JQ is the tool you’re looking for. In the previous part of this JQ and OpenAPI Series, we learned to invoke JQ and how to extract data from JSON documents using some of its many filters. Now we will discover how to build flexible and easily reusable JQ filters by creating functions and modules and also using command line arguments. We will continue working on OpenAPI files, at the end of this second part, we’ll have built a multi-criteria OpenAPI search and some reusable filters, especially one that you’ll be able to reuse anytime you’ll have to deal with JQ command line parameters.

JQ and OpenAPI Series

JQ’s documentation is quite complete and there are many tutorials and Stackoverflow answers, so why bother writing this series? First reason, I regularly meet people working with APIs and/or JSON files who actually don’t know JQ exists and how it could save their life (or at least their time). Second reason, I often use it with OpenAPI specification files and I found that showing how JQ can be used on such a widely adopted and familiar JSON based format could help to learn how to use it (and also writing this post actually helped me to improve my JQ skills!).

1 - Using JQ to extract data from OpenAPI files
2 - Using JQ command line arguments, functions and modules
3 - Modifying OpenAPI files with JQ
4 - Bonus: Coloring JQ's raw output

Get post’s content

All examples shown in this post are based on JQ 1.6 and OpenAPI 3. All examples can be copied using the button and downloaded using the one on code snippets. All source code can be retrieved from the JQ and OpenAPI post series’ github repository.

git clone https://github.com/arno-di-loreto/jq-and-openapi/
cd jq-and-openapi
git checkout part-2

[apihandyman.io]$ git clone https://github.com/arno-di-loreto/jq-and-openapi/
[apihandyman.io]$ cd jq-and-openapi
[apihandyman.io]$ git checkout part-2

Listing operations using functions and modules

In previous post, we built a filter that lists the operations available in an OpenAPI file. In this first section, we will just refactor the JQ code to make it more readable and reusable using functions and modules. The following listing shows what happens when using the new version of list-operations.jq on the demo OpenAPI file.

jq -r -f list-operations.jq demo-api-openapi.json

Same result as in part 1 but list-operations.jq has changed under the hood

[apihandyman.io]$ jq -r -f list-operations.jq demo-api-openapi.json
get     /accounts       List accounts
get     /accounts/{id}  Get an account
post    /beneficiaries  Register a beneficiary
get     /beneficiaries  List beneficiaries
delete  /beneficiaries/{id}     Delete a beneficiary (deprecated)
patch   /beneficiaries/{id}     Updates a beneficiary (deprecated)
get     /beneficiaries/{id}     Get a beneficiary
get     /sources        List transfer sources
get     /sources/{id}/destinations      List transfer source's destinations
post    /transfers      Transfer money
get     /transfers      List money transfers
get     /transfers/{id} Get a money transfer
patch   /transfers/{id}
delete  /transfers/{id} Cancel a money transfer

It seems nothing has changed, it still outputs operations HTTP methods, paths and summaries, but under the hood, the JQ file used has changed as shown in the following listing.

list-operations.jq

include "module-openapi"; # Imports module-openapi.jq file

oas_operations | # Function coming from module-openapi.jq file
oas_operations_to_text  # Function coming from module-openapi.jq file

Let’s see how it was created, we’ll discover functions and then modules.

Creating functions

As a reminder, here’ the previous version of the list-operations.jq file we created in previous part. It is composed of three steps. Steps 1 and 2 build an array of operation object containing a (HTTP) method, path, summary and deprecated indicator. Step 3 aims to print this array as tab separated text.

list-operations-original.jq

# 1 - Selects paths objects
#--------------------------
# returns [{key: path, value: path value}]
.paths # Selects the paths property content
| to_entries # Transforms
             # { "/resources": { "get": {operation data}}} 
             # to 
             # [ { "key": "/resources", 
             #     "value": { "get": {operation data}} ]
| map(select(.key | test("^x-") | not)) # Gets rid of x-tensions
# 2 - Creates an array of operations
#-----------------------------------
# returns [{path, method, summary, deprecated}]
| map ( # Applies a transformation to each element
  .key as $path # Stores the path value (.key) 
                  # in a variable ($path) for later use
  | .value # Keeps only the path's content 
           # { "get": {operation data}}
  | to_entries # Transforms 
               # { "get": {operation data}}
               # to
               # [ { "key": "get", 
               #     "value": {operation data}} ]
  | map( # Applies a transformation to each element
    select( # Keeps only elements for which the following is true
      # With IN, which returns true if the value is one of its
      # parameters, we can get rid of x- , parameters
      # description and summary properties
      .key | IN("get", "put", "post", "delete", 
         "options", "head", "patch", "trace")
    )
    | # Creates a new JSON object
    {
      method: .key,
      path: $path, # Using the variable defined on line 4
      summary: .value.summary?,
      deprecated: .value.deprecated?
    }
  )[] # Flattens array to avoid having an array 
      # of array of {path, method, summary, deprecated}
) # Now we have an array of {path, method, summary, deprecated}
# 3 - Outputs tab separated raw text
#-----------------------------------
| map( # Applies a transformation to each element
  .method + "\t" + 
  .path + "\t" + 
  .summary + 
  (if .deprecated then " (deprecated)" else "" end)
)
[] # Flattens array for raw output

Let’s focus on step 3, which is shown below, and build a function that does the same job.

We will create a function for step 3

# 3 - Outputs tab separated raw text
#-----------------------------------
| map( # Applies a transformation to each element
  .method + "\t" + 
  .path + "\t" + 
  .summary + 
  (if .deprecated then " (deprecated)" else "" end)
)
[] # Flattens array for raw output

Defining a function in JQ is quite simple: at the beginning of the file, add a def function_name: put some filters and end by ; and you’re done. The oas_operation_to_text which basically contains step 3’s filters is shown below.

Defining the oas_operation_to_text function

def oas_operations_to_text: # Defining a function that
                            # Prints operations as raw text
  map( # Applies a transformation to each element
    .method + "\t" + 
    .path + "\t" + 
    .summary + 
    (if .deprecated then " (deprecated)" else "" end)
  )
  [] # Flattens array for raw output
; # oas_operations_to_text function's end

If defining a function in JQ is quite simple, using it is even more simple. Just call it like any regular JQ filter. The following listing shows how step 3’s code has been replaced by the new oas_operation_to_text custom filter which is on top of the file.

Using the oas_operation_to_text function

# 3 - Outputs tab separated raw text
#-----------------------------------
| oas_operations_to_text

Here’s the full modified list-operations.jq file including the oas_operation_to_text definition at the beginning and its calling on the last line.

list-operations-with-to-text-function.jq

def oas_operations_to_text: # Defining a function that
                            # Prints operations as raw text
  map( # Applies a transformation to each element
    .method + "\t" + 
    .path + "\t" + 
    .summary + 
    (if .deprecated then " (deprecated)" else "" end)
  )
  [] # Flattens array for raw output
; # oas_operations_to_text function's end

# 1 - Selects paths objects
#--------------------------
# returns [{key: path, value: path value}]
.paths # Selects the paths property content
| to_entries # Transforms
             # { "/resources": { "get": {operation data}}} 
             # to 
             # [ { "key": "/resources", 
             #     "value": { "get": {operation data}} ]
| map(select(.key | test("^x-") | not)) # Gets rid of x-tensions
# 2 - Creates an array of operations
#-----------------------------------
# returns [{path, method, summary, deprecated}]
| map ( # Applies a transformation to each element
  .key as $path # Stores the path value (.key) 
                  # in a variable ($path) for later use
  | .value # Keeps only the path's content 
           # { "get": {operation data}}
  | to_entries # Transforms 
               # { "get": {operation data}}
               # to
               # [ { "key": "get", 
               #     "value": {operation data}} ]
  | map( # Applies a transformation to each element
    select( # Keeps only elements for which the following is true
      # With IN, which returns true if the value is one of its
      # parameters, we can get rid of x- , parameters
      # description and summary properties
      .key | IN("get", "put", "post", "delete", 
         "options", "head", "patch", "trace")
    )
    | # Creates a new JSON object
    {
      method: .key,
      path: $path, # Using the variable defined on line 4
      summary: .value.summary?,
      deprecated: .value.deprecated?
    }
  )[] # Flattens array to avoid having an array 
      # of array of {path, method, summary, deprecated}
) # Now we have an array of {path, method, summary, deprecated}
# 3 - Outputs tab separated raw text
#-----------------------------------
| oas_operations_to_text

That’s great, using functions big JQ filters are far more readable. But what about being able to reuse these functions?

Creating a module with reusable functions

Creating JQ modules that define reusable functions is, again, quite simple. Just put some functions in a JQ file and you’re done. The following listing shows a module-openapi.jq module file defining two functions. There’s the oas_operation_to_text we have just created and also an oas_operations which do the same as steps 1 and 2 of the list-operations.jq file (returning an array of operations). Note that there’s a light modification (line 43/44), this function returns also the input_filename and the original value of each operations (for a later use) besides its HTTP method, path, summary and deprecated flag.

module-openapi.jq

# This is a reusable JQ module defining useful
# OpenAPI specification (OAS) processing functions

def oas_operations: # Defining a listoperations function
                    # returning {path, method, summary, original}
  # 1 - Selects paths objects
  #--------------------------
  # returns [{key: path, value: path value}]
  .paths # Selects the paths property content
  | to_entries # Transforms
              # { "/resources": { "get": {operation data}}} 
              # to 
              # [ { "key": "/resources", 
              #     "value": { "get": {operation data}} ]
  | map(select(.key | test("^x-") | not)) # Gets rid of x-tensions
  # 2 - Creates an array of operations
  #-----------------------------------
  # returns [{path, method, summary, deprecated}]
  | map ( # Applies a transformation to each element
    .key as $path # Stores the path value (.key) 
                    # in a variable ($path) for later use
    | .value # Keeps only the path's content 
            # { "get": {operation data}}
    | to_entries # Transforms 
                # { "get": {operation data}}
                # to
                # [ { "key": "get", 
                #     "value": {operation data}} ]
    | map( # Applies a transformation to each element
      select( # Keeps only elements for which the following is true
        # With IN, which returns true if the value is one of its
        # parameters, we can get rid of x- , parameters
        # description and summary properties
        .key | IN("get", "put", "post", "delete", 
          "options", "head", "patch", "trace")
      )
      | # Creates a new JSON object
      {
        method: .key,
        path: $path, # Using the variable defined on line 4
        summary: .value.summary?,
        deprecated: .value.deprecated?,
        original: .value, # Keeping original value, just in case 😉
        source: input_filename # Adding source file, also just in case 😉
      }
    )[] # Flattens array to avoid having an array 
        # of array of {path, method, summary, deprecated}
  ) # Now we have an array of {path, method, summary, deprecated}
; # oas_operations function's end

def oas_operations_to_text: # Defining a function that
                            # Prints operations as raw text
  map( # Applies a transformation to each element
    .method + "\t" + 
    .path + "\t" + 
    .summary + 
    (if .deprecated then " (deprecated)" else "" end)
  )
  [] # Flattens array for raw output
; # oas_operations_to_text function's end

Let’s get back to the new version of list-operations.jq (shown below) to see how this module is actually used. The module is include with the include <module name without extension>; line. Then any functions defined in it can be used like any other regular JQ filter as shown on line 3 and 4 where oas_operations and oas_operations_to_text are used.

list-operations.jq

include "module-openapi"; # Imports module-openapi.jq file

oas_operations | # Function coming from module-openapi.jq file
oas_operations_to_text  # Function coming from module-openapi.jq file

Managing modules locations

Managing JQ modules location

The following listings shows different ways of managing reusable modules location with JQ (see modules in the JQ’s documentation for a complete description of what can be done). It starts by a a first command done inside the jq-and-openapi folder. It simply returns the first operation’s summary of the demo-api-openapi.json file using the oas_operations[0] filter composed of the oas_operations function and the [] array filter. As you can see, there’s no need to create a JQ file to use a module, just use the include directive in the '<filter>' argument on the command line. Then we go a level up, and obviously redoing the same exact command does not work anymore: the module-openapi.jq cannot be found in the current folder as it is in the jq-and-openapi one. Hopefully, you can use the -L <path list> argument to tell JQ where to look for modules.

jq -r 'include "module-openapi"; oas_operations[0].summary' demo-api-openapi.json
cd ..
jq -r 'include "module-openapi"; oas_operations[0].summary' jq-and-openapi/demo-api-openapi.json
jq -r -L jq-and-openapi 'include "module-openapi"; oas_operations[0].summary' jq-and-openapi/demo-api-openapi.json

Indicating where to find modules with -L argument

[apihandyman.io]$ jq -r 'include "module-openapi"; oas_operations[0].summary' demo-api-openapi.json
List accounts
[apihandyman.io]$ cd ..
[apihandyman.io]$ jq -r 'include "module-openapi"; oas_operations[0].summary' jq-and-openapi/demo-api-openapi.json
jq: error: module not found: module-openapi

jq: 1 compile error
[apihandyman.io]$ jq -r -L jq-and-openapi 'include "module-openapi"; oas_operations[0].summary' jq-and-openapi/demo-api-openapi.json
List accounts

If there are modules that you use extensively, it would be interesting to put them in a ~/.jq folder. Therefore, no longer need for the -L argument as shown below. JQ looks for the modules mentioned in include directives in this folder automatically.

mkdir ~/.jq
cp jq-and-openapi/module-openapi.jq ~/.jq
jq -r 'include "module-openapi"; oas_operations[0].summary' jq-and-openapi/demo-api-openapi.json

Using ~/.jq default folder to store modules

[apihandyman.io]$ mkdir ~/.jq
[apihandyman.io]$ cp jq-and-openapi/module-openapi.jq ~/.jq
[apihandyman.io]$ jq -r 'include "module-openapi"; oas_operations[0].summary' jq-and-openapi/demo-api-openapi.json
List accounts

Note that ~/.jq can also be a file. In that case, you don’t even need to include anything, as shown below. Any function defined in this file is usable inside any of your filters. I personally do not recommend to do this because that makes your filters dependencies invisible (and can also result in a quite huge unmaintainable .jq file).

rm -rf ~/.jq
cp jq-and-openapi/module-openapi.jq ~/.jq
jq -r 'oas_operations[0].summary' jq-and-openapi/demo-api-openapi.json

Using ~/.jq default file to store functions

[apihandyman.io]$ rm -rf ~/.jq
[apihandyman.io]$ cp jq-and-openapi/module-openapi.jq ~/.jq
[apihandyman.io]$ jq -r 'oas_operations[0].summary' jq-and-openapi/demo-api-openapi.json
List accounts

Searching operations using command line arguments

Now that we have a reusable module that provides functions to list operations of an OpenAPI specification file and print them as tab separated text, let’s work on a multiple-criteria and multiple-file search.

Passing an argument to JQ filters

In order to make this search flexible, we’ll need to be able to accept search arguments coming from outside our filter in order to avoid having to modify it on each different search. Passing arguments to JQ is done with --arg <name> <value> as shown below. Inside the filter, you can access a --arg with $<name>. In this case $foo returns bar. Note also in this example the -n flag which is used to tell JQ to not expect any JSON input. That’s pretty useful to make demos of some JQ’s features but also to generate JSON from scratched based on some arguments values.

jq -n --arg foo bar '{foo: $foo}'

Passing an argument with --arg (and discovering -n flag)

[apihandyman.io]$ jq -n --arg foo bar '{foo: $foo}'
{
  "foo": "bar"
}

Searching operations accessible for a scope

The following listing shows which operations are accessible to a consumer when it is given the transfer:admin security scope. The scope value is provided to the filter using --arg <name> <value>.

jq -r --arg scope transfer:admin -f search-operations-using-scope.jq demo-api-openapi.json

Searching operations using a given scope

[apihandyman.io]$ jq -r --arg scope transfer:admin -f search-operations-using-scope.jq demo-api-openapi.json 
post    /transfers      Transfer money
get     /transfers      List money transfers
get     /transfers/{id} Get a money transfer
delete  /transfers/{id} Cancel a money transfer

In an OpenAPI file, you’ll find the scopes that will grant access to an operation in its security property under a {name}. According the OpenAPI Specification, each name MUST correspond to a security scheme which is declared in the Security Schemes under the Components Object. If the security scheme is of type “oauth2” or “openIdConnect”, then the value is a list of scope names required for the execution. For other security scheme types, the array MUST be empty.

For our use case, we just need to list all values (scopes) under all security.{name} of each operation and keep the operations for which the provided scope is found in this list. The following listing shows how this is achieved in the search-operations-using-scope.jq.

First (line 4), it lists existing operations using the oas_operations function (coming from module-openapi included on line 1)
Then (line 5), it filters the returned operations based on their scopes by working on each of the original operation’s data coming from the OpenAPI file. To do so:
- It first checks if there’s a security property (line 7)
- Then creates a list of scopes (line 9 and 10)
- And (line 11 to 14), if the index of $scope (provided through the --arg scope <value>) is greater than 0 (meaning it is in the list), the operation is returned
And finally (line 19), it prints the remaining operations as tab separated values using the oas_operations_to_text function (coming from module-openapi included on line 1)

search-operations-using-scope.jq

include "module-openapi"; # Looks for a module-openapi.jq file

# Expects a --arg scope value parameter
oas_operations # Comes from module-operations.jq
| map(select( # Filters on operation scopes
    # security is not always present
    if .original.security? != null then
      # Creating an array containg all scopes
      [ .original.security | 
        map(to_entries | map(.value)[])[][] ] | 
      index( # Index returns the index of a value in array
        $scope # $scope value is provided on the command line
              # --arg scope value
      ) >= 0 # If < 0, it has not been found
    else
      false # No security defined, so return false
    end
  ))
| oas_operations_to_text  # Comes from module-operations.jq

That’s cool, but there’s a little problem. When using the search-operations-using-scope.jq without providing the scope value, it does not work: JQ complains that $scope is not defined, as shown below.

jq -r -f search-operations-using-scope.jq demo-api-openapi.json

What happens when scope is not provided

[apihandyman.io]$ jq -r -f search-operations-using-scope.jq demo-api-openapi.json
jq: error: $scope is not defined at <top-level>, line 12:
        $scope # $scope value is provided on the command line        
jq: 1 compile error

Does that mean we can’t do a multi-criteria search because it requires to be able to provide multiple optional parameters? Of course not, that problem can be solved.

Solving the command line argument “problem”

Solving the command line argument problem

The following listing shows how to safely access a command line named argument using the $ARGS.named filter. If $name causes an error if no --arg name value is provided on the command line, $ARGS.named['name'] will return null without causing any.

jq -n --arg foo hello --arg bar world '{foo: $foo, bar: $bar}'
jq -n --arg foo hello '{foo: $foo, bar: $bar}'
jq -n --arg foo hello '{foo: $ARGS.named["foo"], bar: $ARGS.named["bar"]}'

Using $ARGS.named

[apihandyman.io]$ jq -n --arg foo hello --arg bar world '{foo: $foo, bar: $bar}'
{
  "foo": "hello",
  "bar": "world"
}
[apihandyman.io]$ jq -n --arg foo hello '{foo: $foo, bar: $bar}'
jq: error: $bar is not defined at <top-level>, line 1:
{foo: $foo, bar: $bar}                 
jq: 1 compile error
[apihandyman.io]$ jq -n --arg foo hello '{foo: $ARGS.named["foo"], bar: $ARGS.named["bar"]}'
{
  "foo": "hello",
  "bar": null
}

That’s very handy, but what if I want to set an argument to a default value if it is not provided? I just need to use the following module-args module. It defines a init_parameter(default_values) function returning an object containing parameters set to the value coming from --arg <name> or a default value it is not provided. To do so, for each entry (key/value) of a default_values object parameter, it checks if the named arguments ($ARGS.named) contains the key and if so, sets the output value to the one provided on the command line. If not, it keeps the default one. By the way, that means that JQ functions can also use parameters besides their regular input. But note that you don’t need to prefix their name by $ to access them.

module-args.jq

# Initializes parameters based on provided named arguments (--arg).
# If an argument is not provided, its default value is used.
# default_values example:
# {
#   argument: "default value",
#   anotherArgument: null,
# }
def init_parameters(default_values):
  default_values | 
  # Updates values for provided parameters
  with_entries(
    # $ARGS contains all --arg parameters
    if $ARGS.named[.key] != null then 
      .value = $ARGS.named[.key] 
    else 
      .value = .value
    end
  )
;

The following listing shows how this function can be used. Just call the init_parameter function with an object containing the default values and put its result in a variable (here $parameter) for later use ($parameter.foo for example). Here the default value of foo is default foo and bar’s is null. Only bar is provided, so the output contains foo’s default value and bar command-line-provided value.

jq -n --arg bar "bar from command line" 'include "module-args"; init_parameters({foo: "default foo", bar: null}) as $parameters| {foo: $parameters.foo, bar: $parameters.bar}'

Optional parameters with default values

[apihandyman.io]$ jq -n --arg bar "bar from command line" 'include "module-args"; init_parameters({foo: "default foo", bar: null}) as $parameters| {foo: $parameters.foo, bar: $parameters.bar}'
{
  "foo": "default foo",
  "bar": "bar from command line"
}

Searching operations on multiple criteria and multiple files

Searching operations demo

Now that we know how to provide multiple optional parameters, let’s do a multi-criteria search. The following listing shows the get operations on paths containing sources across all available *.json files. The first value on each line is the filename (limited to 20 characters).

jq --arg path_contains sources --arg method get -r -f search-operations.jq *.json

Operations on path containing source with method get

[apihandyman.io]$ jq --arg path_contains sources --arg method get -r -f search-operations.jq *.json
[demo-another-api-swa]  get     /resources
[demo-api-openapi.jso]  get     /sources        List transfer sources
[demo-api-openapi.jso]  get     /sources/{id}/destinations      List transfer source's destinations

Here’s the search-operations.jq file who does that. It reuses functions we have seen before, oas_operations from the module_openapi.jq file and init_parameters from the module-args.jq file. It also uses new functions filter_operations, default_filters, print_oas_operations and default_print_parameters from module-openapi-search.jq. There are 3 steps: getting operations data, filtering them and finally printing them. There’s nothing new on the first step, we already have used this function. Let’s see what is happening on the second and after that the third step.

search-operations.jq

include "module-openapi";
include "module-args";
include "module-openapi-search";

# Gets operations data
oas_operations
# Filters operations
| filter_oas_operations(init_parameters(default_filters))
# Prints operations
| print_oas_operations(init_parameters(default_print_parameters).format)

The following listing shows the new functions used to filter operations. The default_filters only returns the search filters default value to be used in conjunction with init_parameters and so get cleans values from optional command line arguments. The filter_oas_operation expects a filter object whose structure is the same as the one returned by default filters. This operations runs a map(select()) on the operations list. Each filter is triggered if filters.<name> is not null. There’s nothing really new regarding JQ’s filters besides line 41. The filtering on paths is done using the contains filter which we hadn’t seen before.

Filtering operations (module-openapi-search.jq)

# Available filters and their default values
# To be used with init_parameters
def default_filters:
{
  deprecated: null,
  method: null,
  code: null,
  scope: null,
  path_contains: null
};

# Filters operations coming from oas_operations
# Each filter is used only if corresponding filters.<name> parameter is provided
def filter_oas_operations(filters):
  map(
    select(
    # Filters on deprecated
    (filters.deprecated == null or 
      (.deprecated | tostring) == filters.deprecated) and
    # Filters on HTTP method
    (filters.method == null or 
      .method == filters.method) and
    # Filters on HTTP status code
    (filters.code == null or 
      (.original.responses | has(filters.code))) and
    # Filters on security scope
    (filters.scope == null or
      (if .value.security? != null then
        [ .value.security | 
          map(to_entries | 
          map(.value)[])[][]] | 
        index(filters.scope) >= 0
      else
        false
      end)
    ) and
    # Filters on path
    (filters.path_contains == null or 
      (.path | contains(filters.path_contains))
    )
  )
);

The following listing shows the new functions used to print the operations. It uses the same mechanism as the filter functions regarding the command line arguments.

Printing operations (module-openapi-search.jq)

# Same as oas_operations_to_text but with source
def oas_operations_to_text_with_source: 
  map( # Applies a transformation to each element
    "[" + .source[0:20] + "]\t" +
    .method + "\t" + 
    .path + "\t" + 
    .summary + 
    (if .deprecated then " (deprecated)" else "" end)
  )
  [] # Flattens array for raw output
; # oas_operations_to_text function's end

# To be used with init_parameters
def default_print_parameters:
{
  format: "text_with_source"
  # All values: 
  #  text_with_source, text_without_source, json_flat or null for json
};

# Prints oas_operations (filtered or not) in various format
def print_oas_operations(format):
  if format == "text_with_source" then
      oas_operations_to_text_with_source
  elif format == "text_without_source" then
      oas_operations_to_text
  elif format == "json_flat" then
    .[] # Flattening for multifiles, pipe result into a jq -s
  else
    .
  end
;

Here’s the full file:

Filtering operations (module-openapi-search.jq)

include "module-openapi";

# Available filters and their default values
# To be used with init_parameters
def default_filters:
{
  deprecated: null,
  method: null,
  code: null,
  scope: null,
  path_contains: null
};

# Filters operations coming from oas_operations
# Each filter is used only if corresponding filters.<name> parameter is provided
def filter_oas_operations(filters):
  map(
    select(
    # Filters on deprecated
    (filters.deprecated == null or 
      (.deprecated | tostring) == filters.deprecated) and
    # Filters on HTTP method
    (filters.method == null or 
      .method == filters.method) and
    # Filters on HTTP status code
    (filters.code == null or 
      (.original.responses | has(filters.code))) and
    # Filters on security scope
    (filters.scope == null or
      (if .value.security? != null then
        [ .value.security | 
          map(to_entries | 
          map(.value)[])[][]] | 
        index(filters.scope) >= 0
      else
        false
      end)
    ) and
    # Filters on path
    (filters.path_contains == null or 
      (.path | contains(filters.path_contains))
    )
  )
);

# Same as oas_operations_to_text but with source
def oas_operations_to_text_with_source: 
  map( # Applies a transformation to each element
    "[" + .source[0:20] + "]\t" +
    .method + "\t" + 
    .path + "\t" + 
    .summary + 
    (if .deprecated then " (deprecated)" else "" end)
  )
  [] # Flattens array for raw output
; # oas_operations_to_text function's end

# To be used with init_parameters
def default_print_parameters:
{
  format: "text_with_source"
  # All values: 
  #  text_with_source, text_without_source, json_flat or null for json
};

# Prints oas_operations (filtered or not) in various format
def print_oas_operations(format):
  if format == "text_with_source" then
      oas_operations_to_text_with_source
  elif format == "text_without_source" then
      oas_operations_to_text
  elif format == "json_flat" then
    .[] # Flattening for multifiles, pipe result into a jq -s
  else
    .
  end
;

Summary

That concludes this second path of the JQ and OpenAPI series. Here’s the summary of what we have seen in this post:

Functions and modules

Creating a function is done with def name: <filters>;
To invoke a function just use its name like for any regular filter
Functions can have parameters def name(parameter)
Inside a function a parameter can be used with parameter (without $)
A module is a JQ file containing reusable functions
A module is loaded using the include filename_without_extension directive
Use -L command line parameter to tell JQ where to find modules
Put your favorite modules in ~/.jq folder so JQ can find them without using -L

Command line arguments

Passing a named argument to JQ filters is done with --arg name value
A named argument value can be retrieved with $name
Using $name will provoke an error if no --arg name value is provided
All named arguments are available with $ARGS.named
$ARGS.named[name] returns null (wihout error) if no --arg name value is provided

The null argument

The -n (--null) arguments tells JQ to not expect input JSON

New filters

JQ Filters
`index(element)`	Returns the index of an element inside an array (-1 if not found)
`contains(element)` `"resources" \| contains("source")`	Returns true the element is in input
`$ARGS.named`	Returns the command line named argument (--arg name value)

What’s next

In next post, we’ll learn to modify OpenAPI files with JQ.