Comment on page

Using JQ in PixieBrix

What is jq?

jq is a powerful expression language for filtering and transforming structured JSON data. It's available for the command line and a variety of programming languages, including Java, Python, and Javascript.

Helpful Resources

Brick Arguments

The jq brick takes two input arguments:
  • data: an object, array, or single value which is passed to jq
  • filter: the jq "filter", which controls how to filter/transform the data

Brick Argument Examples

Scenario #1: passing a variable for data
You can pass a variable or template directly to data:
id: "@pixiebrix/jq"
# @apiResult = {items: []}
data: "@apiResult"
filter: ".items"
Scenario #2: passing multiple variables to data
You can also pass multiple variables by passing a dictionary to data, including using template syntax:
id: "@pixiebrix/jq"
# @apiResult = {items: [{text: "foo"}]}
result: "@apiResult"
# @input = {query: "foo"}
query: "{{ @input.query }}!"
filter: ".query as $query | [.result.items[] | select(.text == $query)]"

Common Pitfalls

  • jq only has access to the data you passed in via the data argument. You should not use @variable syntax in the jq filter
  • jq uses a . to refer to its input (the data you passed to the brick)
  • jq can return multiple results. However, PixieBrix expects a single result from jq. Therefore, you must wrap an expression that returns multiple results in Array construction [] operator

Filter Examples

Creating a new object from the input
▶️ View on jqplay
{foo: 42, bar: .}
Parsing and summing values
▶️ View on jqplay
.transactions | map(.amount | capture("(?<val>\\\\d+) USD") | .val | tonumber) | add
Filtering an array
▶️ View on jqplay
Assign the query to a variable so you can use it in the select clause.
NOTE: jq's select operator returns multiple results. Wrap the expression in an Array Construction [] operator to return the results as a single array
.query as $query | [.items[] | select(.text == $query)]
Looking up a value in a pre-defined map (Workshop Example)
▶️ View on jqplay
Pass the pre-defined map as an input to the jq brick. Then, assign the map to a variable (e.g., $m) so you can lookup up the value later in the filter:
# ["A", "B", "C"]
items: "@items"
A: Alpha
B: Bravo
C: Charlie
filter: ".mapping as $m | .items | map($m[.])"
You can use jq's // defaulting operator to handle elements that aren't in the map. E.g.,
filter: '.mapping as $m | .items | map($m[.] // "Unknown")'
Writing multi-line expressions (Workshop Example)
You can make long jq filters in the Workshop more readable by splitting them over multiple lines. To split a filter over multiple lines, use the >- YAML operator that replaces newlines with spaces, and strips off the last new line (Read more about multi-line YAML strings)
For example, to clean up the filter in the "Looking up a value in a pre-defined map" example above:
filter: >-
.mapping as $m |
.items |
Multi-line expressions are also helpful for constructing objects with many properties. (Note that JQ requires parentheses around the object's values):
filter: >-
fooCheck: (.foo > 42),
barCheck: (.bar | length > 0),