# Using JQ in PixieBrix

{% hint style="info" %}
New User? JavaScript is easier to learn and is now the preferred way to transform data in PixieBrix: [using-javascript-in-pixiebrix](https://docs.pixiebrix.com/developing-mods/developer-concepts/transforming-data/using-javascript-in-pixiebrix "mention")
{% endhint %}

## What is jq?

jq is a powerful expression language for filtering and transforming structured JSON data. It's available for the command line and [a variety of programming languages](https://github.com/stedolan/jq/wiki/FAQ#language-bindings), including Java, Python, and Javascript.

## Helpful Resources

* [jqplay: A playground for jq 1.6](https://jqplay.org/)
* [jq Manual](https://stedolan.github.io/jq/manual/): has runnable examples of each operator
* [jq Homepage](https://stedolan.github.io/jq/)
* [jq tag on StackOverflow](https://stackoverflow.com/questions/tagged/jq)

## Brick Arguments

The jq brick takes two input arguments:

* `data`: an object, array, or single value which is passed to jq
* `filter`: the jq "filter", which controls how to filter/transform the data

#### Brick Argument Examples

**Scenario #1: passing a variable for data**

You can pass a variable or template directly to data:

```yaml
id: "@pixiebrix/jq"
config:
  # @apiResult = {items: []}
  data: "@apiResult"
  filter: ".items"
```

**Scenario #2: passing multiple variables to data**

You can also pass multiple variables by passing a dictionary to data, including using template syntax:

```yaml
id: "@pixiebrix/jq"
config:
  data: 
    # @apiResult = {items: [{text: "foo"}]}
    result: "@apiResult"
    # @input = {query: "foo"}
    query: "{{ @input.query }}!"
  filter: ".query as $query | [.result.items[] | select(.text == $query)]"
```

## Common Pitfalls

* jq only has access to the data you passed in via the `data` argument. You should not use `@variable` syntax in the jq `filter`
* jq uses a `.` to refer to its input (the data you passed to the brick)
* jq can return multiple results. However, PixieBrix expects a single result from jq. Therefore, you must wrap an expression that returns multiple results in Array construction `[]` operator

## Filter Examples

**Creating a new object from the input**

▶️  [View on jqplay](https://jqplay.org/s/fDr44s8zoA)

```
{foo: 42, bar: .}
```

**Parsing and summing values**

▶️  [View on jqplay](https://jqplay.org/s/DGSZvfgt6a)

```
.transactions | map(.amount | capture("(?<val>\\\\d+) USD") | .val | tonumber) | add
```

**Filtering an array**

▶️  [View on jqplay](https://jqplay.org/s/L2GTYwQNqO)

Assign the query to a variable so you can use it in the `select` clause.

NOTE: jq's `select` operator returns multiple results. Wrap the expression in an Array Construction `[]` operator to return the results as a single array

```
.query as $query | [.items[] | select(.text == $query)]
```

**Looking up a value in a pre-defined map (Workshop Example)**

▶️  [View on jqplay](https://jqplay.org/s/w9fCJwgroD)

Pass the pre-defined map as an input to the jq brick. Then, assign the map to a variable (e.g., `$m`) so you can lookup up the value later in the filter:

```yaml
data:
  # ["A", "B", "C"]
  items: "@items"
  mapping:
    A: Alpha
    B: Bravo
    C: Charlie
filter: ".mapping as $m | .items | map($m[.])"
```

You can use jq's `//` defaulting operator to handle elements that aren't in the map. E.g.,

```yaml
filter: '.mapping as $m | .items | map($m[.] // "Unknown")'
```

**Writing multi-line expressions (Workshop Example)**

You can make long jq filters in the Workshop more readable by splitting them over multiple lines. To split a filter over multiple lines, use the `>-` YAML operator that replaces newlines with spaces, and strips off the last new line ([Read more about multi-line YAML strings](https://yaml-multiline.info/))

For example, to clean up the filter in the "Looking up a value in a pre-defined map" example above:

```yaml
filter: >-
  .mapping as $m |
  .items |
  map($m[.])
```

Multi-line expressions are also helpful for constructing objects with many properties. (Note that JQ requires parentheses around the object's values):

```yaml
filter: >-
  {
    fooCheck: (.foo > 42),
    barCheck: (.bar | length > 0),
  }
```