jq

Website: https://jqlang.github.io/jq/ CLI Tool: jq Authentication: N/A

Description

jq is a lightweight and flexible command-line JSON processor. It's like sed for JSON data - you can use it to slice, filter, map, and transform structured data. Essential for working with JSON APIs, log files, and configuration data.

Commands

Basic Usage

Pretty Print

jq . file.json
echo '{"name":"John","age":30}' | jq .
curl https://api.example.com | jq .

Pretty print JSON with syntax highlighting.

Compact Output

jq -c . file.json
jq --compact-output . file.json

Output JSON in compact form (single line).

Raw Output

jq -r .field file.json
jq --raw-output .name file.json

Output raw strings without JSON quotes.

Accessing Fields

Get Field

jq .field file.json
jq .name file.json
jq .user.email file.json

Access object fields using dot notation.

Nested Fields

jq .user.address.city file.json
jq .data.items[0].name file.json

Access nested object fields.

Optional Field

jq .field? file.json
jq .user.email? file.json

Optional access (no error if field missing).

Multiple Fields

jq .field1,.field2 file.json
jq .name,.email,.age file.json

Select multiple fields (output multiple values).

Arrays

Array Index

jq .[0] file.json
jq .[2] file.json
jq .items[0] file.json

Access array element by index.

Array Slice

jq .[1:3] file.json
jq .[0:5] file.json
jq .items[2:] file.json

Slice array (start:end, end exclusive).

Array Length

jq length file.json
jq .items | length
jq .[].tags | length

Get length of array or object.

All Elements

jq .[] file.json
jq .items[] file.json

Iterate over array elements.

First/Last

jq first file.json
jq last file.json
jq .items | first

Get first or last element.

Filtering

Select

jq '.[] | select(.age > 30)' file.json
jq '.users[] | select(.active == true)' file.json
jq 'select(.name == "John")' file.json

Filter elements based on condition.

Has Key

jq 'select(has("email"))' file.json
jq '.[] | select(has("optional_field"))' file.json

Filter objects that have a specific key.

Type Filter

jq '.[] | select(type == "string")' file.json
jq 'select(type == "number")' file.json

Filter by value type.

Empty Filter

jq 'select(.field != null)' file.json
jq '.[] | select(length > 0)' file.json

Filter out null or empty values.

Mapping

Map

jq 'map(.name)' file.json
jq '.items | map(.id)' file.json
jq '[.[] | .name]' file.json

Transform array elements.

Map Values

jq 'map_values(.+1)' file.json
jq 'map_values(. * 2)' file.json

Transform object values.

Map Select

jq '[.[] | select(.active) | .name]' file.json
jq 'map(select(.age > 18))' file.json

Filter and map in one operation.

Constructing Objects

Create Object

jq '{name: .name, id: .id}' file.json
jq '{user: .username, mail: .email}' file.json

Create new object with selected fields.

Rename Fields

jq '{username: .name, userEmail: .email}' file.json

Create object with renamed fields.

Add Field

jq '. + {newField: "value"}' file.json
jq '. + {timestamp: now}' file.json

Add new field to object.

Merge Objects

jq '. * {override: "value"}' file.json
jq '. + .extra' file.json

Merge objects (+ recursive, * non-recursive).

Constructing Arrays

Array Construction

jq '[.field1, .field2]' file.json
jq '[.users[].name]' file.json

Construct array from values.

Add to Array

jq '. + ["new"]' file.json
jq '.items + [.newItem]' file.json

Append to array.

Flatten

jq 'flatten' file.json
jq '[.[] | .tags] | flatten' file.json

Flatten nested arrays.

Unique

jq 'unique' file.json
jq '.tags | unique' file.json

Get unique values from array.

Sorting

Sort

jq 'sort' file.json
jq '.items | sort' file.json

Sort array (ascending).

Sort By

jq 'sort_by(.age)' file.json
jq '.users | sort_by(.name)' file.json

Sort array by field.

Reverse

jq 'reverse' file.json
jq '.items | sort | reverse' file.json

Reverse array order.

Grouping

Group By

jq 'group_by(.category)' file.json
jq '.items | group_by(.type)' file.json

Group array elements by field value.

Unique By

jq 'unique_by(.id)' file.json
jq '.users | unique_by(.email)' file.json

Remove duplicates based on field.

Aggregation

Min/Max

jq 'min' file.json
jq 'max' file.json
jq '.prices | min' file.json

Find minimum or maximum value.

Min By/Max By

jq 'min_by(.age)' file.json
jq '.users | max_by(.score)' file.json

Find element with min/max field value.

Sum

jq 'add' file.json
jq '.prices | add' file.json
jq '[.[] | .quantity] | add' file.json

Sum array values.

Count

jq '[.[] | select(.active)] | length' file.json

Count filtered elements.

String Operations

String Interpolation

jq '"Hello \(.name)"' file.json
jq '"\(.first) \(.last)"' file.json

Interpolate values into string.

Split

jq 'split(",")' file.json
jq '.tags | split(";")' file.json

Split string into array.

Join

jq 'join(",")' file.json
jq '.items | join(" | ")' file.json

Join array into string.

Test (Regex)

jq 'test("pattern")' file.json
jq '.email | test("@gmail\\.com$")' file.json

Test if string matches regex.

Match

jq 'match("pattern")' file.json
jq '.text | match("\\d+")' file.json

Extract regex matches.

Replace

jq 'gsub("old"; "new")' file.json
jq '.text | gsub("\\s+"; " ")' file.json

Replace string patterns.

Case Conversion

jq 'ascii_upcase' file.json
jq 'ascii_downcase' file.json
jq '.name | ascii_upcase' file.json

Convert string case.

Conditionals

If-Then-Else

jq 'if .age > 18 then "adult" else "minor" end' file.json
jq 'if .status == "active" then .id else null end' file.json

Conditional expressions.

Alternative Operator

jq '.field // "default"' file.json
jq '.optional // empty' file.json

Use default if value is false or null.

Functions

Keys

jq 'keys' file.json
jq 'keys_unsorted' file.json

Get object keys or array indices.

Values

jq 'values' file.json
jq '.[] | values' file.json

Get all values.

To Entries

jq 'to_entries' file.json
jq '. | to_entries | map({key: .key, val: .value})' file.json

Convert object to key-value array.

From Entries

jq 'from_entries' file.json

Convert key-value array to object.

Type

jq 'type' file.json
jq '.[] | type' file.json

Get type of value.

Advanced

Pipe

jq '.items | map(.name) | unique | sort' file.json

Chain operations with pipe.

Recursive Descent

jq '.. | .id?' file.json
jq '.. | select(type == "string")' file.json

Recursively descend through structure.

Variables

jq '.users | map({name, email: .email} | .email |= ascii_upcase)' file.json

Use update operators.

Try-Catch

jq 'try .field catch "error"' file.json
jq '.[] | try (.value / 0) catch "infinity"' file.json

Handle errors gracefully.

Input/Output

Slurp

jq -s . file1.json file2.json
jq --slurp '. | add' file.json

Read entire input as single array.

Raw Input

jq -R . file.txt
jq --raw-input . file.txt

Read input as raw strings.

Null Input

jq -n '{key: "value"}'
jq --null-input '{date: now}'

Start with null input.

Tab Output

jq -r '@tsv' file.json

Output as tab-separated values.

CSV Output

jq -r '@csv' file.json
jq -r '.[] | [.name, .age, .city] | @csv' file.json

Output as CSV format.

Examples

API Response Processing

# Extract specific fields
curl https://api.github.com/users/octocat | jq '{name, bio, followers}'

# Get array of values
curl https://api.github.com/repos/jqlang/jq/issues | jq '.[].title'

# Filter and map
curl https://api.example.com/users | jq '[.[] | select(.active) | .email]'

# Count filtered items
curl https://api.example.com/items | jq '[.[] | select(.price > 100)] | length'

Data Transformation

# Transform structure
jq '{users: [.[] | {id, name: .full_name}]}' users.json

# Flatten nested data
jq '[.[] | .items[]]' nested.json

# Group and count
jq 'group_by(.category) | map({category: .[0].category, count: length})' items.json

# Merge arrays
jq '. + (input | .[])' file1.json file2.json

Filtering Examples

# Complex filter
jq '.[] | select(.age > 18 and .active == true) | .name' users.json

# Filter by date
jq '.[] | select(.created > "2024-01-01")' records.json

# Filter by array membership
jq '.[] | select(.tags | contains(["urgent"]))' tickets.json

# Non-null values only
jq '[.[] | select(.field != null)]' data.json

Sorting and Grouping

# Sort by multiple fields
jq 'sort_by(.priority, .created)' tasks.json

# Top N items
jq 'sort_by(.score) | reverse | .[0:10]' results.json

# Group and aggregate
jq 'group_by(.department) | map({dept: .[0].department, total: map(.salary) | add})' employees.json

String Manipulation

# Build formatted string
jq -r '.[] | "\(.id): \(.name) (\(.email))"' users.json

# Clean and format
jq '.description | gsub("\\s+"; " ") | ascii_downcase' data.json

# Extract with regex
jq '.text | match("\\d+") | .string' data.json

# Split and filter
jq '.tags | split(",") | map(select(length > 0))' data.json

Combining Multiple Files

# Merge JSON objects
jq -s '.[0] * .[1]' config.json overrides.json

# Concatenate arrays
jq -s 'add' file1.json file2.json file3.json

# Join data
jq -s '{users: .[0], roles: .[1]}' users.json roles.json

Output Formatting

# Generate CSV
jq -r '.[] | [.name, .age, .city] | @csv' users.json > output.csv

# Generate TSV
jq -r '.[] | [.id, .title, .status] | @tsv' tasks.json

# Pretty print specific fields
jq -r '.[] | "\(.name)\t\(.email)"' users.json

# JSON Lines format
jq -c '.[]' data.json > output.jsonl

Advanced Patterns

# Pivot data
jq '[.[] | {(.category): .value}] | add' data.json

# Recursive search
jq '.. | select(.type? == "error")' logs.json

# Calculate statistics
jq '{min: min, max: max, avg: (add/length)}' numbers.json

# Build hierarchical structure
jq 'group_by(.parent) | map({parent: .[0].parent, children: map(.id)})' items.json

Notes

Syntax: Uses dot notation for field access, pipes for chaining
Filters: All jq programs are filters (input → output)
Streams: Processes JSON streams efficiently
Errors: Use ? for optional access, try-catch for error handling
Arrays: Use [] to iterate, .[n] for index access
Objects: Use .field or .["field"] for field access
Operators: | pipe, , multiple outputs, + addition/concatenation
Comparison: ==, !=, <, >, <=, >=
Logic: and, or, not
Null: Use // "default" for null coalescing
Types: string, number, boolean, array, object, null
Raw Output: -r flag removes JSON quotes from strings
Compact: -c flag outputs single-line JSON
Slurp: -s flag reads all inputs into array
Null Input: -n flag starts with null instead of reading input
Exit Code: 0 success, 1 error, 5 no output
Comments: Use # comment in jq programs
Variables: Use as $var to bind variables
Functions: Can define custom functions
Modules: Support for importing modules
Performance: Very fast, can handle large JSON files
Unicode: Full unicode support
Regex: Uses PCRE (Perl Compatible Regular Expressions)