How to parse JSON with jq

jq is a versatile JSON processor that operates on the command line. It shares similarities with the sed utility but focuses specifically on JSON data manipulation. jq acts as a filter receiving input and generating output. It enables chaining filters together by piping the output of one filter into another or collecting filter results into an array.

Installing jq

Ubuntu is a popular Linux distribution known for its ease of use and a large community of users. In this topic, we will use the Ubuntu Linux distribution. You can install jq on Ubuntu with the command:

sudo apt-get install jq

Windows and macOS are also popular among developers. You can install jq on Windows with the Chocolatey package manager with the following command:

choco install jq

While if you are a macOS user, you can install jq with the Homebrew package manager using the following command:

brew install jq

If you use other Linux distributions or package managers, the official jq guide provides all the information you need to install jq.

Parsing JSON with jq

Let's begin by downloading a JSON file — school.json. The simplest method of parsing JSON with jq is by using the . filter. This filter pretty prints the entire JSON object in a readable format. You can use the filter with the following command:

jq '.' school.json

This command prints the entire content of the school.json file. Suppose you have the following JSON object that has not been formatted nicely:

{"firstName": "Emily", "lastName": "Anderson", "age": 20}

The jq utility prints it with appropriate indentation.

{
  "firstName": "Emily",
  "lastName": "Anderson",
  "age": 20
}

By default, jq indents with two spaces. But you can change this with the --indent n option. You can change the indentation using:

echo '{"firstName": "Emily", "lastName": "Anderson", "age": 20}' | jq --indent 7

The maximum value for n is 7. You can also use the --tab option for indentation, which gives the same output as using --indent 7.

Recall that JSON is a collection of key-value pairs. You can understand the structure of a JSON file by looking at its keys. Do so using the inbuilt keys function:

jq 'keys' school.json

This returns the keys present in an array:

[
  "students"
]

View the value associated with each key using:

jq '.[]' school.json

You can also write the above command as:

jq '.students' school.json

Both commands return the array of objects that are associated with the students key.

Slicing JSON with jq

While working more with JSON files, you may encounter situations where you want subsets of the JSON object. You can use array indexing to access specific elements within an array.

If you are interested in the first entry in the array, you can output it with:

jq '.students[0]' school.json

The command outputs:

{
  "firstName": "Emily",
  "lastName": "Anderson",
  "age": 20,
  "major": "Computer Science",
  "grades": {
    "math": 85,
    "programming": 92,
    "data structures": 88
  }
}

You may find it more convenient to pipe the value associated with the .students key into the .[0] filter. You get the same output with the following command using pipe (|):

jq '.students | .[0]' school.json

In both cases, use the index of the first element (0) to access it in the array. You can perform the same action with the inbuilt first function:

jq 'first(.students[])' school.json

You can also access the last element in the array similarly. First, try to access the last element with array indexing:

jq '.students[-1]' school.json

Next, with the inbuilt last function:

jq 'last(.students[])' school.json

Both commands output:

{
  "firstName": "Ethan",
  "lastName": "Evans",
  "age": 23,
  "major": "Business Administration",
  "grades": {
    "business management": 92,
    "economics": 78,
    "marketing": 85
  }
}

Apart from getting specific elements in the array, you can use slicing to get a subarray of elements. Execute the following command to get a subarray of the first two elements:

jq '.students[:2]' school.json

The command below outputs a subarray of the last two elements:

jq '.students[-2:]' school.json

While the following command outputs a subarray of the second and third elements:

jq '.students[2:4]' school.json

The jq utility can also be used to get the substring of a string by slicing.

Accessing data values

To access data values, use their corresponding keys and perform operations on them. The following command uses jq to output the first name of all the students.

jq '.students | .[] | .firstName' school.json

This command outputs:

"Emily"
"Daniel"
"Bob"
"Jada"
"Ava"
"Ethan"

Maybe you want the raw unquoted strings. You can get the output with unquoted strings using the -r option:

jq -r '.students | .[] | .firstName' school.json

You know how to access the value associated with a key. But did you know that you can also access the values associated with multiple keys? For example, the following command gets the first name and last name of each student:

jq '.students | .[] | .firstName, .lastName' school.json

The output of the command is:

"Emily"
"Anderson"
"Daniel"
"Nguyen"
"Bob"
"Lee"
"Jada"
"Jordan"
"Ava"
"Evans"
"Ethan"
"Evans"

Although the command outputs the first and last names, each value is printed on a new line. To get the first name and last name separated by single whitespace on a single line for each student, use the following command:

jq '.students | .[] | "\(.firstName) \(.lastName)"' school.json

The output is:

"Emily Anderson"
"Daniel Nguyen"
"Bob Lee"
"Jada Jordan"
"Ava Evans"
"Ethan Evans"

The command given above uses string interpolation: "\(.firstName) \(.lastName)". String interpolation is a feature of the jq processor that allows you to insert variables and expressions into a string to create more dynamic and readable code.

String interpolation in jq can be accomplished using the \(expression) syntax. It is crucial to include double quotation marks ("") around the interpolated part of the command for it to function correctly. These quotation marks are necessary to handle spaces between the first and last names. Omitting the quotation marks would result in any space being interpreted as a delimiter. This might lead to incorrect output or error.

Let's take one more look at string interpolation with the following JSON object:

echo '{"firstName": "Emily", "lastName": "Anderson", "age": 20}' | jq '"\(.firstName) \(.lastName) \(.age)"'

The command gives the following output:

"Emily Anderson 20"

Now look at how you can access values from a nested JSON. Let's get the grades of the first element in the array with the following command:

jq '.students | .[0] | .grades' school.json

This command outputs:

{
  "math": 85,
  "programming": 92,
  "data structures": 88
}

You can specifically get the grade for math with this command:

jq '.[] | .[0] | .grades | .math' school.json

Conclusion

You now know the capabilities of jq utility for parsing JSON objects. Its concise and expressive syntax makes it easy to navigate complex JSON hierarchies. jq offers a wide range of capabilities to meet your data processing needs, including:

• Accessing values by keys
• Applying filters
• Performing string interpolations

It is an invaluable tool for working with JSON. Explore the documentation of the jq utility to discover more features.