How to fix "jq: error string and number cannot be added" in AWS CloudShell?

Note: This article is an English translation of my original article, which you can find here.

Overview

This article describes the causes and solutions to the error when using jq join in AWS CloudShell.

jq error in AWS CloudShell

This section describes an error that occurs when using jq in AWS CloudShell.

If you try to join an array that contains numbers, you will get jq: error string and number cannot be added.

Expected:

$ echo '["example",0]' | jq 'join(".")'
"example.0"

Actual:

$ echo '["example",0]' | jq 'join(".")'
jq: error (at <stdin>:1): string (".") and number (0) cannot be added

Cause

This section describes the cause of this error.

The version of jq preinstalled in CloudShell is 1.5¹.

$ jq --version
jq-1.5

A bug in the 1.5 version of join does not convert number types into strings, resulting in this error.

L1422 - jq/builtin.c at jq-1.5 · stedolan/jq

  "def join($x): reduce .[] as $i (null; (.//\"\") + (if . == null then $i else $x + $i end))//\"\";",

Note:

Ver 1.6 fixed this bug. You can confirm that in the joined source, which converts the type number to a string with tostring.

def join($x): reduce .[] as $i (null;
            (if .==null then "" else .+$x end) +
            ($i | if type=="boolean" or type=="number" then tostring else .//"" end)
        ) // "";

The issue:

string and number cannot be used for join() #1595

jpcarey posted on Feb 06, 2018

If you run the join example, it errors:

jq: error (at <stdin>:0): string (" ") and number (1) cannot be added
exit status 5

 jq 'join(" ")' input: ["a",1,2.3,true,null,false] output: "a 1 2.3 true false"

version 1.5

From the docs:

Numbers and booleans in the input are converted to strings. Null values are treated as empty strings. Arrays and objects in the input are not supported.

View on GitHub

Solution 1

I would describe two means of resolving this error.

First, I explain how to fix it to work with jq1.5.

If you convert the number to a string before passing it to join, it will work as expected in 1.5. And, of course, it also works fine with 1.6.

$ echo '["example",0]' | jq 'map(.|tostring) | join(".")'
"example.0"

Solution 2

Next, I explain how to install jq1.6 in CloudShell.

However, the yum provided with CloudShell does not upgrade the pre-installed jq², so you will download the binary in an appropriate directory.

Note: (about directory persistence)

Please note that if you place binary files in /usr/bin, the files will be deleted when CloudShell is restarted.

If you place the file in your home directory, the data will be persistent, but please note that the file will be deleted when you do not use that CloudShell environment for more than 120 days.

Service quotas and restrictions for AWS CloudShell - AWS CloudShell

Practical Examples

Here is a concrete example to explanate the solution.

Create an installation directory in your home directory and download the binary. Of course, the directory is arbitrary, but here we assume jq1.6.

cd ~
mkdir jq1.6
cd jq1.6
wget -O jq https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64
chmod +x ./jq 

As mentioned earlier, this bug has been Fixed in 1.6, so it works as expected.

$ ~/jq1.6/jq --version
jq-1.6
$ echo '["example",0]' | ~/jq1.6/jq 'join(".")'
"example.0"

It is helpful to set up an alias.

$ echo "alias jq=~/jq1.6/jq" >> ~/.bashrc
$ . ~/.bashrc  # Reflect the alias in the shell where you are logged in:
$ echo '["example",0]' | jq 'join(".")'
"example.0"

To run as a shell script

Note that when running as a shell script, the login shell environment is not inherited, so the alias is undefined, and it executes with jq 1.5.

./join.sh:

#!/bin/bash
echo '["example",0]' | jq 'join(".")'

$ bash ./join.sh 
jq: error (at <stdin>:1): string (".") and number (0) cannot be added

In this case, if you specify interactive behavior with the -i option, the shell in the execution environment will read ~/.bashrc and execute the script, as expected.

$ bash -i ./join.sh 
"example.0"

Conclusion

This section summarizes how to deal with the error when using jq in CloudShell.

I have described two solutions.

Solution 1 has the advantage of not requiring any changes to the CloudShell environment and is portable as it works with both 1.5 and 1.6. ³

Solution 2, however, has the advantage of running on 1.6, so it can also handle behaviors other than join error.

One of the changes other than join in 1.6:

Different behavior when omitting jq arguments

Please use one of these methods depending on your situation.

Reference

jq: error: number and string cannot be added

Oct 24 '17 Comments: 1 Answers: 4

28

I'd like to print the value of members and id from below jq output:

$ cat test_|jq -r '.[] | select(.name=="AAA") | .'
{
  "name": "AAA",
  "members": 10,
  "profiles": 0,
  "templates": 0,
  "ldapGroups": 0,
  "ldapMembers": 0,
  "id": "20"
}

Unfortunately it works for one of each only:

$ cat test_|jq

…

Open Full Question

---

1. This is the version at the time of the article submission. It may be upgraded in the future. ↩

2. I confirmed with AWS support. ↩

3. If you have a development PC or other environment where you can install freely, you will likely install the latest version, 1.6. ↩