Redis JSON Datatype overview - Amazon MemoryDB for Redis

Redis JSON Datatype overview

MemoryDB supports a number of Redis commands for working with the JSON datatype. Following is an overview of the JSON datatype and a detailed list of Redis commands that are supported.

Terminology

Term Description

JSON document

refers to the value of a Redis JSON key

JSON value

refers to a subset of a JSON Document, including the root that represents the entire document. A value could be a container or an entry within a container

JSON element

equivalent to JSON value

Supported JSON standard

JSON format is compliant with RFC 7159 and ECMA-404 JSON data interchange standard. UTF-8 Unicode in JSON text is supported.

Root element

The root element can be of any JSON data type. Note that in earlier RFC 4627, only objects or arrays were allowed as root values. Since the update to RFC 7159, the root of a JSON document can be of any JSON data type.

Document size limit

JSON documents are stored internally in a format optimized for rapid access and modification. This format typically results in consuming somewhat more memory than does the equivalent serialized representation of the same document. The consumption of memory by a single JSON document is limited to 64MB, which is the size of the in-memory data structure, not the JSON string. The amount of memory consumed by a JSON document can be inspected by using the JSON.DEBUG MEMORY command.

JSON ACLs

  • JSON datatype is fully integrated into the Redis Access Control Lists (ACL) capability. Similar to the existing per-datatype categories (@string, @hash, etc.) a new category @json is added to simplify managing access to JSON commands and data. No other existing Redis commands are members of the @json category. All JSON commands enforce any keyspace or command restrictions and permissions.

  • There are five existing Redis ACL categories that are updated to include the new JSON commands: @read, @write, @fast, @slow and @admin. The table below indicates the mapping of JSON commands to the appropriate categories.

ACL
JSON Command @read @write @fast @slow @admin

JSON.ARRAPPEND

y

y

JSON.ARRINDEX

y

y

JSON.ARRINSERT

y

y

JSON.ARRLEN

y

y

JSON.ARRPOP

y

y

JSON.ARRTRIM

y

y

JSON.CLEAR

y

y

JSON.DEBUG

y

y

y

JSON.DEL

y

y

JSON.FORGET

y

y

JSON.GET

y

y

JSON.MGET

y

y

JSON.NUMINCRBY

y

y

JSON.NUMMULTBY

y

y

JSON.OBJKEYS

y

y

JSON.OBJLEN

y

y

JSON.RESP

y

y

JSON.SET

y

y

JSON.STRAPPEND

y

y

JSON.STRLEN

y

y

JSON.STRLEN

y

y

JSON.TOGGLE

y

y

JSON.TYPE

y

y

JSON.NUMINCRBY

y

y

Nesting depth limit

When a JSON object or array has an element that is itself another JSON object or array, that inner object or array is said to “nest” within the outer object or array. The maximum nesting depth limit is 128. Any attempt to create a document that contains a nesting depth greater than 128 will be rejected with an error.

Command syntax

Most commands require a Redis key name as the first argument. Some commands also have a path argument. The path argument defaults to the root if it is optional and not provided.

Notation:

  • Required arguments are enclosed in angle brackets, e.g. <key>

  • Optional arguments are enclosed in square brackets, e.g. [path]

  • Additional optional arguments are indicated by ..., e.g. [json ...]

Path syntax

JSON-Redis supports two kinds of path syntaxes:

  • Enhanced syntax – Follows the JSONPath syntax described by Goessner, as shown in the table below. We've reordered and modified the descriptions in the table for clarity.

  • Restricted syntax – Has limited query capabilities.

Note

Results of some commands are sensitive which type of path syntax is used.

If a query path starts with '$', it uses the enhanced syntax. Otherwise, the restricted syntax is used.

Enhanced Syntax

Symbol/Expression Description

$

the root element

. or []

child operator

..

recursive descent

*

wildcard. All elements in an object or array.

[]

array subscript operator. Index is 0-based.

[,]

union operator

[start:end:step]

array slice operator

?()

applies a filter (script) expression to the current array or object

()

filter expression

@

used in filter expressions referring to the current node being processed

==

equal to, used in filter expressions.

!=

not equal to, used in filter expressions.

>

greater than, used in filter expressions.

>=

greater than or equal to, used in filter expressions.

<

less than, used in filter expressions.

<=

less than or equal to, used in filter expressions.

&&

logical AND, used to combine multiple filter expressions.

||

logical OR, used to combine multiple filter expressions.

Examples

The below examples are built on Goessner's example XML data, which we have modified by adding additional fields.

{ "store": { "book": [ { "category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95, "in-stock": true, "sold": true }, { "category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99, "in-stock": false, "sold": true }, { "category": "fiction", "author": "Herman Melville", "title": "Moby Dick", "isbn": "0-553-21311-3", "price": 8.99, "in-stock": true, "sold": false }, { "category": "fiction", "author": "J. R. R. Tolkien", "title": "The Lord of the Rings", "isbn": "0-395-19395-8", "price": 22.99, "in-stock": false, "sold": false } ], "bicycle": { "color": "red", "price": 19.95, "in-stock": true, "sold": false } } }
Path Description

$.store.book[*].author

the authors of all books in the store

$..author

all authors

$.store.*

all members of the store

$["store"].*

all members of the store

$.store..price

the price of everything in the store

$..*

all recursive members of the JSON structure

$..book[*]

all books

$..book[0]

the first book

$..book[-1]

the last book

$..book[0:2]

the first two books

$..book[0,1]

the first two books

$..book[0:4]

books from index 0 to 3 (ending index is not inclusive)

$..book[0:4:2]

books at index 0, 2

$..book[?(@.isbn)]

all books with isbn number

$..book[?(@.price<10)]

all books cheaper than $10

'$..book[?(@.price < 10)]'

all books cheaper than $10. (The path must be quoted if it contains whitespaces)

'$..book[?(@["price"] < 10)]'

all books cheaper than $10

'$..book[?(@.["price"] < 10)]'

all books cheaper than $10

$..book[?(@.price>=10&&@.price<=100)]

all books in the price range of $10 to $100, inclusive

'$..book[?(@.price>=10 && @.price<=100)]'

all books in the price range of $10 to $100, inclusive. (The path must be quoted if it contains whitespaces)

$..book[?(@.sold==true||@.in-stock==false)]

all books sold or out of stock

'$..book[?(@.sold == true || @.in-stock == false)]'

all books sold or out of stock. (The path must be quoted if it contains whitespaces)

'$.store.book[?(@.["category"] == "fiction")]'

all books in the fiction category

'$.store.book[?(@.["category"] != "fiction")]'

all books in non-fiction categories

More filter expression examples:

127.0.0.1:6379> JSON.SET k1 . '{"books": [{"price":5,"sold":true,"in-stock":true,"title":"foo"}, {"price":15,"sold":false,"title":"abc"}]}' OK 127.0.0.1:6379> JSON.GET k1 $.books[?(@.price>1&&@.price<20&&@.in-stock)] "[{\"price\":5,\"sold\":true,\"in-stock\":true,\"title\":\"foo\"}]" 127.0.0.1:6379> JSON.GET k1 '$.books[?(@.price>1 && @.price<20 && @.in-stock)]' "[{\"price\":5,\"sold\":true,\"in-stock\":true,\"title\":\"foo\"}]" 127.0.0.1:6379> JSON.GET k1 '$.books[?((@.price>1 && @.price<20) && (@.sold==false))]' "[{\"price\":15,\"sold\":false,\"title\":\"abc\"}]" 127.0.0.1:6379> JSON.GET k1 '$.books[?(@.title == "abc")]' [{"price":15,"sold":false,"title":"abc"}] 127.0.0.1:6379> JSON.SET k2 . '[1,2,3,4,5]' 127.0.0.1:6379> JSON.GET k2 $.*.[?(@>2)] "[3,4,5]" 127.0.0.1:6379> JSON.GET k2 '$.*.[?(@ > 2)]' "[3,4,5]" 127.0.0.1:6379> JSON.SET k3 . '[true,false,true,false,null,1,2,3,4]' OK 127.0.0.1:6379> JSON.GET k3 $.*.[?(@==true)] "[true,true]" 127.0.0.1:6379> JSON.GET k3 '$.*.[?(@ == true)]' "[true,true]" 127.0.0.1:6379> JSON.GET k3 $.*.[?(@>1)] "[2,3,4]" 127.0.0.1:6379> JSON.GET k3 '$.*.[?(@ > 1)]' "[2,3,4]"

Restricted syntax

Symbol/Expression Description

. or []

child operator

[]

array subscript operator. Index is 0-based.

Examples

Path Description

.store.book[0].author

the author of the first book

.store.book[-1].author

the author of the last book

.address.city

city name

["store"]["book"][0]["title"]

the title of the first book

["store"]["book"][-1]["title"]

the title of the last book

Note

All Goessner content cited in this documentation is subject to the Creative Commons License.

Common error prefixes

Each error message has a prefix. The following is a list of common error prefixes:

Prefix Description

ERR

a general error

LIMIT

size limit exceeded error. e.g., the document size limit or nesting depth limit exceeded

NONEXISTENT

a key or path does not exist

OUTOFBOUNDARIES

array index out of bounds

SYNTAXERR

syntax error

WRONGTYPE

wrong value type

JSON related metrics

The following JSON info metrics are provided:

Info Description

json_total_memory_bytes

total memory allocated to JSON objects

json_num_documents

total number of documents in Redis

To query core metrics, run Redis command:

info json_core_metrics

How MemoryDB interacts with JSON

The following illustrates how MemoryDB interacts with the JSON datatype.

Operator precedence

When evaluating conditional expressions for filtering, &&s take precedence first, and then ||s are evaluated, as is common across most languages. Operations inside of parentheses will be executed first.

Maximum path nesting limit behavior

MemoryDB's maximum path nesting limit is 128. So a value like $.a.b.c.d... can only reach 128 levels.

Handling numeric values

JSON does not have separate data types for integers and floating point numbers. They are all called numbers.

When a JSON number is received, it is stored in one of two formats. If the number fits into a 64-bit signed integer, then it is converted to that format; otherwise, it is stored as a string. Arithmetic operations on two JSON numbers (e.g. JSON.NUMINCRBY and JSON.NUMMULTBY) attempt to preserve as much precision as possible. If the two operands and the resulting value fit into a 64-bit signed integer, then integer arithmetic is performed. Otherwise, the input operands are converted into 64-bit IEEE double-precision floating point numbers, the arithmetic operation is performed, and the result is converted back into a string.

Arithmetic commands NUMINCRBY and NUMMULTBY:

  • If both numbers are integers, and the result is out of the range of int64, it will automatically become a double precision floating point number.

  • If at least one of the numbers is a floating point, the result will be a double precision floating point number.

  • If the result exceeds the range of double, the command will return an OVERFLOW error.

Note

Prior to Redis engine version 6.2.6.R2 when a JSON number is received on input, it is converted into one of the two internal binary representations: a 64-bit signed integer or a 64-bit IEEE double precision floating point. The original string and all of its formatting are not retained. Thus, when a number is output as part of a JSON response, it is converted from the internal binary representation to a printable string that uses generic formatting rules. These rules might result in a different string being generated than was received.

  • If both numbers are integers and the result is out of the range of int64, it automatically becomes a 64-bit IEEE double precision floating point number.

  • If at least one of the numbers is a floating point, the result is a 64-bit IEEE double precision floating point number.

  • If the result exceeds the range of 64-bit IEEE double, the command returns an OVERFLOW error.

For a detailed list of available commands, see Supported commands.

Strict syntax evaluation

MemoryDB does not allow JSON paths with invalid syntax, even if a subset of the path contains a valid path. This is to maintain correct behavior for our customers.