Welcome back!
As you already know, this is my adventure series on the NATS JetStream Key/Value Store!
In Part 2, we continue our journey into the uncharted territories of distributed messaging. Imagine us as astronauts exploring the cosmos of JetStream - yes, that's totally us! Let's dive in right where we left off in Part 1.
Where Did We Leave Off?
In Part 1, we got the NATS CLI and NATS Server up and running with JetStream. If you missed it, don't worry - just refer back to Part 1.
Now, let's explore what we can do with the CLI
Meet your best Friend: The -h Flag. The -h flag provides guidance for commands at various levels. Start at the top level to explore commands for multiple parts of NATS.
Let's try it.
$ nats -h
usage: nats [<flags>] <command> [<args> ...]
NATS Utility
NATS Server and JetStream administration.
See '**nats cheat**' for a quick cheatsheet of commands
Commands:
account Account information and status
bench Benchmark utility
consumer JetStream Consumer management
context Manage nats configuration contexts
errors Error code documentation
events Show Advisories and Events
kv Interacts with a JetStream based Key-Value store
latency Perform latency tests between two NATS servers
micro Micro Services discovery and management
object Interacts with a JetStream based Object store
publish Generic data publish utility
request Generic request-reply request utility
reply Generic service reply utility
rtt Compute round-trip time to NATS server
schema Schema tools
server Server information
stream JetStream Stream management
subscribe Generic subscription client
Global Flags:
-h, --help Show context-sensitive help
--version Show application version.
-s, --server=URL NATS server urls ($NATS_URL)
--user=USER Username or Token ($NATS_USER)
--password=PASSWORD Password ($NATS_PASSWORD)
--connection-name=NAME Nickname to use for the underlying NATS Connection
--creds=FILE User credentials ($NATS_CREDS)
--nkey=FILE User NKEY ($NATS_NKEY)
--tlscert=FILE TLS public certificate ($NATS_CERT)
--tlskey=FILE TLS private key ($NATS_KEY)
--tlsca=FILE TLS certificate authority chain ($NATS_CA)
--[no-]tlsfirst Perform TLS handshake before expecting the server greeting
--timeout=DURATION Time to wait on responses from NATS ($NATS_TIMEOUT)
--socks-proxy=PROXY SOCKS5 proxy for connecting to NATS server
($NATS_SOCKS_PROXY)
--js-api-prefix=PREFIX Subject prefix for access to JetStream API
--js-event-prefix=PREFIX Subject prefix for access to JetStream Advisories
--js-domain=DOMAIN JetStream domain to access
--inbox-prefix=PREFIX Custom inbox prefix to use for inboxes
--colors=SCHEME Sets a color scheme to use ($NATS_COLOR)
--context=NAME Configuration context ($NATS_CONTEXT)
--trace Trace API interactions
--no-context Disable the selected context
Ok that's a lot. There's commands and flags in there. We will go over a lot of them in another adventure. Let's focus on the commands we want. You see we can use nats cheat to get a quick glimpse of the commands available to us.
$ nats cheat
Available Cheats:
account
bench
consumer
contexts
errors
events
kv
latency
obj
pub
reply
schemas
server
stream
sub
As you can see, there’s a lot to explore! However, for this adventure, we’re focusing solely on the JetStream Key/Value Store. This would be kv for us. Don’t worry — there will be another adventure dedicated entirely to the CLI and its many features.
Understanding the Key-Value Store
Okay, let's break down this Key/Value Store thing, locker room style. It's not rocket science really.
KV Store
Think of this like a building filled with different Locker rooms or "Buckets". Each Locker room might have it's own name like "Team Awesome's Lockers," "The Losers' Lockers" or what not. Hopefully not that last one. Each Locker room can also have it's own setup or configuration.
Bucket
Think of this as a specific locker room like "Team Awesome's Lockers". It's a way to organize your stuff and contains a bunch of "lockers" which could be considered Key/Value pairs.
Key
This is like the number on a locker. It's unique, and helps us find our stuff.
Value
This is the actual stuff inside the locker. Your sweaty gym socks, your lucky rabbit's foot, or maybe even that half-eaten protein bar you forgot about. It's the data you're storing.
So there it is. It's basically like a Map!
So we will break this down into sections:
- The KV Store
- Buckets
- Keys/Values
KV Store
Let's start by seeing what we can do now that we're using the kv command for the KV Store.
$ nats kv -h
usage: nats kv <command> [<args> ...]
Interacts with a JetStream based Key-Value store
The JetStream Key-Value store uses streams to store key-value pairs for an indefinite period or a per-bucket configured TTL.
Subcommands:
kv add Adds a new KV Store Bucket
kv put Puts a value into a key
kv get Gets a value for a key
kv create Puts a value into a key only if the key is new or it's last operation was a delete
kv update Updates a key with a new value if the previous value matches the given revision
kv del Deletes a key or the entire bucket
kv purge Deletes a key from the bucket, clearing history before creating a delete marker
kv history Shows the full history for a key
kv revert Reverts a value to a previous revision using put
kv info View the status of a KV store
kv watch Watch the bucket or a specific key for updated
kv ls List available buckets or the keys in a bucket
kv compact Reclaim space used by deleted keys
Global Flags:
-h, --help Show context-sensitive help
--version Show application version.
-s, --server=URL NATS server urls ($NATS_URL)
--user=USER Username or Token ($NATS_USER)
--password=PASSWORD Password ($NATS_PASSWORD)
--connection-name=NAME Nickname to use for the underlying NATS Connection
--creds=FILE User credentials ($NATS_CREDS)
--nkey=FILE User NKEY ($NATS_NKEY)
--tlscert=FILE TLS public certificate ($NATS_CERT)
--tlskey=FILE TLS private key ($NATS_KEY)
--tlsca=FILE TLS certificate authority chain ($NATS_CA)
--[no-]tlsfirst Perform TLS handshake before expecting the server greeting
--timeout=DURATION Time to wait on responses from NATS ($NATS_TIMEOUT)
--socks-proxy=PROXY SOCKS5 proxy for connecting to NATS server ($NATS_SOCKS_PROXY)
--js-api-prefix=PREFIX Subject prefix for access to JetStream API
--js-event-prefix=PREFIX Subject prefix for access to JetStream Advisories
--js-domain=DOMAIN JetStream domain to access
--inbox-prefix=PREFIX Custom inbox prefix to use for inboxes
--colors=SCHEME Sets a color scheme to use ($NATS_COLOR)
--context=NAME Configuration context ($NATS_CONTEXT)
--trace Trace API interactions
--no-context Disable the selected context
Wow. Ok. There’s quite a few commands there but it seems like there’s two for the KV Store itself — info and ls
info says it will give us the status of the KV Store.
$ nats kv info
nats: error: no KV buckets found
No buckets found? We’ll I guess that makes sense because we never created one! Let’s just check to be sure. We can list the buckets of a KV Store with ls
$ nats kv ls
No Key-Value buckets found
Yep. Not a bucket in sight! So how do we create a bucket?
Creating Buckets
As we stated earlier a Bucket is like a locker room for your lockers (KV Pairs). So let’s get on with it and create a locker room — a Bucket! Let's create a Bucket called Bucket1.
$ nats kv add Bucket1
Information for Key-Value Store Bucket Bucket1 created 2025-01-18T20:18:27-05:00
Configuration:
Bucket Name: Bucket1
History Kept: 1
Values Stored: 0
Compressed: false
Backing Store Kind: JetStream
Bucket Size: 0 B
Maximum Bucket Size: unlimited
Maximum Value Size: unlimited
Maximum Age: unlimited
JetStream Stream: KV_Bucket1
Storage: File
Cluster Information:
Name:
Leader: NDJ4PFQB5RVJXFDKC6G7ZZRDBEH3M4OMEUMDQFXJE5BFNSALQLAHMTEV
You will see information on the bucket we just created.
# Bucket Name: Bucket1 # Name of the Key-Value bucket.
# History Kept: 1 # Number of historical versions retained per key.
# Values Stored: 0 # Current number of keys (lockers) in the bucket.
# Compressed: false # Indicates if data compression is enabled for this bucket.
# Backing Store Kind: JetStream # Storage backend used (JetStream in this case).
# Bucket Size: 0 B # Total size of the bucket's stored data.
# Maximum Bucket Size: unlimited # Maximum storage size allowed for the bucket (no limit here).
# Maximum Value Size: unlimited # Maximum size for a single key's value (no limit here).
# Maximum Age: unlimited # Maximum time a key-value pair is retained (no expiration here).
# JetStream Stream: KV_Bucket1 # JetStream stream backing this Key-Value bucket.
# Storage: File # Type of storage used (file-based in this case).
Let's go again!
$ nats kv add Bucket2
Information for Key-Value Store Bucket Bucket2 created 2025-01-18T20:18:35-05:00
Configuration:
Bucket Name: Bucket2
History Kept: 1
Values Stored: 0
Compressed: false
Backing Store Kind: JetStream
Bucket Size: 0 B
Maximum Bucket Size: unlimited
Maximum Value Size: unlimited
Maximum Age: unlimited
JetStream Stream: KV_Bucket2
Storage: File
Cluster Information:
Name:
Leader: NDJ4PFQB5RVJXFDKC6G7ZZRDBEH3M4OMEUMDQFXJE5BFNSALQLAHMTEV
Now let's try nats kv info again.
$ nats kv info
You should get a menu - you can select which bucket you want info for.
? Select a Bucket [Use arrows to move, type to filter]
> Bucket1
Bucket2
Let's pick Bucket1.
Information for Key-Value Store Bucket Bucket1 created 2025-01-18T20:18:27-05:00
Configuration:
Bucket Name: Bucket1
History Kept: 1
Values Stored: 0
Compressed: false
Backing Store Kind: JetStream
Bucket Size: 0 B
Maximum Bucket Size: unlimited
Maximum Value Size: unlimited
Maximum Age: unlimited
JetStream Stream: KV_Bucket1
Storage: File
Cluster Information:
Name:
Leader: NDJ4PFQB5RVJXFDKC6G7ZZRDBEH3M4OMEUMDQFXJE5BFNSALQLAHMTEV
As you can see, you can always access information for various Buckets. Each Bucket can have its own configuration with different properties. You can define these properties during creation using flags.
Let’s create a bucket using the compress flag!
$ nats kv add Bucket3 --compress
Information for Key-Value Store Bucket Bucket3 created 2025-01-18T22:40:43-05:00
Configuration:
Bucket Name: Bucket3
History Kept: 1
Values Stored: 0
Compressed: true
Backing Store Kind: JetStream
Bucket Size: 0 B
Maximum Bucket Size: unlimited
Maximum Value Size: unlimited
Maximum Age: unlimited
JetStream Stream: KV_Bucket3
Storage: File
Cluster Information:
Name:
Leader: NDJ4PFQB5RVJXFDKC6G7ZZRDBEH3M4OMEUMDQFXJE5BFNSALQLAHMTEV
What does the compressed flag do? If it’s not configured then the bucket is not using internal compression for its stored data. Data is stored as-is in the underlying stream (uncompressed).
If enabled then compression is enabled for the data stored in the underlying stream, which can reduce disk usage at the cost of slightly higher CPU usage during compression and decompression.
Pretty neat! As you can see there’s a few other configuration options that come with their own flags — but for now let’s keep going.
Let’s try to list all the Buckets again…
$ nats kv ls
╭───────────────────────────────────────────────────────────────────────────╮
│ Key-Value Buckets │
├─────────┬─────────────┬─────────────────────┬──────┬────────┬─────────────┤
│ Bucket │ Description │ Created │ Size │ Values │ Last Update │
├─────────┼─────────────┼─────────────────────┼──────┼────────┼─────────────┤
│ Bucket1 │ │ 2025-01-19 02:55:03 │ 0 B │ 0 │ never │
│ Bucket2 │ │ 2025-01-19 02:55:05 │ 0 B │ 0 │ never │
│ Bucket3 │ │ 2025-01-19 02:55:31 │ 0 B │ 0 │ never │
╰─────────┴─────────────┴─────────────────────┴──────┴────────┴─────────────╯
Boom! There they are!
There’s a lot to explore when it comes to creating buckets, but we’ll save that for a deep dive later.
Reading from Buckets
Let’s check the keys in a bucket and make sure they’re empty.
$ nats kv ls Bucket1
No keys found in bucket
This ones obvious. We haven’t created any keys yet!
Bucket Update
Alrighty, how do we update a bucket. Well — it’s not that simple. The bucket’s configuration, such as TTL, history, maximum size, and compression settings, is fixed at the time of creation.
Why Can’t a Bucket Be Modified?
This behavior ensures:
- Consistency: Prevents unexpected changes that could affect how data is stored or retrieved.
- Integrity: Protects existing data from being impacted by configuration changes like reduced history or TTL adjustments.
- Simplicity: Keeps the design predictable and easy to reason about.
Alright cool. Three buckets seems like too many so let’s get rid of one.
What you can do is export the data and recreate the bucket, a process that your team would typically have workflows and protocols for. This isn’t something that would be done frequently if at all. Something my dad used to tell me — the 5 P’s.
"Proper planning prevents piss poor performance!"
Bucket Delete
Well I don’t like Bucket3 so I want delete him or her. Let’s just get a quick idea of what happens.
- All Keys and Values Are Removed — Every key, its values, and history stored in the bucket are permanently deleted. This action cannot be undone, so proceed with caution.
- Underlying JetStream Stream Is Deleted — A KV bucket in NATS JetStream is backed by a JetStream stream. When you delete the bucket, the corresponding stream is also deleted.
- Storage Space Is Reclaimed — All disk space previously used by the bucket is freed, making it available for other operations. Audit Trails Are Lost — Unlike individual key deletions or purges, deleting a bucket does not retain any markers or history. The bucket and all its contents are entirely erased.
Sorry bucket 3!
$ nats kv del Bucket3
? Delete bucket Bucket3? Yes
Let's make sure it's gone.
$ nats kv ls
╭───────────────────────────────────────────────────────────────────────────╮
│ Key-Value Buckets │
├─────────┬─────────────┬─────────────────────┬──────┬────────┬─────────────┤
│ Bucket │ Description │ Created │ Size │ Values │ Last Update │
├─────────┼─────────────┼─────────────────────┼──────┼────────┼─────────────┤
│ Bucket1 │ │ 2025-01-18 20:18:27 │ 0 B │ 0 │ never │
│ Bucket2 │ │ 2025-01-18 20:18:35 │ 0 B │ 0 │ never │
╰─────────┴─────────────┴─────────────────────┴──────┴────────┴─────────────╯
Yep! A goner!
Deleting a Bucket isn’t something that would typically be done. While there may be specific scenarios where it’s necessary, deleting a Bucket is almost akin to deleting an entire database. In such cases, protocols and workflows should be in place to manage the process — much like updating a Bucket.
It’s worth noting that NATS keeps track of history for operations like deleting a Bucket, but diving into those details is beyond our current scope.
So far, we’ve explored how to create Bucket with configurations, check their information, and delete them. Neat! Now, it’s time to dive into actually using Buckets. Let’s explore how keys work.
I think we’ve done enough here for now; so let’s skip on ahead to Part 3 with keys!
Top comments (0)