1. Redis Pub/Sub

A Beatles inspired submarine cocktail (Source: Shutterstock)

The theme of “Pub” pops up frequently in my blogs. A few blogs ago we overheard a conversation in an outback pub (“Apache ZooKeeper™️ Meets the Dining Philosophers”), and now in this blog, we are investigating Redis™️ Pub/Sub. This sounds like some sort of (Beatles-inspired) exotic (yellow) submarine cocktail, but Pub/Sub is actually short for Publish/Subscribe—the well-known pattern for loosely-coupled distributed messaging systems.

Redis Pub/Sub is the oldest style of messaging pattern supported by Redis, and uses a data type called a “channel” which supports typical Pub/Sub operations, such as Publish and Subscribe. It’s considered loosely coupled because publishers and subscribers don’t know about each other. Publishers publish messages to a channel (or multiple channels), and subscribers subscribe to 1 or more channels. A channel can have 0 or more subscribers, and the messages are delivered to all the current connected subscribers.

Redis Pub/Sub is therefore flexible, and supports multiple topologies including fan-in (multiple producers, single subscriber), fan-out (single producer, multiple subscribers), and 1-1 (1 producer, 1 consumer). So far this sounds like a fairly typical Pub/Sub system, however, one feature is important to highlight: “connected” delivery semantics.

2. Connected Delivery Semantics

Connected delivery functions like radio—radio stations are constantly broadcasting on different frequencies (channels), but listeners can only hear the broadcast while their receiver is plugged in, turned on, and they are tuned in to a station (“stay tuned” could be the motto for Redis Pub/Sub).

(Source: Shutterstock)

Redis Pub/Sub—to listen, plugin, turn on and tune in.

Connected delivery means that:

1. Only connected subscribers receive messages
2. Every connected subscriber receives each message
3. Once the message is delivered to all current subscribers it is deleted from the channel (i.e. there’s no “memory” in the system)

This implies that:

1. If a subscriber unsubscribes (disconnects) and later subscribes to a channel again:
   1. It will not receive any of the intervening messages that it missed while disconnected, and
   2. It doesn’t know if it’s missed any messages
2. If there are no current subscribers to the channel then the message will simply be discarded and not delivered to any subscribers
3. The delivery semantics are therefore “at-most-once” per subscriber
4. Because the message must be delivered to all current subscribers before being deleted:
   1. This will take longer with more subscribers
   2. This is unlike a radio broadcast, which delivers content currently at the speed of light to every receiver in range

Note that “disconnection” is intentional by design, but can also be due to network or client failures, so may be unexpected, and this will also result in potential message loss.

From reading the Redis Pub/Sub documentation (and other blogs) it appears that Redis uses push notification to ensure messages are delivered to all current subscribers, which has potential performance penalties for large numbers of subscribers

3. An Example Using PUBLISH and SUBSCRIBE Commands

Let’s have a look at a simple example to send part of the nonsense poem “Jabberwocky” by Lewis Caroll. So grab your vorpal blade and see how we can snicker-snack the poem in different ways with Redis!

(Source: Shutterstock)

The first verse starts out innocently enough like this:

’Twas brillig, and the slithy toves

      Did gyre and gimble in the wabe:

All mimsy were the borogoves,

      And the mome raths outgrabe.

For this experiment, I’m using Instaclustr Managed Redis (which is actually a Redis cluster), and the Redis CLI.  

3.1 Connected Delivery

Farm family listening to their radio, Michigan, Aug. 15, 1930. Photo by George Ackerman.1930’s farm family listening to their radio (Source: Shutterstock)

Let’s create 2 subscribers by subscribing 2 clients to the channel “jabberwocky” with the SUBSCRIBE command (which takes 1 or more channel names, we’ll just use a single channel for this example).

Subscriber 1:

SUBSCRIBE jabberwocky
Reading messages… (press Ctrl-C to quit)
1. “subscribe”
2.”jabberwocky”
3. (integer) 1

Subscriber 2:

SUBSCRIBE jabberwocky
Reading messages… (press Ctrl-C to quit)
1. “subscribe”
2. “jabberwocky”
3. (integer) 1

The results from the SUBSCRIBE command indicate that the subscribe operation was successful on the “jabberwocky” channel, and that the subscribers are active and waiting for any messages published on the channel (note that in the Redis CLI you can no longer execute any commands, the only option is Ctrl-C to quit the client).

Next, we can start publishing messages to the channel in another client with the PUBLISH command.

Publisher:

PUBLISH jabberwocky ‘Twas

(integer) 2

The result (2) indicates that the message was published to 2 subscribers, which we can confirm by looking at the output of the clients. Note that the received result is an array of 3 elements (message, channel, and value).

Subscriber 1:

1. “message”
2. “jabberwocky”
3. “‘Twas”

Subscriber 2:

1. “message”
2. “jabberwocky”
3. “‘Twas”

So, this demonstrates that the message “‘Twas” was delivered twice, to both active subscribers—connected delivery works!

I wondered what data type a message can be. Oddly, the Redis Pub/Sub documentation doesn’t say explicitly, but it looks like Pub/Sub only supports the most basic Redis string data type.

3.2 Disconnected “Delivery”

An old radio receiver unplugged. Unplugged “undelivery”  (Source: Shutterstock)

Let’s assume that the remainder of the verse is published in the same way (one word at a time, excluding punctuation), but with the subscribers disconnecting and reconnecting at different times.

Subscriber 2 thinks this is just too much nonsense, and unsubscribes after “toves”, but then changes their mind and subscribes just in time before “outgrabe” is published.

Also, Subscriber 1 has something else to do for a while and unsubscribes after “wabe”, but resubscribes after “borogoves” has already been published. 

So, both subscribers are subscribed or unsubscribed as follows:

[subscribe 1][subscribe 2]
‘Twas brillig and the slithy toves
[unsubscribe 2]
Did gyre and gimble in the wabe
[unsubscribe 1]
All mimsy were the borogoves
[subscribe 1]
And the mome raths
[subscribe 2]
outgrabe

Focussing on Subscriber 1 they are therefore connected for the blue messages:

[subscribe 1][subscribe 2]
’Twas brillig and the slithy toves
[unsubscribe 2]
Did gyre and gimble in the wabe
[unsubscribe 1]
All mimsy were the borogoves
[subscribe 1]
And the mome raths
[subscribe 2]
outgrabe

So Subscriber 1 receives:

’Twas brillig and the slithy toves
Did gyre and gimble in the wabe
And the mome raths outgrabe

And from the perspective of Subscriber 2, they are connected for the purple messages:

[subscribe 1][subscribe 2]
’Twas brillig and the slithy toves
[unsubscribe 2]
Did gyre and gimble in the wabe
[unsubscribe 1]
All mimsy were the borogoves
[subscribe 1]
And the mome raths
[subscribe 2]
outgrabe

So Subscriber 2 receives:

’Twas brillig and the slithy toves outgrabe

This example proves a couple of things. First, subscribers really do only receive the messages published while they are subscribed (both subscribers only get a subset of the poem), and second, if there are no subscribers, published messages are discarded—in this example the messages:

All mimsy were the borogoves

are published when there are no subscribers, so they are not delivered to anyone and simply vanish into the “ether”.

To receive messages, Redis Pub/Sub subscribers must “stay tuned” in!
(Source: Shutterstock)

And stay tuned for the next part of the Redis Pub/Sub blog series when we’ll hear about some “extras”, Pub/Sub Use Cases, and compare with Kafka.