Announcing the Pub/Sub Compute app

We are excited to announce the Pub/Sub app for Fastly Compute, making it easy to push real-time updates to browsers and other devices. Messages are delivered via the Server-Sent Events or MQTT protocols. Check out the demo to see it in action, and read on to learn how to set up your own instance in minutes!

The Pub/Sub app is one of the first user-installable apps provided by Fastly that is production ready out of the box. It is not a template or starter kit that requires modification. Just install and go. Of course, since the code is open source, you are free to fork it and add your own functionality. We also welcome contributions.

You may know that Fastly also offers powerful real-time messaging capability via Fanout, however Fanout is designed for API operators with rigid requirements around messaging protocols, authentication, and storage. If you aren’t building or operating a push API and you simply want to send messages to browsers connected to your own website, then the Pub/Sub app is for you. Fun fact: the Pub/Sub app is built on top of Fanout!

The Pub/Sub app is highly scalable and there are no hard limits. It supports millions of concurrent connections and channels, and there is no limit to the number of subscriptions a single channel can have. Even though the app is user-installable, there is nothing to manage. Fastly runs the app entirely serverlessly, worldwide.

Getting started

To get started, clone the repo and use the fastly tool to deploy it:

git clone https://github.com/fastly/pubsub.git
cd pubsub
fastly compute publish

Assuming the fastly.toml file hasn't been modified, the above command will create a new Compute service and set up any related resources.

After that, enable Fanout on the service:

fastly products --enable=fanout

Add a backend called "self" to your service, for directing Fanout-managed requests back to the service itself. Replace {DOMAIN} with the domain selected during deployment:

fastly backend create \
  --name self \
  --address {DOMAIN} \
  --port 443 \
  --version latest --autoclone
fastly service-version activate --version latest

Determine the ID of the "secrets" Secret Store:

fastly resource-link list --version latest

Look for a block of output where the Resource Type is secret-store and the Name is whatever name you gave to the "secrets" Secret Store. The Resource ID is the store's ID.

Create a Fastly API token with the ability to publish Fanout messages. Do this by going to the Fastly management panel -> Accounts -> API tokens -> Personal tokens -> Create Token. Select Automation type, purge_select scope, limited to the service, and never expiring, and click Create Token. Then save the token in the above store:

fastly secret-store-entry create \
  -s {STORE_ID} \
  --name publish-token

The above command will prompt for the token value, which you can paste in.

Congrats, you have a pub/sub service! Next, we'll discuss how to use it.

Usage

Messages can be sent and received over HTTP or MQTT, and access must be authorized using tokens. In this section, we'll discuss how to send messages with HTTP.

First, you'll need a token signing key. Create one by sending a POST to the app's /admin/keys endpoint:

curl \
  -X POST \
  -H "Fastly-Key: $FASTLY_API_TOKEN" \
  https://{DOMAIN}/admin/keys

The app will respond with a key ID and value. Note them in a safe place. The value is used for signing JWTs. The ID must be included in the kid header field of the JWTs.

Once you have a signing key, you can create authorization tokens for subscribers and publishers as needed. Below is an example using Python and the PyJWT library to create a token allowing access to a topic named "test". Replace {KEY_ID} and {KEY_VALUE} with your key ID and value.

import jwt
import time

jwt.encode({
    "exp": int(time.time())+3600,
    "x-fastly-read": ["test"],
    "x-fastly-write": ["test"],
}, "{KEY_VALUE}", headers={"kid":"{KEY_ID}"})

Now that you have a token, you can subscribe a client. Here's an example using curl:

curl \
  -H "Authorization: Bearer $TOKEN" \
  "https://{DOMAIN}/events?topic=test"

The above command establishes a never-ending response body that the client can receive messages over.

And here's how to publish a message:

curl \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"text":"hello world"}'
  "https://{DOMAIN}/events?topic=test"

After running the above command, the earlier curl should output:

event: message
data: {"text":"hello world"}

For further details, including how to use the app with MQTT, see the README.

Questions/Comments?

Feel free to file issues on the GitHub repository for specific code related bugs or features, or chat with us on the Fastly Community Forum.