AWS Tutorial: Using LocalStack for testing AWS SNS and SQS Locally with Filter Policies

Introduction

Working with AWS services locally can be challenging due to cloud dependencies, costs, and network constraints. LocalStack helps overcome these issues by providing a fully functional local simulation of AWS services. It allows developers to run and test cloud applications in a controlled environment without needing access to real AWS infrastructure.

With a simple Docker setup, LocalStack enables you to experiment with multiple AWS services, such as SNS (Simple Notification Service) and SQS (Message Queuing Service), making it ideal for sandbox projects and integration testing. However, it's important to remember that LocalStack is a simulated environment and not a fully integrated AWS service.

In this guide, we'll explore how to use LocalStack with AWS CLI commands to set up and test messaging services like SNS and SQS.

---

Environment Preparation

Installing AWS CLI for LocalStack

If you haven’t installed the AWS CLI yet, you can download it from: https://aws.amazon.com/fr/cli/

Verify the installation:

aws --version

Setting up LocalStack

• Install LocalStack:
  Create a LocalStack account here and follow the setup documentation for your environment (ensure Docker is installed).

• Start LocalStack:
  

localstack start -d

• Verify LocalStack Instance: Go to the LocalStack website UI: LocalStack Cloud, and under the "Status" section, check if the instance and services are running.

---

Configuring AWS CLI for LocalStack

Using AWS CLI Profile

To configure AWS CLI for LocalStack, create a new profile named dev-account:

aws configure set aws_access_key_id "dev-account-key-id" --profile dev-account
aws configure set aws_secret_access_key "dev-account-access-key" --profile dev-account
aws configure set region "eu-west-1" --profile dev-account

The information is stored in ~/.aws/credentials and ~/.aws/config.

---

This tutorial demonstrates how to send a message to an SNS topic and process it through an SQS queue using a FilterPolicy.

Architecture schema

Creating SNS Topic and SQS Queue in LocalStack

Create SNS Topic (FIFO)

Create bookingairline.fifo

aws sns create-topic \
  --endpoint-url=http://localhost:4566 \
  --name bookingairline.fifo \
  --profile dev-account \
  --region eu-west-1 \
  --attributes '{"FifoTopic": "true", "ContentBasedDeduplication": "true"}' \
  --output table | cat

Note: FIFO topics must end with .fifo.

Create SQS Queues (FIFO)

Create airbus.fifo

aws sqs create-queue \
  --endpoint-url=http://localhost:4566 \
  --queue-name airbus.fifo \
  --profile dev-account \
  --region eu-west-1 \
  --attributes FifoQueue=true,ContentBasedDeduplication=true \
  --output table | cat

Create boeing.fifo

aws sqs create-queue \
  --endpoint-url=http://localhost:4566 \
  --queue-name boeing.fifo \
  --profile dev-account \
  --region eu-west-1 \
  --attributes FifoQueue=true,ContentBasedDeduplication=true \
  --output table | cat

Note: FIFO queues must end with .fifo.

Attributes and Parameters

Parameters	Explanation
VisibilityTimeout	Time (in seconds) that a message is hidden after being received.
MessageRetentionPeriod	Time (in seconds) a message stays in the queue before deletion. Default: 345600 (4 days).
DelaySeconds	Delay (in seconds) before a message becomes visible. Max: 900 (15 minutes).
MaximumMessageSize	Max size (in bytes) of a message. Default: 262144 (256 KB).
ReceiveMessageWaitTimeSeconds	Long polling wait time (in seconds).
FifoQueue	true for FIFO queues. Cannot be changed after queue creation.
ContentBasedDeduplication	true to enable deduplication for FIFO queues.

Attributes	Explanation
FifoTopic	Indicating whether the topic is a FIFO (First-In-First-Out) topic. FIFO ensures message order is maintained.
ContentBasedDeduplication	Deduplication prevents messages with the same content from being processed twice for FIFO topics.

---

SNS Subscription to SQS Queue

Subscribe airbus.fifo to SNS

aws sns subscribe \
  --endpoint-url=http://localhost:4566 \
  --topic-arn arn:aws:sns:eu-west-1:000000000000:bookingairline.fifo \
  --profile dev-account \
  --protocol sqs \
  --notification-endpoint arn:aws:sqs:eu-west-1:000000000000:airbus.fifo \
  --region eu-west-1 \
  --output table | cat

Subscribe boeing.fifo to SNS

aws sns subscribe \
  --endpoint-url=http://localhost:4566 \
  --topic-arn arn:aws:sns:eu-west-1:000000000000:bookingairline.fifo \
  --profile dev-account \
  --protocol sqs \
  --notification-endpoint arn:aws:sqs:eu-west-1:000000000000:boeing.fifo \
  --region eu-west-1 \
  --output table | cat

Parameters and Protocols

Parameters	Explanation
topic-arn	The ARN (Amazon Resource Names) of the SNS topic that you want to subscribe to.
protocol	This specifies the protocol for the subscription (see below for more protocol).
notification-endpoint	This is the ARN (Amazon Resource Names) of the AWS Lambda function that will process the messages.
endpoint-url	In the case of LocalStack or a local setup, this would be http://localhost:4566. This is where the AWS CLI will send requests.

Protocols	Explanation
HTTP/HTTPS	Delivers messages to an HTTP(s) endpoint as a POST request.
Email/Email-JSON	Sends the message as an email or as a JSON-formatted email to the subscriber's email address.
SMS	Sends the message as a text message (SMS) to the subscriber's phone number.
SQS	Sends the message to an Amazon SQS queue.
Application	Sends the message to a mobile application (for example, a mobile push notification).
Lambda	Invokes an AWS Lambda function with the SNS message as the event.

---

Add a filter to the SNS Topic to filter messages

Note: For this part, you have to modify the subscription-arn with the subscription that you have created before for airbus and boeing.

Add FilterPolicy for airbus.fifo queue

aws sns set-subscription-attributes \
  --endpoint-url=http://localhost:4566 \
  --subscription-arn arn:aws:sns:eu-west-1:000000000000:bookingairline.fifo:a43062ba-55cd-4fd8-8c81-88480985a889 \
  --attribute-name FilterPolicy \
  --attribute-value '{"Type": ["AirbusAirline"]}' \
  --region eu-west-1 \
  --profile dev-account \
  --output json | cat

Add FilterPolicy for boeing.fifo queue

aws sns set-subscription-attributes \
  --endpoint-url=http://localhost:4566 \
  --subscription-arn arn:aws:sns:eu-west-1:000000000000:bookingairline.fifo:02d28ea6-e3ea-48e7-92cf-02ac0e8cf3b7 \
  --attribute-name FilterPolicy \
  --attribute-value '{"Type": ["BoeingAirline"]}' \
  --region eu-west-1 \
  --profile dev-account \
  --output json | cat

• subscription-arn: The ARN (Amazon Resource Names) of the SNS topic subscription that you have created before for the airbus.fifo or boeing.fifo queue.
• attribute-name: The attribute name to use FilterPolicy.
• attribute-value: The type of message to filter. In this case, only AirbusAirline type will be forwarded to the airbus.fifo queue and the BoeingAirline type will be forwarded to the boeing.fifo.

---

Publishing Messages to SNS

This example is to show that the filterpolicy is working.

Create Message Attributes for AirbusAirline

You can create a JSON file (for exemple: message-attributes-airbus.json) to prepare a body to send via the AWS CLI :

message-attributes-airbus.json

{
  "Type": { "DataType": "String", "StringValue": "AirbusAirline" },
  "FlightID": { "DataType": "String", "StringValue": "AIRBUS_456" }
}

Publish Message

Now that we have attributes, we can send it to the SNS Topic :

aws sns publish \
  --endpoint-url=http://localhost:4566 \
  --topic-arn arn:aws:sns:eu-west-1:000000000000:bookingairline.fifo \
  --message 'New booking flight for Airbus has been created!' \
  --message-attributes file://message-attributes-airbus.json \
  --message-group-id 'MessageGroup1' \
  --region eu-west-1 \
  --profile dev-account \
  --output json | cat

We can see on Localstack that airbus.fifo has received the message.

Create Message Attributes for BoeingAirline

You can create a JSON file (for exemple: message-attributes-boeing.json) to prepare a body to send via the AWS CLI :

message-attributes-boeing.json

{
  "Type": { "DataType": "String", "StringValue": "BoeingAirline" },
  "FlightID": { "DataType": "String", "StringValue": "BOEING_962" }
}

Publish Message

Now that we have attributes, we can send it to the SNS Topic :

aws sns publish \
  --endpoint-url=http://localhost:4566 \
  --topic-arn arn:aws:sns:eu-west-1:000000000000:bookingairline.fifo \
  --message 'New booking flight for Boeing has been created!' \
  --message-attributes file://message-attributes-boeing.json \
  --message-group-id 'MessageGroup1' \
  --region eu-west-1 \
  --profile dev-account \
  --output json | cat

Parameters	Explanation
message-group-id	The message group ID to use for the message. This is used to group messages that are sent to the same queue.
message-attributes	The message attributes to use for the message. In this case, we are using the FilterPolicy attribute to filter messages.
message	The message to send to the SNS Topic.
topic-arn	The ARN (Amazon Resource Names) of the SNS Topic that you want to send the message to.
endpoint-url	In the case of LocalStack or a local setup, this would be http://localhost:4566. This is where the AWS CLI will send requests.

---

Consuming Messages from SQS Queues

aws sqs receive-message \
  --endpoint-url=http://localhost:4566 \
  --queue-url http://localhost:4566/000000000000/airbus.fifo \
  --profile dev-account \
  --region eu-west-1 \
  --output json | cat

aws sqs receive-message \
  --endpoint-url=http://localhost:4566 \
  --queue-url http://localhost:4566/000000000000/boeing.fifo \
  --profile dev-account \
  --region eu-west-1 \
  --output json | cat

Launching this command will display the messages in the SQS Queue for airbus.fifo and the other for boeing.fifo.

---

Cleaning Up Resources

aws sns delete-topic \
  --endpoint-url=http://localhost:4566 \
  --region eu-west-1 \
  --profile dev-account \
  --topic-arn arn:aws:sns:eu-west-1:000000000000:bookingairline.fifo

aws sqs delete-queue \
  --endpoint-url=http://localhost:4566 \
  --region eu-west-1 \
  --profile dev-account \
  --queue-url http://localhost:4566/000000000000/airbus.fifo

aws sqs delete-queue \
  --endpoint-url=http://localhost:4566 \
  --region eu-west-1 \
  --profile dev-account \
  --queue-url http://localhost:4566/000000000000/boeing.fifo

localstack stop

---

Conclusion

LocalStack provides a powerful way to simulate AWS services locally, making it an excellent tool for sandbox projects and integration testing. With a simple Docker setup, you can experiment with multiple AWS services without incurring costs or relying on cloud infrastructure.

By using AWS CLI commands, as demonstrated above, you can create and test essential services like SNS and SQS in a local environment. However, since LocalStack is a simulated tool and not a fully integrated AWS service, it's important to validate your workflows before deploying them to a real AWS environment.

With LocalStack, developers can streamline their testing process, improve efficiency, and reduce dependencies on live cloud resources, making development and debugging more seamless.

Thank you for reading!

If you have any questions, feedback, or suggestions, please feel free to leave them in the comments below. I'm eager to hear from you and respond to your thoughts!