How To Federate Events Between AWS Organizations for a Multi-Account SaaS Platform

In today’s cloud-native world, multi-account architectures are becoming increasingly common, especially for SaaS platforms. These architectures allow organizations to isolate resources, enforce security boundaries, and simplify billing. However, managing and federating events across multiple AWS accounts can be challenging. AWS EventBridge and SQS (Simple Queue Service) provide a powerful combination to build cross-account, event-driven architectures that enable seamless communication between accounts in a secure and scalable manner.

In this article, we’ll explore how to set up a cross-account event-driven architecture using AWS EventBridge and SQS. We’ll cover both CLI-based and AWS Console-based steps, including micro-steps, to ensure you can implement this solution in your own environment.

Understanding the Problem

In a multi-account SaaS platform, events generated in one account often need to be processed or acted upon in another account. For example:

• A new user sign-up event in Account A might trigger a workflow in Account B to provision resources.
• A billing event in Account C might need to be logged in Account D for compliance purposes.

To achieve this, we need a mechanism to securely and reliably share events across accounts. AWS EventBridge is a serverless event bus that allows you to route events between AWS services and accounts. When combined with SQS, it provides a decoupled, scalable, and durable way to process these events.

Architecture Overview

The architecture we’ll build consists of the following components:

1. Event Producer Account: The account where events are generated.
2. Event Consumer Account: The account where events are consumed.
3. EventBridge Event Bus: A custom event bus in the producer account to route events.
4. SQS Queue: A queue in the consumer account to receive and process events.
5. IAM Roles and Policies: To enable cross-account access and ensure security.

Step 1: Set Up the Event Producer Account

1.1 Create a Custom Event Bus in the Producer Account

AWS Console Steps:

1. Open the EventBridge console in the producer account.
2. In the left navigation pane, choose Event Buses.
3. Click Create event bus.
4. Enter a name for your event bus, e.g., CrossAccountEventBus.
5. Click Create.

CLI Steps:

Run the following command to create a custom event bus:

aws events create-event-bus --name CrossAccountEventBus --region us-east-1

1.2 Create an Event Rule to Route Events to the Consumer Account

AWS Console Steps:

1. In the EventBridge console, choose Rules from the left navigation pane.
2. Click Create rule.
3. Enter a name for the rule, e.g., RouteEventsToConsumerAccount.
4. Under Event bus, select the custom event bus you created (CrossAccountEventBus).
5. For Rule type, select Rule with an event pattern.
6. Click Next.
7. Define the event pattern. For example:

{
  "source": ["com.mycompany.producer"],
  "detail-type": ["UserSignUp"]
}

8. Click Next.
9. Under Targets, click Add target.
10. Choose SQS queue as the target type.
11. Select the SQS queue in the consumer account (we’ll create this in Step 2).
12. Click Next and then Create rule.

CLI Steps:

Run the following command to create the event rule:

aws events put-rule --name RouteEventsToConsumerAccount --event-pattern '{"source":["com.mycompany.producer"],"detail-type":["UserSignUp"]}' --event-bus-name CrossAccountEventBus --region us-east-1

Step 2: Set Up the Event Consumer Account

2.1 Create an SQS Queue in the Consumer Account

AWS Console Steps:

1. Open the SQS console in the consumer account.
2. Click Create queue.
3. Choose Standard queue.
4. Enter a name for the queue, e.g., CrossAccountEventQueue.
5. Click Create queue.

CLI Steps:

Run the following command to create the SQS queue:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::PRODUCER_ACCOUNT_ID:root"
      },
      "Action": "sqs:SendMessage",
      "Resource": "arn:aws:sqs:us-east-1:CONSUMER_ACCOUNT_ID:CrossAccountEventQueue"
    }
  ]
}

2.2 Update the SQS Queue Policy to Allow Cross-Account Access

AWS Console Steps:

1. In the SQS console, select the queue you created (CrossAccountEventQueue).
2. Click the Access policy tab.
3. Add the following policy to allow the producer account to send messages to the queue:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::PRODUCER_ACCOUNT_ID:root"
      },
      "Action": "sqs:SendMessage",
      "Resource": "arn:aws:sqs:us-east-1:CONSUMER_ACCOUNT_ID:CrossAccountEventQueue"
    }
  ]
}

4. Replace PRODUCER_ACCOUNT_ID and CONSUMER_ACCOUNT_ID with the appropriate account IDs.
5. Click Save.

CLI Steps:

Run the following command to update the queue policy:

aws sqs set-queue-attributes --queue-url https://sqs.us-east-1.amazonaws.com/CONSUMER_ACCOUNT_ID/CrossAccountEventQueue --attributes '{"Policy":"{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"arn:aws:iam::PRODUCER_ACCOUNT_ID:root\"},\"Action\":\"sqs:SendMessage\",\"Resource\":\"arn:aws:sqs:us-east-1:CONSUMER_ACCOUNT_ID:CrossAccountEventQueue\"}]}"}' --region us-east-1

Step 3: Link the Producer and Consumer Accounts

3.1 Add the SQS Queue as a Target in the Producer Account

AWS Console Steps:

1. In the EventBridge console of the producer account, navigate to the rule you created earlier (RouteEventsToConsumerAccount).
2. Click Edit.
3. Under Targets, click Add target.
4. Choose SQS queue as the target type.
5. Select the SQS queue in the consumer account (CrossAccountEventQueue).
6. Click Next and then Update rule.

CLI Steps:

Run the following command to add the SQS queue as a target:

aws events put-targets --rule RouteEventsToConsumerAccount --targets "Id"="1","Arn"="arn:aws:sqs:us-east-1:CONSUMER_ACCOUNT_ID:CrossAccountEventQueue" --event-bus-name CrossAccountEventBus --region us-east-1

Step 4: Test the Setup

4.1 Send a Test Event from the Producer Account

AWS Console Steps:

1. In the EventBridge console of the producer account, navigate to the custom event bus (CrossAccountEventBus).
2. Click Send events.
3. Enter the following event details:

{
  "Source": "com.mycompany.producer",
  "DetailType": "UserSignUp",
  "Detail": "{\"userId\":\"123\",\"email\":\"[e-mail]\"}"
}

4. Click Send.

CLI Steps:

Run the following command to send a test event:

aws events put-events --entries '[{"Source":"com.mycompany.producer","DetailType":"UserSignUp","Detail":"{\"userId\":\"123\",\"email\":\"[e-mail]\"}","EventBusName":"CrossAccountEventBus"}]' --region us-east-1

4.2 Verify the Event in the Consumer Account

AWS Console Steps:

1. In the SQS console of the consumer account, select the queue (CrossAccountEventQueue).
2. Click Poll for messages.
3. Verify that the test event appears in the queue.

CLI Steps:

Run the following command to receive messages from the queue:

aws sqs receive-message --queue-url https://sqs.us-east-1.amazonaws.com/CONSUMER_ACCOUNT_ID/CrossAccountEventQueue --region us-east-1

Step 5: Process Events in the Consumer Account

Once events are in the SQS queue, you can process them using AWS Lambda, EC2 instances, or any other compute service. Here’s an example of using AWS Lambda to process events:

5.1 Create a Lambda Function in the Consumer Account

AWS Console Steps:

1. Open the Lambda console in the consumer account.
2. Click Create function.
3. Choose Author from scratch.
4. Enter a name for the function, e.g., ProcessCrossAccountEvents.
5. Choose a runtime, e.g., Python 3.9.
6. Click Create function.
7. Add the following code to the function:

import json

def lambda_handler(event, context):
    for record in event['Records']:
        message = json.loads(record['body'])
        print("Received event:", message)
    return {
        'statusCode': 200,
        'body': json.dumps('Event processed successfully')
    }

8. Click Deploy.

CLI Steps:

Run the following commands to create and deploy the Lambda function:

aws lambda create-function --function-name ProcessCrossAccountEvents --runtime python3.9 --handler lambda_function.lambda_handler --role arn:aws:iam::CONSUMER_ACCOUNT_ID:role/lambda-execution-role --zip-file fileb://function.zip --region us-east-1

5.2 Trigger the Lambda Function from the SQS Queue

AWS Console Steps:

1. In the SQS console, select the queue (CrossAccountEventQueue).
2. Click Lambda triggers.
3. Click Configure Lambda function trigger.
4. Select the Lambda function you created (ProcessCrossAccountEvents).
5. Click Save.

CLI Steps:

Run the following command to add the Lambda trigger:

aws lambda create-event-source-mapping --event-source-arn arn:aws:sqs:us-east-1:CONSUMER_ACCOUNT_ID:CrossAccountEventQueue --function-name ProcessCrossAccountEvents --region us-east-1

Conclusion

By following the steps outlined in this article, you can build a robust cross-account event-driven architecture using AWS EventBridge and SQS. This architecture is ideal for multi-account SaaS platforms, enabling seamless event federation while maintaining security and scalability. Whether you prefer the AWS Console or CLI, the process is straightforward and can be adapted to meet your specific requirements.

With this setup, you can unlock the full potential of event-driven architectures in a multi-account environment, ensuring your SaaS platform is both flexible and future-proof.