🔰 Beginners new to AWS CDK, please do look at my previous articles one by one in this series.
If in case missed my previous article, do find it with the below links.
🔁 Original previous post at 🔗 Dev Post
🔁 Reposted the previous post at 🔗 dev to @aravindvcyber
Also, we have started to develop an open source project which we would be using to play around with refracting the architecture as well as learn cdk stuff at the same time we will provide something useful for our community. Find more about this discussed in the article below.
🔁 Original project post at 🔗 Dev Post
🔁 Reposted project post at 🔗 dev to @aravindvcyber
Cross Account PutEvents 🍡
Earlier in our article, we have seen how to use custom Eventbridge and SQS by configuring an event rule and target which shifts the messages to sqs queue.
Original post at 🔗 Dev Post
Reposted at 🔗 dev to @aravindvcyber
But in this article, we will be using the default event bus only since we are trying to source the cloud formation events for the above-mentioned project. To start with we will be first discussing how to make the event bridge events be routed to an sqs queue in our main processor stack.
First, let us get the reference to the default target bus in the processor region
export const getDefaultBus = (
scope: Construct,
region: string,
account: string
): IEventBus => {
return EventBus.fromEventBusArn(
scope,
`DefaultBus-${region}-${account}`,
`arn:aws:events:${region}:${account}:event-bus/default`
);
};
Then we will create a target queue, a processor queue for lambda trigger, and processor dlq. Since we have to use the queue and dlq construct multiple times, let us refract them to some utils below.
export const dlqQueueProps = {
retentionPeriod: Duration.days(7),
removalPolicy: RemovalPolicy.DESTROY,
};
export const normalQueueProps = {
retentionPeriod: Duration.days(1),
removalPolicy: RemovalPolicy.DESTROY,
deliveryDelay: Duration.seconds(1),
visibilityTimeout: Duration.minutes(3),
};
export const generateDLQ = (scope: Construct, queueName: string): Queue => {
return new Queue(scope, queueName, {
...dlqQueueProps,
queueName,
});
};
export const generateQueue = (
scope: Construct,
queueName: string,
deadLetterQueue: DeadLetterQueue
): Queue => {
return new Queue(scope, queueName, {
...normalQueueProps,
queueName,
deadLetterQueue,
});
};
Let provision these necessary queues now
const stackEventProcessorQueue = generateQueue(
this,
"stackEventProcessorQueue",
stackEventProcessorQueueDLQ
);
const stackEventTargetDlq: DeadLetterQueue = {
queue: generateDLQ(this, "stackEventTargetDlq"),
maxReceiveCount: 100,
};
const stackEventProcessorQueueDLQ: DeadLetterQueue = {
queue: generateDLQ(this, "stackEventProcessorQueueDLQ"),
maxReceiveCount: 100,
};
Event Rule 🍻
Let us now source these cloud formation events from the current region
const stackEventsRule = new Rule(this, "stack-events-rule", {
eventPattern: {
detail: {
"stack-id": [{ exists: true }],
},
source: ["aws.cloudformation"],
},
eventBus,
});
And please remember this event pattern can be very extended to filter the relevant stack events only or source from all stacks like me and filter in the upcoming steps.
Event Rule target ☕
Now let us use the target dlq and processor queue to build our event rule target to wire up to the event rule which is defined above.
const stackEventTarget = new aws_events_targets.SqsQueue(
stackEventProcessorQueue,
{
retryAttempts: 3,
deadLetterQueue: stackEventTargetDlq.queue,
}
);
stackEventsRule.addTarget(stackEventTarget);
From then on the processor queue will take over the events as per the projects we discussed above and we are not interested in that in this article.
Multi-region or multi-account cloud formation events processing 🧃
What we have just seen is that we can source the events from the current region. But what if we may need to pull the events from another region or even another account? That is when we use the other stack part of the project which we will put into every account-region pair and source the cloud formation events necessary for our project.
TargetBus cross-account policy 🥢
const remoteAccounts = config.get("remoteAccounts").split(",");
const remoteRegions = config.get("remoteRegions").split(",");
remoteAccounts.map((account: string) => {
remoteRegions.map((region: string) => {
new CfnEventBusPolicy(this, `CrossAccountPolicy-${region}-${account}`, {
action: "events:PutEvents",
eventBusName: eventBus.eventBusName,
principal: account,
statementId: `Accept-PutEvents-From-${region}-${account}`,
// condition: {
// }
});
});
});
Here I have used an array of remote accounts or remote regions to be used allowed to forward events to the processor stack besides that you can be a little more restrictive by adding some conditions as per your requirement for security regions or to reduce unnecessary or unwanted resources accessing our target default bus.
Single region used for processing and storage 🏖️
And why we do why, this is because of multiple reasons.
- Need not deploy the complex processing setup in every region or accounts
- Also this also means we are also not starving our environment much our solution scales up reaching the limits of the respective services
- Easily to inspect the processing stack in a particular region with xray, cloudwatch, and we could have a single dashboard with cloudwatch metrics and it will be easier for maintainable
- Lightweight remote stacks are portable and we could very easily provision them in multiple regions or accounts with less footprint.
Remote stack configuration 🏜️
Remote Eventbus Rule
With the similar approach used earlier, we could extend this further in this remote stack by building an event bus rule and event bus target with a failed event queue
const targetAccount = config.get("account");
const targetRegion = config.get("region")
const targetBus = getDefaultBus(this, targetRegion, targetAccount);
const remoteBus = getDefaultBus(this, props.region, props.account)
const remoteStackEventsRule = new Rule(this, "remote-stack-events-rule", {
eventPattern: {
detail: {
"stack-id": [{ exists: true }],
},
source: ["aws.cloudformation"],
},
eventBus: remoteBus,
});
Remote Eventbus Target
Adding a dlq to the target like the processor stack once again but now we use an eventbus as the target which is the processor region default bus that we have referenced. This should be able to connect and push messages even across regions into even different accounts. Since we have already granted this using the policy above. Also, the event bus fails it puts into a regional dlq.
const remoteStackEventTargetDlq: DeadLetterQueue = {
queue: generateDLQ(this, "remoteStackEventTargetDlq"),
maxReceiveCount: 100,
}
const remoteStackEventTarget = new aws_events_targets.EventBus(targetBus, {
deadLetterQueue: remoteStackEventTargetDlq.queue,
})
remoteStackEventsRule.addTarget(remoteStackEventTarget);
Thus we can demonstrate how we would be able to pool cloud formation events across the region and even from different accounts. This will be extremely useful when you build similar event-driven solutions.
We will be talking about more similar engineering concepts as we refactor and refine the event forwarder project. Keep following for similar posts on engineering with IaC primarily using AWS CDK and Serverless.
Also feel free to contribute to the progress of the below solution with your comments, issues, maybe you can also do a pr if you feel if can help our community.
🔁 Original project post at 🔗 Dev Post
🔁 Reposted project post at 🔗 dev to @aravindvcyber
⏭ We have our next article in serverless and IaC, do check out
https://dev.to/aws-builders/aws-cdk-101-send-message-across-accounts-using-sns-topic-and-sqs-h2i
🎉 Thanks for supporting! 🙏
Would be great if you like to ☕ Buy Me a Coffee, to help boost my efforts 😍.
🔁 Original post at 🔗 Dev Post
🔁 Reposted at 🔗 dev to @aravindvcyber
Top comments (0)