Automating Well-Architected reviews

I've been leading our Well-Architected partnership at Knowit for some years now.

Last year, AWS introduced a workshop for using automation in conducting reviews, and it piqued my interest. Automating reviews can significantly enhance efficiency and accuracy, so I decided to delve into it further.

To gain deeper insights, I traveled to the AWS offices in Munich to join the Well-Architected team for a day-long workshop. Doing hands on labs against prebuilt environments to gain insight into possibilities. The experience was enlightening and reinforced the potential benefits of automating reviews wherever possible.

Integrating this into our reviews at Knowit required minimal discussion.

For the duration of this blog post, I assume you're familiar with Well-Architected Framework by AWS. If you're not take a look at the below link before reading forward:

https://aws.amazon.com/architecture/well-architected/

Can well-architected be automated?

Not all aspects of the Well-Architected Framework can be automated. For example, understanding how people operate, assessing the client’s processes, and their perspective on their workload are inherently human tasks. So prepare to talk to people still.

However, technical aspects, especially those within the Security pillar, along with some Cost Optimization and Sustainability items, can be automated as they align closely with your system’s usage and data.

If you can formulate a request to AWS APIs that will provide the answers you need, it can be automated.

What did we do?

In our quest to make the Well-Architected practitioner’s (person conducting the review( work easier, we took a look at multiple approaches. We found that integrating Prowler and Steampipe into our workflow was particularly effective.

1. Data Collection with Prowler: We used Prowler, a security tool that performs AWS security best practices assessments, audits, incident response, continuous monitoring, and hardening. Prowler crawled our AWS environment to gather insights into clients security posture.
2. Data Aggregation with Steampipe: Steampipe allowed us to query cloud resources using SQL. We used it to aggregate data across multiple AWS accounts, making it easier to gather comprehensive insights from accounts in AWS organizations.
3. Centralized Reporting: The data collected through Prowler was fed into the AWS Well-Architected Tool in a centralized account. This allowed us to consolidate our findings and generate a comprehensive report.

Running these automatically monthly, will give you insight into how you're progressing with your environment.

Another tool worth mentioning is Former2, I've used that a few times to create visualization for the review report in form of architectural diagrams from the environment in the target account.

By running these scans as a prerequisite to the review workshop, the practitioner can gain a better understanding of the client’s environment before the workshop even begins. This allows for a more informed discussion of the findings during the workshop, moving beyond assumptions to having an automated, data-driven view into the account.

Want to scan your own account for findings?

I recommend you try prowler and see what kind of findings are in your AWS account, try the following on your Mac (if you have windows, consider running this on AWS cloud 9). Example assumes you have AWS CLI already installed and read-only access working to your AWS account.

# install prowler
brew install prowler

# run the scan
prowler aws -f <insert your aws region here> --compliance aws_well_architected_framework_security_pillar_aws -p <aws-cli-profile>

You'll get something along the lines (sensitive information retracted):

 _ __  _ __ _____      _| | ___ _ __
| '_ \| '__/ _ \ \ /\ / / |/ _ \ '__|
| |_) | | | (_) \ V  V /| |  __/ |
| .__/|_|  \___/ \_/\_/ |_|\___|_|v4.2.4
|_| the handy multi cloud security tool

Date: today, right now

-> Using the AWS credentials below:
  · AWS-CLI Profile: your-profile
  · AWS Regions: eu-west-1
  · AWS Account: account-id

-> Using the following configuration:
  · Config File: config-yaml
  · Scanning unused services and resources: False

Executing 227 checks, please wait...
-> Scan completed! |▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉| 227/227 [100%] in 1:34.9 

Overview Results:
╭─────────────────────┬─────────────────────┬────────────────╮
│ 37.91% (105) Failed │ 60.29% (167) Passed │ 0.0% (0) Muted │
╰─────────────────────┴─────────────────────┴────────────────╯

Account account-id Scan Results (severity columns are for fails only):
╭────────────┬───────────────┬───────────┬────────────┬────────┬──────────┬───────┬─────────╮
│ Provider   │ Service       │ Status    │   Critical │   High │   Medium │   Low │   Muted │
├────────────┼───────────────┼───────────┼────────────┼────────┼──────────┼───────┼─────────┤
│ aws        │ account       │ PASS (0)  │          0 │      0 │        0 │     0 │       0 │
├────────────┼───────────────┼───────────┼────────────┼────────┼──────────┼───────┼─────────┤
│ aws        │ cloudtrail    │ FAIL (4)  │          0 │      0 │        1 │     3 │       0 │
├────────────┼───────────────┼───────────┼────────────┼────────┼──────────┼───────┼─────────┤
│ aws        │ cloudwatch    │ FAIL (15) │          0 │      0 │       15 │     0 │       0 │
├────────────┼───────────────┼───────────┼────────────┼────────┼──────────┼───────┼─────────┤
│ aws        │ config        │ FAIL (1)  │          0 │      0 │        1 │     0 │       0 │
├────────────┼───────────────┼───────────┼────────────┼────────┼──────────┼───────┼─────────┤
│ aws        │ ec2           │ FAIL (12) │          0 │      1 │        8 │     3 │       0 │
├────────────┼───────────────┼───────────┼────────────┼────────┼──────────┼───────┼─────────┤
│ aws        │ emr           │ PASS (1)  │          0 │      0 │        0 │     0 │       0 │
├────────────┼───────────────┼───────────┼────────────┼────────┼──────────┼───────┼─────────┤
│ aws        │ guardduty     │ FAIL (1)  │          0 │      0 │        1 │     0 │       0 │
├────────────┼───────────────┼───────────┼────────────┼────────┼──────────┼───────┼─────────┤
│ aws        │ iam           │ FAIL (56) │          1 │     27 │       21 │     7 │       0 │
├────────────┼───────────────┼───────────┼────────────┼────────┼──────────┼───────┼─────────┤
│ aws        │ macie         │ FAIL (1)  │          0 │      0 │        0 │     1 │       0 │
├────────────┼───────────────┼───────────┼────────────┼────────┼──────────┼───────┼─────────┤
│ aws        │ organizations │ FAIL (1)  │          0 │      0 │        1 │     0 │       0 │
├────────────┼───────────────┼───────────┼────────────┼────────┼──────────┼───────┼─────────┤
│ aws        │ route53       │ FAIL (3)  │          0 │      0 │        3 │     0 │       0 │
├────────────┼───────────────┼───────────┼────────────┼────────┼──────────┼───────┼─────────┤
│ aws        │ s3            │ FAIL (9)  │          0 │      1 │        8 │     0 │       0 │
├────────────┼───────────────┼───────────┼────────────┼────────┼──────────┼───────┼─────────┤
│ aws        │ securityhub   │ FAIL (1)  │          0 │      0 │        1 │     0 │       0 │
├────────────┼───────────────┼───────────┼────────────┼────────┼──────────┼───────┼─────────┤
│ aws        │ vpc           │ FAIL (1)  │          0 │      0 │        1 │     0 │       0 │
╰────────────┴───────────────┴───────────┴────────────┴────────┴──────────┴───────┴─────────╯
* You only see here those services that contains resources.

Detailed results are in:
 - JSON-OCSF: output/prowler-output-accountid-datetime.ocsf.json
 - CSV: output/prowler-output-accountid-datetime.csv
 - HTML: output/prowler-output-accountis-datetime.html

Now take a look at the output files, to see where your environment is failing. And see if you can remediate that.

Useful links

https://steampipe.io/
https://github.com/prowler-cloud/prowler
https://former2.com

Disclaimer

The pictures in this blog post are AI generated, and have clear mistakes in them. They're for visual illustration only.