Domain 1 SDLC Automation - Key AWS Services
CodePipeline
Orchestrates CI/CD with Source → Build → Test → Deploy → Invoke stages
CodeBuild
- Uses buildspec.yml for build configuration
- Key features:
- Artifacts stored in S3
- Supports environment variables, SSM, Secrets Manager
- Can run in VPC
- Provides test visualization
- If you need to cache Docker images you can setup Docker layer cache inside CodeBuild
CodeDeploy
- Key concepts:
- Uses appspec.yml
- Requires CodeDeploy Agent on EC2
- Supports in-place and blue/green deployments
- Deployment Options:
- EC2/On-premises: In-place or blue/green
- ECS: Blue/green only, supports canary/linear
- Lambda: Blue/green only, version-based
CodeArtifact
Package management service with multi-account access and upstream repositories
CodeGuru
- Reviewer: Automated code reviews
- Profiler: Runtime performance optimization
EC2 Image Builder
Automates AMI and Container image creation with scheduling capabilities
Defines the image creation process, which can then be utilized by all other departments
Whenever they talk about AMI think about image builder native functions. Also, when building base container images without application code, think of using AMI creation rather than CodeBuild.
AWS RAM
- Control sharing AMI across accounts, minimizing permission config overhead
- Use AWS RAM to share Images, Recipes and Components across AWS accounts or AWS Organizations
AWS Amplify
Simplified deployment platform for web and mobile applications
Domain 2 - Configuration
Management and IaC
CloudFormation
- Template uploaded to S3
- Deploy using Application Composer, YAML file with CLI or CD tool
- Almost every service included, you can work with no supported AWS service using CloudFormation Custom Resources
- CloudFormation custom resource that invokes Lambda used for some conditions to check current state of accounts
-
Template Components
- AWSTemplateFormatVersion
- Description
- Resources (Mandatory)
service-provider::service-name::data-type-name
- there are 700+ resources
- Parameters
- Type, Description, ConstraintDescription, Min/MaxLenght, Min/MaxValue, Default, AllowedValues(array), AllowedPattern(regex), NoEcho(Boolean)
- Reference:
!Ref MyVPC
orFn::Ref MyVPC
- Pseudo Parameters:
AWS::AccountID, AWS::Region, AWS::StackId, AWS::StackName, AWS::NotificationARNs, AWS::NoValue
-
Mappings
- You Just Declare HashMap to use it later
-
Fn::FindInMap
or!FindInMap[MapName, TopKey, SecondKey]
eg.
ImageId: !FindInMap [RegionMap, !Ref "AWS::Region", HVM64]
-
Outputs
- Export generated variable so other CloudFormation stacks can use it
Outputs: StackSSHSecurityGroup: Value: !Ref MyCompanyWideSSHSecurityGroup Export: Name: SSHSecurityGroup # Use it from other stack SecurityGroups: - !ImportValue SSHSecurityGroup
Conditionals
!And, !Equals, !If , !Not, !Or
# How to create Conditions: CreateProdResources: !Equals [ !Ref EnvType, prod] # Usage example Resources: MountPoint: Type: AWS::EC2::VolumeAttachment Condition: CreateProdResources
-
Intrinsic Functions
- !Ref - Use other variable value
- !GetAtt - Get attribute from resource eg.
!GetAtt EC2Instance.AvailabilityZone
- !FindInMap
- !ImportValue - value from other stacks
- ConditionFunctions (!If, !Not, !Equals etc…)
- !Base64
!Base64 "ValueToEncode" # Encoded userdata UserData: Fn::Base64: | #!/bin/bash dnf update -y dnf install -y httpd
Service Role - By default it uses your permissions, if you passRole stack uses permissions from that role only
Capabilities - needs to declare capabilities in CLI or Control Panel for specific cases eg.
CAPABILITY_NAMED_IAM, CAPABILITY_IAM, CAPABILITY_AUTO_EXPAND, InsufficientCapabilitiesException
-
DeletionPolicy
- Delete - default policy, S3 bucket cannot deleted if it is not empty
- Retain - Resources retains
- Snapshot - Creates one final snapshot before deleted, supports few resources
# Example Resources: MyEC2: Type: AWS::EC2::Instance DeletionPolicy: Delete
Stack Policies - Its a JSON document something like bucket policy, to ALLOW or Deny updates to specific resources
Termination Protection - Additional stack delete protection
-
Dynamic Reference
- Get value from SSM, SSM Secure and Secrets Manager
resolve:ssm:parameter-name:version
EC2 User data can be saved into
Fn::Base64
-
cfn Helper Scripts - cfn scripts comes with Amazon Linux and you can install others by yum etc.. After Cloudformation provided EC2 resources, after that cfn scans scripts from Cloudformation inside EC2 and retrieves scripts.
- cfn-init
- helps complex EC2 config more readable
- need AWS::CloudFormation::Init under the Metadata
- cfn-signal
- receives signal from EC2 to CFN that cfn-init completed successfully or not
- If signal received failure, CFN rollbacks stack
- Need to define custom WaitCondition
- cfn-get-metadata
- cfn-hup → Check metadata change every 15 minutes
- cfn-init
Nested Stacks - Use already exist template to create child stack inside current stack
DependsOn - Before creating this resource wait for dependent stack to be created, eg.
DependsOn: DBInstance
-
StackSets - Create stacks from admin account, and it will be created in all child account and different regions
- Self-managed Permissions: Create IAM roles with established trusted relationship between accounts
- Service-managed Permissions: relationships handled auto by AWS Organizations, auto deployment to new accounts into that organization
ChangeSets - Create change set and make sure the resources are correct and what is gonna be change, then if you execute the resources actually will be created
cfn-hup
- Look for Metadata changes of EC2, and update it every 15 minutesStackSet Drift - Detects changes manually did by somebody
AWS Service Catalog
- Admins creates bunch of CFN templates
- Users cannot create new template only can uses created by admins
- Stack Set Constraints - Each template can be configured to accounts, regions, permissions
- Launch Constraints - Is a Role used to create resources in this template, even real used don’t have permission
AWS Beanstalk
- Deployment modes
- All at once - stop deploy start
- Rolling - deploy to instances one by one
- Rolling with additional batches - with additional instances
- Immutable - Create new environment then swap and delete old environment
- Blue Green - create new environment and swap when ready
- Traffic Splitting - canary
- We can setup web and worker tier
- Integrate with EventBridge to SNS and Lambda
AWS SAM
- Serverless Application Model
- Sam can use CodeDeploy to deploy Lambda functions
- SAM can help run Lambda, API Gateway, DynamoDB locally
- SAM Accelerate
CDK
- SAM focused on Serverless services, with CDK you can build any aws service, with your favorite programming language
- You can use SAM CLI to locally test your CDK apps
- Commands:
cdk init --language python
cdk bootstrap
cdk synth # optional synth as a CFN template
cdk deploy
cdk destroy #first empty buckets if there is files
AWS AppConfig - Feature flag and configuration service
AWS SSM
- Choose SSM if it says use fleet of EC2 instances, or patching
- AWS OpsWorks → Deprecated, we have OpsCenter in SSM
- Documents - JSON or YAML, with parameters and actions
- Run Command - Executes Documents or just Run command. Triggered by EventBridge, output can seen CloudWatch and S3, could integrated with SNS, using resource groups command can run in multiple instances
- Automation - auto ebs snapshots, restart instances etc, triggered by scheduled (Maintenance Windows), EventBridge or manually
- Parameter Store
- Standard - 10,000 parameters, 4KB, no policy, Free
- Advanced - 100,000 parameters, 8KB, yes policy, charges apply
- Policies allow to assign TTL to a parameter, can assign multiple parameter at a time
- Patch Manager
- OS updated, app updates, security updates. Supports linux, mac, windows
- Patch Baseline - configure what patches are allowed or not
- Patch Group - for different envs (dev, prod etc.)
- Maintenance Windows - Define schedule for when to perform actions
- SSM State Manager - Automating process of maintaining consistent configurations. Eg: Ensure some app installed across all instances
- Session Manager - shell on your EC2 and on-premise servers, no need ssh, with IAM permissions you can restrict which instance and what command to run, export logs to s3 and CloudWatch logs
- Default Host Management Configuration (DHMC)
- new way to control your EC2 instances with SSM without EC2 Instance Profile, must be enabled per AWS Region, SSM Agent should be installed
- Auto enables Session Manager, Patch Manager and Inventory
- Hybrid Environments
- You can manage on-premise servers, IoT devices etc..
- EC2 instance “i-” on premises “mi-” prefix
- can automate with API GW and Lambda so they send request to SSM and get Activation code & ID
- You can also register IoT devices using IoT Greengrass
- SSM Automation Use cases
- Reduce cost by scheduled EC2 instance stop and start
- Reduce cost by scheduled downsize of EC2 and RDS instances
- Build golden AMI
- Compliance - Displays current data about Patch Manager, State Manager and sync data into S3 using Resource Data Sync and you can Analyze with Athena and QuickSight, then can send data to Security Hub
- OpsCenter - Allows you to view investigate issues in one place
- Security issues, performance issues, failures
- If you wanna access and manage instances in VPC private subnet, you have to create VPC endpoints for those instances
Doman 3 - Resilient Cloud Solutions
AWS Lambda
- Lambda provisioned concurrency → means you have Lambda functions pre-loaded in memory ready for immediate use (expensive)
- Lambda reserved concurrency → means you have a guaranteed number of concurrent executions reserved at all times
- AWS Lambda Internal, External Extension
EKS
- ControlPlane fully managed by AWS
- Node Types
- Managed Node Groups - creates and Manages Nodes (EC2 instances) for you
- Self-Managed Nodes - nodes created by you and registered to the EKS cluster and managed by and ASG
- AWS Fargate
- Data Volumes
- EBS, EFS (works with Fargate), FSx for Lustre, FSx for NetApp ONTAP
- Container Service Storage(CSI) - You have to install CSI Driver from helm or create EKS add-on(recommended by AWS) in order to integrate with Data Volumes
- Control Plane Logging - Send EKS Control Plane audit and diagnostic logs to CloudWatch Logs
- Ability to select the exact log types to send to CloudWatch
- Node and Container Logging
- Use CloudWatch Agent to send metrics to CloudWatch
- Use Fluent Bit, or Fluentd log drivers to send logs to CloudWatch Logs
- Pod Identity Access - When pods wanna call AWS API, they need pod level role in that case need to map IAM Roles to K8s Service Accounts (new way)
- IAM Roles for Service Accounts (IRSA) - Old way of Identity Access, uses K8s ServiceAccounts as IdP and connects thought OpenId Connect and assigns IAM Roles to the ServiceAccounts
- You need to use IRSA if you use Promotheus or OpenTelemetry
- AutoScaling:
- Kubernetes Metrics Server - Kubernetes feature that collects resource metrics from kubelets, metrics can be accessed by API
- Horizontal Pod Autoscaler - Increases pod count depending on metrics
- Vertical Pod AutoScaler - Increases pod specs
- Kubernetes Cluster AutoScaler - Node scaling
- Prefetching container images - means pulls image to the pods before they needed
Kinesis
- Data Streams - Enhanced one has 2MB/s per shard per consumer in other hand normal one has 2MB/s per shard for all consumers, retention between 1-365 days
- Data Firehose
- Analytics - analyze data streams with SQL or Apache Flink
- Video Streams
AWS Storage Gateway
- File Gateway
- RefreshCacheAPI - if somebody uploaded directly to S3 you File GW don’t know about that file, so need to invoke this API
- Automating Cache Refresh - Automatically and Periodically refresh cache, knows new uploaded data right away
- Volume Gateway
- Tape Gateway
ASG
- Dynamic Scaling
- Target Tracking Scaling - average CPU stay 40%
- Simple/Step Scaling - if X% Then scale up down Z
- Scheduled Scaling
- Predictive Scaling - continuously forecasting
- Warm Pools
- Then can be running, stopped, hibernated states
- Cheaper than over provisioning
ALB
- DualStack - Allow both ip4, ip6 access you can disable ip4 so you don’t pay for public ip4 addresses
- ALB health check grace period: Some amount of time to setup so ALB does not care health check at that time
- Cross-zone Load Balancing: If disabled, ALB sends traffic only to targets in the own AZ. If enabled, it distributes traffic across targets in different AZs as well.
AWS Application Auto Scaling - Monitors your apps and automatically adjusts capacity to maintain steady.
ECR
- Fully integrated with ECS, backed by Amazon S3 (image layers stored in S3)
- Built image replication feature
- ECR can be configured to submit Rejected status to the CICD pipeline when the image scan returns CRITICAL or HIGH findings
- Image vulnerability scanning - basic and enhanced scanning. Enhanced version scans continuously
Domain 4 - Monitoring and Logging
CloudWatch Metrics
- Built-in metrics for all AWS services
- Metric Streams provide near real-time delivery
- Custom metrics via PutMetricData API (±2 weeks window)
- ML-based Anomaly Detection that can trigger alarms
CloudWatch Logs
- Organized in log groups and streams
- Can export to S3, Kinesis, Lambda, OpenSearch
- Supports multi-account/region aggregation
- EC2 requires CloudWatch agent installation
CloudWatch Alarms
- Trigger notifications based on metrics
- Three states: OK, INSUFFICIENT_DATA, ALARM
- Can target EC2, ASG, SNS
- Composite Alarms combine multiple metrics with AND/OR logic
- Synthetic Canary for API/website monitoring
Amazon Athena
- SQL queries for S3 data (CSV, JSON, ORC, Parquet)
- Federated Query supports various data sources via Lambda
Domain 5 - Incident and Event Response
Amazon EventBridge - schedule cron jobs, triggers by source event and do action to target service
S3 Event Notifications - Object created, removed, restored, replication etc…
- Typically delivers in seconds sometimes take a minute or longer
- EventBridge could receive events from S3 bucket
AWS Health Dashboard
- Service History - Shows all region all services health
- Your Account - dashboard provides alerts and remediation guidance when AWS is experiencing events that may impact you. Global Service
- Receive Event notifications to integrating events with EventBridge
EC2 Instances Status Checks
- Automated checks to identify hardware and software issues
- System Status Checks - monitors problem with AWS systems (software/hardware issues on the physical hosts, loss of system power…)
- Instance Status Checks - monitors software/network configuration of your instance
- Attached EBS Status Checks - monitor attached EBS (reachable & complete I/O operations)
- can Recover using CloudWatch alarm trigger
AWS CloudTrail - provides governance, compliance and audit for your AWS account
- enabled by default, get an history of events / API calls made within AWS account by Console, SDK, CLI, AWS Services, can put logs into CloudWatch logs or S3
- Management Events - AWS account operations
- Data Events - resource operations performed on resource, by default events are not logged because high volume of operations
- CloudTrail Insights - enable it to detect unusual activity in your account
AWS X-Ray - Visual analysis of our applications
- Tracing requests across your micro services (distributed systems)
- Integrated with EC2, ECS, Lambda, Beanstalk, API GW
- EC2, ECS - install X-Ray agent, agent or services need IAM permissions to X-Ray
AWS Distro for OpenTelemetry - provides a standardized way to collect and export metrics, traces, and logs from your applications and send them to multiple monitoring and observability services.
Domain 6 - Security and Compliance
AWS Config
- Helps with auditing and recording compliance of your AWS resources, then receive alerts or even take action using SSM Automation Documents, checks rule is compliant or noncompliant
- To remediate — Use custom(invoke lambda) or managed SSM Automation Documents
- Config Resource - is targets resources you are monitoring, can see historical changes to that resource
- Conformance Pack - Collection of AWS Config Rules and Remediation actions created in YAML files
- Organizational Rules - can managed across all accounts within an AWS Organization
- Delivery method: Choose bucket from your or another account.
- Stream to SNS topic
- AWS Config Aggregators — Multi Region & Multi Account
- Enabling trusted access for AWS config in the organization
AWS Organizations
- Management account is master, you could have OU’s which packs multiple accounts inside of it
- If you create new accounts from Management Account
OrganizationAccountAccessRole
created and assumed automatically, if you invite existing account need to create manually - Service Control Policies (SCP) - Define allowlist or blocklist IAM actions, need explicitly allow to use services. SCP should be defined each OU levels and accounts https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps_evaluation.html
- SCPs do not have a Principal element. While SCPs are applied to accounts and deny permissions, IAM policies are applied to users/groups and grant permissions.
AWS Control Tower
- Easy way to setup and govern a secure and compliant multi-account AWS environment based on best practices
- Runs top of AWS Organizations
- Automate the setup of your environment
- Automate ongoing policy using guardrails
- Detect policy violations and remediate them
- Monitor compliance with dashboard
- Preventive and Detective guardrails
- Account factory - automates account provisioning and deployments, pre-approved baselines (e.g VPC default config, subnet, region), uses AWS Service Catalog
- Account Factory Customization (AFC)
- auto customize resources in new and existing accounts created through Account Factory
- Custom BluePrint - CloudFormation template you want to customize, Defined in Service Catalog Product, and stored in Hub Account
- Landing Zone - consists of:
- AWS Organization - have OUs
- Account Factory - easily config new accounts
- IAM
- Guardrails - SPCs and AWS Config
- Customization for AWS Control Tower (CfCT)
- GitOps-style customization, customize Landing Zone by CFN and SCPs, very similar to AFC but different
- AWS Config Integration - Control Tower uses it to implement Detective Guardrails, CT auto enables AWS config in Regions, Config history and snapshots delivered to S3 in a centralized Log Archive account
AWS IAM Identity Center (SSO)
- One login to AWS Cloud, SAML2.0 apps, Cloud Apps
- Identity Providers (IdP) - IAM Identity Center, 3rd Party: AD, Okta ect..
- SCIM - IAM Identity Center cannot read external IdP users and groups, you need to create them manually in IAM IDC, for that reason you can use SCIM to synch users and groups
- Attribute-Based Access Control (ABAC)
- set tags to user group and tag to resources if their tag matches they have control on it
-
MFA - Always on or When Context changes (device, browser, location etc…)
AWS WAF - Web Application Firewall
- Protects from common web exploits (Layer 7), Not DDoS protection (for that use Shield)
- Deploy on ALB, API GW, CloudFront, AppSync
- Define Web ACL (Web Access Control List):
- Rate based, Geo Match, SQL injection, Cross-Site Scripting protection
- Rules can include IP, HTTP headers, Http body, URI strings
- Managed Rules - over 190 rules, ready to use
- Baseline Rule Groups - general protection
- Use-case Specific
- IP reputation - malicious IP protection
- Bot Control Manager - block requests from bots
- Logging - Can send logs to CloudWatch Logs, Firehose, S3
AWS Firewall Manager
- Manage rules in all accounts of an AWS Org:
- WAF rules, AWS shield, SG, AWS Network Firewall (VPC level), Route 53 resolver DNS firewall — in region level
- There is similar service called: AWS network Firewall — which is VPC level individual account
- Security Policy Types:
- AWS WAF - Apply WebACLs to all ALBs in all accounts, auto remediate any noncompliant resources — auto apply WebALCs to existing resources
- Shield Advanced - Enforce Shield Advanced protections to all accounts
- Common Security Groups - Apply SG to all EC2 in all accounts
- Auditing of SG policy - Check manage
- Usage Audit SG policy - Unused SG cleanup
- Network Firewall - Centrally manage firewall, Distributed or Centralized firewall VPC endpoint
- Route 53 Resolver DNS Firewall
AWS GuardDuty - ML service to scan logs and detect threats
- Findings and Automated response to SNS, Lambda, SQL through EventBridge
- Eg. GuardDuty find suspicious behavior and trigger say to EventBridge. EventBridge notifies to Admin by SNS and using Lambda modify WAF-WebACL, NACL rules
- Can define Trusted and Threat IP lists
- When you enable GuardDuty by CFN, it throws error if already enabled, so you need to setup additional Lambda to check this condition on your CFN
- GuardDuty can works with multiple accounts
Amazon Detective - ML and graph backed
- After identifying potential security issues by GuardDuty, Macie, and Security Hub you can find the root cause of security issues using this service
Amazon Inspector - Detects vulnerabilities (package, OS etc.)/ Specifically designed for EC2 instances to scan software vulnerabilities and configuration deviations. It continuously scans for common security benchmarks in standard
- Package vulnerabilities - EC2, ECR&Lambda
- Network reachability - EC2
- For EC2 - SSM agent must installed, Role setup, outbound 443 port must be enabled to SSM endpoint
Trusted Advisor
AWS Secrets Manager
AWS Tag Editor - Allows to manage multiple resource tags at once
Amazon Quicksight - basically tableau, business analytics, dashboard works with RDS, Athena, Redshift, S3
Other
- Tag Policy: Feature that standardize tagging rule in your organizations. Only available in an organization that all features enabled
- You have to create NAT gateway to each AZ if you want High Availability
- Disaster recovery primarily involves multi-region strategies, while High Availability is typically achieved through multi-AZ deployments
💡Always remember the problem you're solving when reading through the options.
Top comments (0)