Finding and Validating Unused Security Groups in AWS with Python and Boto3

Managing AWS security groups effectively is crucial for maintaining a secure and cost-efficient cloud environment. Security groups are a vital part of the network security in AWS, but over time, unused security groups can accumulate. These unused groups not only clutter your environment but may also pose security risks or increase costs unnecessarily.

In this article, we'll explore how to use Python and Boto3 to identify unused security groups in your AWS environment, validate them, and ensure they are not being referenced by any other resources. We'll also look at how to safely determine if these groups can be deleted.

Prerequisites

To follow along with this tutorial, you'll need the following:

An AWS Account: Make sure you have access to the AWS environment where you want to search for unused security groups.
Boto3 Installed: You can install the Boto3 Python SDK by running:

   pip install boto3

AWS Credentials Configured: Ensure you have AWS credentials configured either using the AWS CLI or directly in your code using IAM roles or environment variables.

Code Breakdown

Let's walk through the code that identifies unused security groups in a given AWS region, validates them, and checks if they are referenced by any other groups.

Step 1: Get All Security Groups and ENIs

import boto3
from botocore.exceptions import ClientError

def get_unused_security_groups(region='us-east-1'):
    """
    Find security groups that are not being used by any resources.
    """
    ec2_client = boto3.client('ec2', region_name=region)

    try:
        # Get all security groups
        security_groups = ec2_client.describe_security_groups()['SecurityGroups']

        # Get all network interfaces
        enis = ec2_client.describe_network_interfaces()['NetworkInterfaces']

        # Create set of security groups in use
        used_sg_ids = set()

        # Check security groups attached to ENIs
        for eni in enis:
            for group in eni['Groups']:
                used_sg_ids.add(group['GroupId'])

        # Find unused security groups
        unused_groups = []
        for sg in security_groups:
            if sg['GroupId'] not in used_sg_ids:
                # Skip default security groups as they cannot be deleted
                if sg['GroupName'] != 'default':
                    unused_groups.append({
                        'GroupId': sg['GroupId'],
                        'GroupName': sg['GroupName'],
                        'Description': sg['Description'],
                        'VpcId': sg.get('VpcId', 'EC2-Classic')
                    })

        # Print results
        if unused_groups:
            print(f"\nFound {len(unused_groups)} unused security groups in {region}:")
            print("-" * 80)
            for group in unused_groups:
                print(f"Security Group ID: {group['GroupId']}")
                print(f"Name: {group['GroupName']}")
                print(f"Description: {group['Description']}")
                print(f"VPC ID: {group['VpcId']}")
                print("-" * 80)
        else:
            print(f"\nNo unused security groups found in {region}")

        return unused_groups

    except ClientError as e:
        print(f"Error retrieving security groups: {str(e)}")
        return None

• Retrieving Security Groups: We first call the describe_security_groups method to get all security groups in the specified region.
• Retrieving Network Interfaces: Next, we retrieve all network interfaces using describe_network_interfaces. Each network interface can have one or more security groups associated with it.
• Identifying Used Security Groups: For each network interface, we add the associated security group IDs to a set called used_sg_ids.
• Finding Unused Groups: We then compare the security group IDs with those in use. If a group is not in use (i.e., its ID is not in the used_sg_ids set), we consider it unused, except for the default security group, which cannot be deleted.

Step 2: Check Security Group References

def check_sg_references(ec2_client, group_id):
    """
    Check if a security group is referenced in other security groups' rules
    """
    try:
        # Check if the security group is referenced in other groups
        response = ec2_client.describe_security_groups(
            Filters=[
                {
                    'Name': 'ip-permission.group-id',
                    'Values': [group_id]
                }
            ]
        )

        referencing_groups = response['SecurityGroups']

        # Check for egress rules
        response = ec2_client.describe_security_groups(
            Filters=[
                {
                    'Name': 'egress.ip-permission.group-id',
                    'Values': [group_id]
                }
            ]
        )

        referencing_groups.extend(response['SecurityGroups'])

        return referencing_groups

    except ClientError as e:
        print(f"Error checking security group references: {str(e)}")
        return None

• Checking for References: This function checks if a specific security group is referenced by any other security groups. It does so by filtering security groups based on their inbound (ip-permission.group-id) and outbound (egress.ip-permission.group-id) rules.
• Return Referencing Groups: If the group is referenced, the function returns a list of referencing security groups. If not, it returns None.

Step 3: Validate Unused Security Groups

def validate_unused_groups(region='us-east-1'):
    """
    Validate and provide detailed information about unused security groups
    """
    ec2_client = boto3.client('ec2', region_name=region)
    unused_groups = get_unused_security_groups(region)

    if not unused_groups:
        return

    print("\nValidating security group references...")
    print("-" * 80)

    for group in unused_groups:
        group_id = group['GroupId']
        referencing_groups = check_sg_references(ec2_client, group_id)

        if referencing_groups:
            print(f"\nSecurity Group {group_id} ({group['GroupName']}) is referenced by:")
            for ref_group in referencing_groups:
                print(f"- {ref_group['GroupId']} ({ref_group['GroupName']})")
        else:
            print(f"\nSecurity Group {group_id} ({group['GroupName']}) is not referenced by any other groups")
            print("This security group can be safely deleted if not needed")

• Validating Unused Security Groups: In this final step, the script first retrieves the unused security groups. Then, for each unused group, it checks if any other security group references it in their rules.
• Output: The script outputs whether the group is referenced or not, and if not, it can be safely deleted.

Running the Script

To run the script, simply execute the validate_unused_groups function. For example, with the region set to us-east-1, the script will:

1. Retrieve all security groups and network interfaces in us-east-1.
2. Identify unused security groups.
3. Validate and report if those unused groups are referenced by other security groups.

Example Output

Found 5 unused security groups in us-east-1:
--------------------------------------------------------------------------------
Security Group ID: sg-12345678
Name: unused-sg-1
Description: Unused security group
VPC ID: vpc-abcde123
--------------------------------------------------------------------------------
Security Group sg-12345678 (unused-sg-1) is not referenced by any other groups
This security group can be safely deleted if not needed
--------------------------------------------------------------------------------
Security Group ID: sg-23456789
Name: unused-sg-2
Description: Another unused group
VPC ID: vpc-abcde123
--------------------------------------------------------------------------------
Security Group sg-23456789 (unused-sg-2) is referenced by:
- sg-34567890 (some-other-sg)

Conclusion

With this script, you can automate the process of finding unused security groups in AWS and ensure that you're not keeping unnecessary resources. This can help reduce clutter, improve security posture, and potentially lower costs by removing unused resources.

You can extend this script to:

• Handle additional filtering based on tags, VPCs, or other criteria.
• Implement more advanced reporting or alerting when unused groups are detected.
• Integrate with AWS Lambda for automated, scheduled checks.

Keep your AWS environment secure and well organized!