DEV Community

Cover image for Deploying SonarQube on AWS EC2 with Cloud Development Kit(CDK): Extending the Database to Amazon RDS PostgreSQL
Kevin Lactio Kemta
Kevin Lactio Kemta

Posted on

Deploying SonarQube on AWS EC2 with Cloud Development Kit(CDK): Extending the Database to Amazon RDS PostgreSQL

Day 016 - 100DaysAWSIaCDevopsChallenge

In my previous post, Step-by-Step Guide to Setting Up and Deploying SonarQube on an AWS EC2 Instance Using AWS Cloud Development Kit (CDK)[↗], I demonstrated how to deploy a SonarQube Server on an EC2 instance. As the database were self-hosted in EC2 Instance, today, I will show you how to extend the database by integrating it with an external service like Amazon RDS or Aurora, rather than keeping it self-hosted within the server.

Why did I need to extend the SonarQube Database to a RDS PostgreSQL Instance?

Extending SonarQube's database to an external service like Amazon RDS (Relational Database Service) or Aurora (Amazon Severless Database managed) instead of keeping it inside the self-hosted SonarQube sever offers several advantages:

  1. Scalability: Managed database like RDS and Aurora can handle inscreased workloads, provide more robust storage and offer auto-scaling feature by ensuring best permformance as your SonarQube instance grows.

  2. Reliability: Provide high availability through automatic backups, replication and failover mechanisms. They reduce the risk of data loss and downtime in case of hardware failure.

  3. Security: Offer built-in security features, including encryption at rest and in transit, network isolation through VPC and private subnet making your data more secure than self-managed (self-hosted) databases.

  4. Maintainance: AWS handles regular maintenance tasks, such as backups, patching, and updates, reducing the burden on your infrastructure team.

Update the previous infrastructure to setting up a RDS PostgreSQL Instance for SonarQube

Image description

The infrastructure consists of the following components:

  • VPC and two (02) Subnets(private and public subnet) to manage network traffic within the server and database instance.
  • Internet Gatewayto allow internet communication between our secured cloud environment and external networks.
  • Security group to control access to the server and database based on IP addresses and ports connections.
  • EC2 Instance to host the SonarQube server (including the Web Server, ElasticSearch, and Postgres Database).
  • Key Pair the private key that allows external hosts to connect securely to the instance.
  • Secret Manager to securely store the database user password.
  • IAM Role to allow EC2 Instance to connect to Secret Manager and retrieve database password
  • Elastic IP (EIP) - The static IP to attach to the EC2 Instance. It allows to maintain a fixed IP address aven if the instance is stopped of restarted.
  • Nat Gateway to allow instance (here database instance) in a private subnet to connect to the internet or other AWS Service without exposing then to internet traffic.
  • RDS PostgreSQL the database instance to host the sonarqube data.

Create the database Construct

interface DatabaseProps extends CustomProps {
  vpc: IVpc,
  sg?: ISecurityGroup // the security group for SonarQube Server Instance
}
export class Database extends Construct {
  private readonly _dbInstanceArn: string
  private readonly _passwordSecretArn: string
  private readonly _instanceEndpointUrl: string
  constructor(scope: Construct, id: string, props: DatabaseProps) {
    super(scope, id)
    const secret: ISecret = new aws_secretsmanager.Secret(this, 'SecretResource', {
      secretName: 'database-instance-secret',
      generateSecretString: {
        secretStringTemplate: JSON.stringify({ username: props.databaseUsername }),
        generateStringKey: 'password',
        excludeCharacters: '/@"`()[]\'',
        includeSpace: false
      }
    })
    const securityGroup = new SecurityGroup(this, 'DatabaseSecurityGroupeResource', {
      securityGroupName: 'sq-database-sg',
      disableInlineRules: false,
      allowAllOutbound: true,
      vpc: props.vpc,
      description: 'Database instance security group'
    })
    securityGroup.addIngressRule(
      Peer.securityGroupId(props.sg!.securityGroupId),
      Port.tcp(props.databasePort ?? 5432), 'Allow EC2 to connect to the database instance')
    const db = new rds.DatabaseInstance(this, 'DatabaseInstanceResource', {
      engine: rds.DatabaseInstanceEngine.postgres({
        version: rds.PostgresEngineVersion.VER_12_16
      }),
      vpc: props.vpc,
      networkType: rds.NetworkType.IPV4,
      vpcSubnets: props.vpc.selectSubnets({ subnetType: SubnetType.PRIVATE_WITH_EGRESS }),
      instanceType: InstanceType.of(InstanceClass.BURSTABLE3, InstanceSize.MICRO),
      publiclyAccessible: false,
      storageType: StorageType.GP3,
      credentials: rds.Credentials.fromSecret(secret, props.databaseUsername),
      allocatedStorage: 20,
      databaseName: props.databaseName,
      instanceIdentifier: 'sonarqube-db-1',
      port: props.databasePort ?? 5432,
      securityGroups: [
        securityGroup
      ]
    })
    this._dbInstanceArn = db.instanceArn
    this._passwordSecretArn = secret.secretArn
    this._instanceEndpointUrl = db.instanceEndpoint.socketAddress
  }
  get dbInstanceArn(): string {
    return this._dbInstanceArn
  }
  get passwordSecretArn() {
    return this._passwordSecretArn
  }
  get instanceEndpointUrl(): string {
    return this._instanceEndpointUrl
  }
}
Enter fullscreen mode Exit fullscreen mode

The important thing to know here is the snippet code:

securityGroup.addIngressRule(
    Peer.securityGroupId(props.sg!.securityGroupId),
    Port.tcp(props.databasePort ?? 5432), 'Allow EC2 to connect to the database instance')
Enter fullscreen mode Exit fullscreen mode

This code allows communication between the database and the EC2 instance server through the specified port.

Update the Stack

This is the update for the stack created in the previous article [↗].

export class SonarServerInfrastructureStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: CustomProps) {
    super(scope, id, props)
    ....
    const vpc = new ec2.Vpc(this, 'PublicVpcResource', {
      ...
      maxAzs: 2,
      subnetConfiguration: [{
        subnetType: ec2.SubnetType.PUBLIC,
        name: 'sq-public-subnet-webserver',
        mapPublicIpOnLaunch: true
      }, {
        subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
        name: 'sq-private-subnet-database'
      }]
    })

    sonarQubeSG.addEgressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(props.databasePort ?? 5432), 'Allow outbound traffic to database instance')

    // set up the database instance
    const db = new Database(this, 'DatabaseResource', {
      ...props,
      vpc: vpc,
      sg: sonarQubeSG
    })

    const sonarQubeServer = new ec2.Instance(this, 'SonarQubeServerInstanceResource', {
      ...
      role: new iam.Role(this, 'Ec2RoleResource', {
        roleName: 'ec2-role-and-permission',
        assumedBy: new iam.ServicePrincipal('ec2.amazonaws.com'),
        path: '/',
        inlinePolicies: {
          secretsManager: new iam.PolicyDocument({
            assignSids: true,
            statements: [
              new iam.PolicyStatement({
                effect: iam.Effect.ALLOW,
                actions: [
                  'secretsManager:GetSecretValue',
                  'secretsManager:DescribeSecret'
                ],
                resources: [db.passwordSecretArn],
                conditions: {}
              })
            ]
          })
        }
      })
    })

    const userDataEncoded = ec2.UserData.forLinux()
    userDataEncoded.addCommands(fs.readFileSync('./assets/sonarqube-server.sh', 'utf-8'))
    sonarQubeServer.addUserData(userDataEncoded.render()
      .replace('{{DB_NAME}}', props.databaseName || 'sonarqube')
      .replace('{{DB_USER}}', props.databaseUsername || 'sonar')
      .replace('{{SECRET_ARN}}', db.passwordSecretArn)
      .replace('{{DB_SOCKET_ADDRESS}}', db.instanceEndpointUrl))
  }
}
Enter fullscreen mode Exit fullscreen mode
  • PRIVATE_WITH_EGRESS - In the vpc configuration, we had added a new entry of subnet of type PRIVATE_WITH_EGRESS, meaning that it can access the internet but is not exposed to inbound internet traffic.
  • Egress rule - Allows the SonarQube server to send traffic to the database instance on the specified port (default 5432 for PostgreSQL)
  • New User data scripts As we no longer need the self-hosted database, we have to replace the database connection properties with the new ones:

    ## Update package index
    sudo apt update
    
    ## Install OpenJDK 17
    sudo apt install -y openjdk-17-jdk zip unzip
    ## Set up the JAVA_HOME environment variable
    echo "export JAVA_HOME=$(dirname $(dirname $(readlink -f $(which java))))" | sudo tee /etc/profile.d/jdk.sh
    source /etc/profile.d/jdk.sh
    
    ## download sonarqube server - community version
    wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-9.9.6.92038.zip
    
    ## unzip content
    unzip sonarqube-*.zip
    sudo mv sonarqube-9.9.6.92038 sonarqube
    sudo mv sonarqube /opt/
    
    ## Create sonarqube user and its group
    sudo groupadd sonarqube
    sudo useradd -d /opt/sonarqube -g sonarqube sonarqube
    sudo chown -R sonarqube:sonarqube /opt/sonarqube
    
    ## Insert user runner after `APP_NAME="SonarQube"` line
    sudo sed -i 's/APP_NAME="SonarQube"/APP_NAME="SonarQube"\n\nRUN_AS_USER=sonarqube/' /opt/sonarqube/bin/linux-x86-64/sonar.sh
    
    ## create sonarqube launcher service
    sudo tee /etc/systemd/system/sonarqube.service <<EOF
    [Unit]
    Description=SonarQube service
    After=syslog.target network.target
    [Service]
    Type=forking
    User=sonarqube
    Group=sonarqube
    PermissionsStartOnly=true
    ExecStart=/opt/sonarqube/bin/linux-x86-64/sonar.sh start
    ExecStop=/opt/sonarqube/bin/linux-x86-64/sonar.sh stop
    StandardOutput=journal
    LimitNOFILE=131072
    LimitNPROC=8192
    TimeoutStartSec=5
    Restart=always
    SuccessExitStatus=143
    
    [Install]
    WantedBy=multi-user.target
    EOF
    
    ## Installing AWS-CLI
    sudo snap install aws-cli --classic
    
    sudo tee /opt/sonarqube/conf/sonar.properties <<EOF
    sonar.web.port=9000
    sonar.jdbc.username={{DB_USER}}
    sonar.jdbc.password=$(aws secretsmanager get-secret-value --secret-id {{SECRET_ARN}} | jq --raw-output '.SecretString' | jq -r .password)
    sonar.jdbc.url=jdbc:postgresql://{{DB_SOCKET_ADDRESS}}/{{DB_NAME}}
    EOF
    
    ## Start SonarQube Server
    sudo systemctl enable sonarqube.service
    sudo systemctl start sonarqube.service
    

Deploy the stack

const app = new cdk.App()
new SonarServerInfrastructureStack(app, 'Day016Stack', {
  cidr: '10.0.0.1/16',
  databaseName: 'sonarqubedb',
  databaseUsername: 'sonar',
  databasePort: 5000,
  env: {
    region: process.env.CDK_DEFAULT_REGION,
    account: process.env.CDK_DEFAULT_ACCOUNT
  }
})
app.synth()
Enter fullscreen mode Exit fullscreen mode

Open the terminal anywhere and run the following commande:

git clone https://github.com/nivekalara237/100DaysTerraformAWSDevops.git
cd 100DaysTerraformAWSDevops/day_016
cdk deploy --profile cdk-user Day016Stack
Enter fullscreen mode Exit fullscreen mode

After the deployment, open the browser and type this url http://PUBLIC_IP_or_DNS:9000

SonarQube Web Server Result

__

🥳✨
We have reached the end of the article.
Thank you so much 🙂


Your can find the full source code on GitHub Repo↗

Top comments (0)