Getting Started with CI/CD: A Beginner's Guide to Automating Your First Pipeline (with Jenkins)

Table of Contents

• Introduction
• What is CI/CD?
  • Continuous Integration (CI)
  • Continuous Delivery (CD)
  • Continuous Deployment
• Benefits of CI/CD
  • Faster Time to Market
  • Improved Code Quality
  • Efficient Collaboration
  • Increased Automation and Consistency
• How to Create Your First CI/CD Pipeline
  • Step 1: Set Up Version Control (GitHub)
  • Step 2: Choose a CI/CD Tool
  • Step 3: Jenkins Pipeline Setup
  • Step 3.1: Installing Jenkins
  • Step 3.2: Jenkins Configuration
  • Step 3.3: Creating a Jenkins Pipeline
  • Step 4: Write a Basic Pipeline Configuration (Jenkinsfile)
  • Step 5: Deploy the Application
• Best Practices for CI/CD Pipelines
  • Automate Everything
  • Fail Fast, Fail Often
  • Version Control Best Practices
  • Keep Pipelines Simple
• Conclusion

---

Introduction

Continuous Integration/Continuous Deployment (CI/CD) pipelines are at the heart of modern software development. They allow teams to automate key steps in the development lifecycle, from integrating new code changes, running automated tests, and deploying applications to various environments. As DevOps practices become more widespread, mastering CI/CD pipelines is essential for developers, DevOps engineers, and IT teams.

In this guide, we will focus on helping you get started with Jenkins, one of the most popular CI/CD tools. Jenkins' flexibility and rich ecosystem of plugins make it a powerful solution for automating a variety of workflows, from simple projects to complex microservices architectures. This comprehensive guide will take you through the entire process of setting up a CI/CD pipeline with Jenkins — from installation to creating your first pipeline, and beyond.

---

What is CI/CD?

Before we dive into Jenkins, let's take a moment to understand the key components of CI/CD.

Continuous Integration (CI)

Continuous Integration (CI) is a software development practice where developers frequently commit code changes to a shared repository, ideally multiple times a day. Each time new code is committed, an automated process is triggered that builds the code, runs tests, and identifies bugs early. The core purpose of CI is to improve collaboration, detect integration problems quickly, and reduce integration issues that can arise when multiple developers work on the same project.

• Frequent Commits: Developers commit code changes more frequently to ensure the repository always has the latest working code.
• Automated Build and Test: After every commit, automated builds and tests run, giving developers quick feedback on their changes.
• Fail Early: CI allows teams to detect issues early, preventing small bugs from snowballing into more significant problems later in the process.

Continuous Delivery (CD)

Continuous Delivery (CD) builds on top of CI, automating the process of delivering code changes to a staging environment for further testing and validation. The goal of CD is to ensure that the codebase is always in a deployable state and that new features, bug fixes, and updates can be delivered to users as soon as they're ready.

• Automated Testing and Packaging: CD ensures that code changes are always tested and packaged for release.
• Manual Approval for Production: With Continuous Delivery, deployment to production is manual but prepared automatically. Human intervention is required for final approval.
• Release Anytime: By having a deployable product at any given time, teams can release new features faster and more frequently.

Continuous Deployment

Continuous Deployment takes Continuous Delivery a step further by automating the deployment of every change that passes through all stages of the pipeline directly to production. Continuous Deployment requires a high degree of confidence in the automated testing process because there’s no manual approval before deploying to production.

---

Benefits of CI/CD

Faster Time to Market

CI/CD pipelines allow teams to deliver software updates to users much faster. By automating the integration, testing, and deployment processes, teams can release features, improvements, and bug fixes in a continuous flow rather than waiting for a major release cycle.

• Shorter Feedback Loop: Developers get faster feedback on their code.
• Faster Deployments: Automated pipelines significantly reduce the time it takes to go from code commit to production.

Improved Code Quality

With CI/CD, code is continuously tested throughout the development process. This leads to higher quality code being integrated into the main repository, as issues are caught early through automated testing.

• Automated Tests: Unit, integration, and end-to-end tests can run automatically every time code is committed, ensuring that the code meets quality standards.
• Frequent Testing: Regular testing reduces the number of bugs that slip into production.

Efficient Collaboration

CI/CD pipelines promote better collaboration between team members. Since code is integrated frequently and tested in a shared environment, teams can collaborate more effectively, and any integration issues are identified and resolved early in the process.

• Shared Codebase: Developers always work with the latest code, which reduces the chances of merge conflicts or broken builds.
• Fewer Surprises: The team is more likely to identify breaking changes earlier in the development cycle.

Increased Automation and Consistency

By automating repetitive and manual tasks such as building, testing, and deployment, CI/CD pipelines reduce human error, improve consistency, and free up developer time to focus on more complex tasks.

• Consistency: Automated pipelines ensure that every build follows the same steps and produces reliable results.
• Reduced Errors: Automation removes the possibility of human error in deployment processes.

---

How to Create Your First CI/CD Pipeline

Now that we understand the benefits of CI/CD, let’s dive into the practical steps of creating your first CI/CD pipeline using Jenkins.

Step 1: Set Up Version Control (GitHub)

Before setting up Jenkins, we need a version-controlled repository for your project code. We’ll use GitHub as the version control platform in this example.

Instructions:

1. Create a GitHub Repository:

   • Go to GitHub and create a new repository.
   • Name the repository (e.g., my-first-pipeline), provide a description, and initialize the repository with a README.md file.
   • Choose whether the repository will be public or private.

2. Clone the Repository to Your Local Machine:

   • Open a terminal and clone the repository using the following command:

    git clone https://github.com/your-username/my-first-pipeline.git
   

• Navigate to the project folder:
  

   cd my-first-pipeline
  

1. Add Project Files:

   • Add your application files (for example, a simple web app) to the repository.
   • For a Node.js application, this might include files such as app.js, package.json, and a tests directory.

2. Commit and Push the Code:

   • Use the following commands to stage, commit, and push your changes:

    git add .
    git commit -m "Initial commit"
    git push origin main
   

Your repository now contains the codebase for your application, and it’s ready for Jenkins to automate the build and deployment processes.

---

Step 2: Choose a CI/CD Tool

There are various CI/CD tools available such as GitHub Actions, CircleCI, GitLab CI, and Jenkins. In this guide, we will focus on Jenkins due to its flexibility, robust community support, and extensive plugin ecosystem.

Step 3: Jenkins Pipeline Setup

Step 3.1: Installing Jenkins

The first step in setting up Jenkins is installing it on your local machine or server. Jenkins can run on various platforms like Windows, macOS, and Linux, and it can also be installed in a Docker container. Here, we will cover the installation process for a Linux machine (Ubuntu/Debian) and Docker.

Installing Jenkins on Linux (Ubuntu/Debian)

1. Install Java: Jenkins requires Java to run. Install OpenJDK:

   sudo apt update
   sudo apt install openjdk-11-jdk

1. Add Jenkins Repository:

   • Add the Jenkins repository key and add it to your sources:

    wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -
    sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/
   

/etc/apt/sources.list.d/jenkins.list'
```

1. Install Jenkins:

   sudo apt update
   sudo apt install jenkins

1. Start and Enable Jenkins:

   sudo systemctl start jenkins
   sudo systemctl enable jenkins

1. Access Jenkins:

   • Open a browser and go to http://localhost:8080 (replace localhost with the server IP if you're using a remote server).
   • You will be prompted to unlock Jenkins. The initial password can be found in the following file:

    cat /var/lib/jenkins/secrets/initialAdminPassword
   

2. Install Suggested Plugins: Jenkins will guide you through the installation of essential plugins like Git, Pipeline, and Docker. Click Install Suggested Plugins to set up the most commonly used plugins.

Installing Jenkins using Docker

Alternatively, you can install Jenkins using Docker with the following commands:

1. Install Docker:

   sudo apt update
   sudo apt install docker.io

1. Run Jenkins in Docker:

   docker run -p 8080:8080 -p 50000:50000 jenkins/jenkins:lts

You can now access Jenkins in your browser by navigating to http://localhost:8080.

Step 3.2: Jenkins Configuration

Once Jenkins is installed, some basic configuration is necessary to integrate it with GitHub and to prepare it for pipeline creation.

1. Install Git:

   • Make sure Git is installed on your Jenkins server:

    sudo apt install git
   

2. Configure Git in Jenkins:

   • Go to Manage Jenkins > Global Tool Configuration and configure the Git executable path. This allows Jenkins to use Git for cloning repositories.

3. Set Up Credentials for GitHub:

   • Jenkins will need access to your GitHub repository. To set up credentials:
     • Go to Manage Jenkins > Manage Credentials > Global.
     • Add credentials such as a GitHub personal access token or SSH key to allow Jenkins to authenticate with GitHub.
     • If you use HTTPS for GitHub, create a Username with password type credential, where the username is your GitHub username, and the password is your personal access token (PAT). If using SSH, add your SSH private key as a SSH Username with private key credential.

Step 3.3: Creating a Jenkins Pipeline

Jenkins offers a powerful pipeline feature that allows you to define the stages of your build, test, and deployment processes using code. This is done by creating a Jenkinsfile in your repository, which Jenkins will automatically detect and use to execute your pipeline.

1. Create a New Pipeline Job:

   • Go to Jenkins Dashboard > New Item.
   • Enter a name for your pipeline (e.g., my-first-pipeline) and select Pipeline from the job types.
   • Click OK to create the job.

2. Configure Pipeline Script from SCM:

   • Scroll down to the Pipeline section in the job configuration page.
   • Under Definition, select Pipeline script from SCM.
   • Choose Git as the source control system.
   • Provide the URL of your GitHub repository and select the appropriate branch (e.g., main).
   • Specify the path to the Jenkinsfile in your repository (this is usually just Jenkinsfile if it’s located in the root of the repository).

At this point, Jenkins is set up to automatically detect the Jenkinsfile in your repository and execute the pipeline defined within it.

---

Step 4: Write a Basic Pipeline Configuration (Jenkinsfile)

The pipeline's behavior is defined in a Jenkinsfile, which is a text file that contains the pipeline’s stages and steps. Let’s write a simple Jenkinsfile for a basic Node.js project.

Example Jenkinsfile for Node.js:

pipeline {
    agent any

    stages {
        stage('Checkout') {
            steps {
                git url: 'https://github.com/your-username/my-first-pipeline.git', branch: 'main'
            }
        }

        stage('Build') {
            steps {
                // Install dependencies
                sh 'npm install'
            }
        }

        stage('Test') {
            steps {
                // Run tests
                sh 'npm test'
            }
        }

        stage('Deploy') {
            steps {
                // Deployment logic goes here (e.g., pushing to a cloud provider)
                echo 'Deploying application...'
            }
        }
    }

    post {
        always {
            echo 'Pipeline completed.'
        }
        failure {
            echo 'Pipeline failed!'
        }
    }
}

Breakdown of the Jenkinsfile:

• agent any: This means the pipeline can run on any available Jenkins agent (a node or worker).
• Stages: Pipelines consist of multiple stages that represent different parts of the CI/CD process:
  • Checkout: This stage checks out the latest code from your GitHub repository.
  • Build: This stage runs the command npm install to install dependencies for a Node.js application.
  • Test: This stage runs npm test to execute the unit tests.
  • Deploy: This is where deployment steps are added, such as pushing the built code to a cloud platform like AWS, Azure, or Heroku.
• Post-actions: The post section allows you to specify actions to take after the pipeline completes, regardless of success or failure. In this case, we print a message depending on whether the pipeline succeeded or failed.

To implement this pipeline, simply create a new file in your repository named Jenkinsfile and add the above pipeline script to it. Once committed and pushed to the repository, Jenkins will detect the Jenkinsfile and execute the pipeline.

---

Step 5: Deploy the Application

Once your pipeline runs successfully and the tests pass, the next logical step is to deploy the application. Jenkins can be integrated with various cloud platforms, such as Heroku, AWS, or Azure, for automated deployments.

Example: Deploying to Heroku

Heroku is a popular cloud platform that can be integrated easily into Jenkins pipelines. To deploy your application to Heroku:

1. Install the Heroku CLI on the Jenkins agent:

   sudo snap install --classic heroku

1. Add Heroku Deployment Step to the Jenkinsfile: Modify the Deploy stage in your Jenkinsfile as follows:

stage('Deploy') {
    steps {
        withCredentials([string(credentialsId: 'heroku-api-key', variable: 'HEROKU_API_KEY')]) {
            sh 'heroku git:remote -a your-heroku-app'
            sh 'git push heroku main'
        }
    }
}

• This example assumes you’ve stored your Heroku API key in Jenkins credentials (under the ID heroku-api-key).
• This command pushes the latest code to Heroku, which will automatically trigger a deployment to the Heroku cloud.

By adding deployment steps to your pipeline, Jenkins can fully automate the process from code commit to production deployment.

---

Best Practices for CI/CD Pipelines

While Jenkins is flexible and powerful, it’s essential to follow best practices to ensure your pipelines are efficient, maintainable, and reliable.

Automate Everything

Aim to automate every step of your development workflow, from building the application, running tests, deploying to staging, and deploying to production. The more you automate, the fewer manual interventions are required, and the faster and more reliable your pipeline becomes.

Fail Fast, Fail Often

Set up your pipeline to detect failures early in the process. For example, run unit tests as early as possible so that issues can be identified before they affect other stages. If the pipeline is going to fail, it should fail early so developers can fix problems quickly.

Version Control Best Practices

Using version control best practices is critical for CI/CD success. Create a branch for each feature, bug fix, or improvement, and merge it into the main branch only after it passes all tests and checks. This ensures that the main branch is always deployable.

Keep Pipelines Simple

Start with a simple pipeline and add complexity as your project evolves. Pipelines with too many stages or complex dependencies can become difficult to maintain. Break large pipelines into smaller jobs or microservices to keep them manageable.

---

Conclusion

This guide provided a comprehensive introduction to setting up your first CI/CD pipeline using Jenkins. From installing Jenkins, setting up version control, writing a Jenkinsfile, and deploying your application, we’ve covered the essential steps to get you started with automation. Jenkins’ flexibility, paired with its extensive plugin ecosystem, makes it a go-to solution for building, testing, and deploying applications.

As you become more comfortable with Jenkins, you can explore more advanced features like parallel builds, distributed builds with Jenkins agents, and integrating other DevOps tools like Docker, Kubernetes, and monitoring systems.

By following the steps in this guide, you’re well on your way to automating your software delivery process and adopting DevOps best practices that lead to faster, higher-quality software releases. Happy automating!

---

👤 Author

Join Our Telegram Community || Follow me on GitHub for more DevOps content!

Comments

[Innocentia Azal]

I take my time to read every single post from you, because it gives me more understanding of DevOps as a beginner, but as said earlier, it would be better if you could make it more practical,with codes and Screenshot. It could even be a YouTube video. If you are recording too, please the time should not be stretched. Say minimum 1hour, maximum 2 hours. My humble suggestion and request

[H A R S H H A A]

Thanks @innocentia_azal , please read the article now made a few changes

[Mauricio Montoya Medrano]

Muchas gracias por la información, estoy iniciando en esto y quiero saber mas, por lo que si alguien tiene mas información al respecto con ejemplos le agradecería mucho que comparta ese conocimiento.

-
Thank you very much for the information, I am starting this and want to know more, so if anyone has more information about it with examples would be very grateful to share that knowledge.

[H A R S H H A A]

Thanks 👍@mcubico

[Dot Net Tips]

It was great. another tool for CI/CD is Azure DevOps which is very famous and handy to use

[H A R S H H A A]

Yess 👍@dot_nettips_a4e90828daa4

[H A R S H H A A]

@innocentia_azal, @red_ Here are the key changes and improvements made from the old article to the new, more detailed version:

1. Expanded Explanation of CI/CD:

   • Provided a more in-depth breakdown of Continuous Integration (CI), Continuous Delivery (CD), and Continuous Deployment. The new article explains each concept in greater detail, highlighting their purpose and benefits.

2. Step-by-Step Guide for Creating a CI/CD Pipeline:

   • Added a detailed step-by-step guide on how to set up a complete CI/CD pipeline using Jenkins, starting from installing Jenkins to writing and configuring a Jenkinsfile.
   • Step 1: Setting up Version Control with GitHub now includes instructions on how to create and push code to a GitHub repository.
   • Step 2: Choosing a CI/CD Tool focuses on Jenkins, explaining why it’s a good option.
   • Step 3: Jenkins Pipeline Setup has been significantly expanded with sub-steps on installing Jenkins, configuring Jenkins for Git, and setting up credentials.
   • Step 4: Writing a Basic Jenkins Pipeline includes a detailed Jenkinsfile example, explaining each section (checkout, build, test, deploy).
   • Step 5: Deploy the Application now features a concrete example of deploying an app to Heroku, providing actual deployment steps.

3. Added Jenkins Pipeline Script:

   • A complete Jenkinsfile example has been added, showing how to automate the build, test, and deployment stages for a Node.js application.

4. Heroku Deployment Integration:

   • New instructions on how to integrate Heroku deployment into your Jenkins pipeline with specific steps for setting up the Heroku CLI and updating the Jenkinsfile with Heroku deployment commands.

5. Best Practices for CI/CD:

   • A new section discussing best practices for building efficient and maintainable CI/CD pipelines, such as automating everything, failing fast, and keeping pipelines simple.

6. Clearer Pipeline Stages Breakdown:

   • Each pipeline stage (checkout, build, test, deploy) is now broken down with explanations of their purpose and Jenkins syntax.

7. Conclusion Section Expanded:

   • The conclusion now emphasizes advanced features of Jenkins and encourages readers to explore distributed builds, Docker, Kubernetes, and other DevOps integrations, providing a path for further learning.

These changes make the article much more detailed and actionable for beginners by offering practical, hands-on steps while explaining the theory behind CI/CD.

[Red Foreman]

Where is the "Beginner guide"? The steps do not demonstrate anything actionable. This is just abstract ideas and not a guide or how-to.

[H A R S H H A A]

Thank you for your honest feedback! @red_ 🙏 I appreciate you taking the time to read the article and share your thoughts.

You're absolutely right that actionable steps are key in a beginner's guide, and I realize I could have included more specific, step-by-step instructions to make it clearer and more practical. I’ll update the article soon with detailed examples, commands, and screenshots to help guide beginners through setting up their first CI/CD pipeline.

In the meantime, if you have any particular areas you’d like me to focus on or if you need help getting started, feel free to let me know! I’d love to make the guide as helpful as possible.

Thanks again for your input—feedback like yours helps me improve future content!

[Red Foreman]

Just create an article that does what it says what it will do : "we will take you through a step-by-step approach to understanding CI/CD concepts and setting up your first pipeline from scratch."

I cannot setup a pipeline from scratch with your article.

[H A R S H H A A]

Thanks @red_ , please read the article now made a few changes

[Red Foreman]

I am extremely excited for this! Thank you!

[KC]

A comprehensive and introductory guide to the concept of CI/CD process. Hopefully there would be a how to section to setup a CI/CD pipeline with one of your recommended tools as an example.

[H A R S H H A A]

Thanks @kc900201 , please read the article now made a few changes

[Hoàng Tiến]

good

[H A R S H H A A]

Thanks 😊@hoangtien2k3

[Innocentia Azal]

I am familiar with AwS, not heroku. Any guidelines to follow

[H A R S H H A A]

Check out thier official documentation @innocentia_azal

[Innocentia Azal]

I just opened the updated article, I have not started practicing but I am moved with the update. Great explanations and details. I will submit my proof of practice after I am done. Thank you Senior

[H A R S H H A A]

Thanks😀 @innocentia_azal

[kince marando]

Wooo! What amazing enlightenment
Kindly thanks

[H A R S H H A A]

Thanks @kin7777