Juan Ibarra for Kubeshop

Posted on Sep 30 • Originally published at testkube.io

Using Testkube for Test Orchestration in Tekton Pipelines

#tekton #testkube #kubernetes #devops

Introduction

Integrating efficient and robust testing frameworks within CI/CD pipelines is essential in today's fast-paced DevOps environment. Tekton, a Kubernetes-native open-source framework, excels in defining and running continuous integration and continuous delivery (CI/CD) pipelines. When paired with Testkube, a cloud-native test orchestration platform, Tekton offers a seamless and automated foundation for efficient software delivery.
By incorporating Testkube into Tekton Pipelines, development teams can achieve more reliable and efficient testing workflows across their entire SDLC. Testkube supports all the popular testing frameworks and you can even Bring Your Own Tool(BYOT). Developers can trigger and manage any test such as functional, acceptance, load, etc., streamline their testing processes from the pipeline themselves or from a variety of other trigger sources, and ultimately deliver high-quality software faster.

Automated testing throughout the development cycle ensures reliability and agility, preventing testing from becoming a bottleneck. This blog post explores how to effectively use Testkube for test orchestration within Tekton Pipelines, enhancing your CI/CD practices.

What is Tekton?

Tekton, a graduated Continuous Delivery Foundation (CDF) project, offers a flexible, cloud-native solution for automating application’s build, test, and deployment. It is highly extensible and can be installed on any Kubernetes cluster.
Let us understand some of the terms related to Tekton:

Task: is a fundamental unit in Tekton. In a Task, we define the steps that will be executed sequentially within a Kubernetes pod using the specified image. Each step runs in its own container within the pod and creates an output that can be used as input to the next step.
Pipeline: is a series of ordered Task. A Task in a Pipeline can use the previous Task output as input.
TaskRun: While Task has the definition of what to do, TaskRun instantiates the defined Task for the execution.
PipelineRun: Similar to TaskRun, a PipelineRun instantiates the defined Pipeline for execution.

A TaskRun is useful when we have to run only one task. To run multiple tasks in some orderly manner or to handle complex workloads such as static analysis, testing, or building complex tasks, we need to create a Pipeline.

Tekton works using its fundamental unit, Task. A Task or a Pipeline created on a cluster is just a definition. It gets executed by Tekton on creating TaskRun(in case of a single Task) or PipelineRun. Once the PipelineRun is created, Tekton schedules the Tasks and Steps to run in the specified order.

Each Task runs in its own pod, and Tekton manages the execution, monitoring, and logging of each step. Developers can monitor the status of the PipelineRun and TaskRun using the Tekton CLI.

Here are some more things that developers can achieve with Tekton:

Define custom workflows as code tailored to their specific needs, enabling precise control over their CI/CD processes.
Isolate each task execution and minimize the risk of conflicts between tasks.
Easily scale pipelines and handle increased workloads efficiently.
Utilize Kubernetes features such as namespaces, RBAC (Role-Based Access Control), and custom resource definitions (CRDs) seamlessly within their CI/CD pipelines.
Integrate with a wide range of tools and services commonly used in DevOps workflows.

In the coming section, let us explore Testkube and the benefits of integration.

What is Testkube?

Testkube is a cloud-native test orchestration platform for Kubernetes environments. It simplifies managing and running any kind of test by providing a unified platform that integrates with various testing tools and frameworks.
This makes it easier for developers to ensure their applications work correctly without needing to handle the complexity of different testing setups. Here are some of the benefits of using Testkube to run tests in a CI/CD pipeline:

Seamless Integration with any testing framework: Whether we are using unit tests, integration tests, or end-to-end tests, Testkube can integrate with our preferred tools, providing a consistent and streamlined testing process.
Automate Parallel Testing: With Testkube, we can run multiple tests simultaneously (in parallel), which speeds up the testing process. This is particularly useful when there are many tests, as it reduces the overall time needed to verify code.
Centralized Test Execution Management: Testkube offers a centralized platform to manage all the test executions. This means we can start, stop, monitor and troubleshoot tests from one place, making tracking our testing activities and results easier.
Scalability: With the increased workload, Testkube can easily scale up to handle executions efficiently. Each test runs in isolation, ensuring that our testing processes remain robust and reliable.
Trigger tests from outside CI/CD: Testkube allows us to trigger tests from outside the CI/CD pipeline on the cluster through various mechanisms, including Kubernetes events, scheduled executions, the Testkube CLI, etc.. This flexibility ensures we can run tests whenever needed, not just during automated build and deployment processes.

In the next section, we will integrate Testkube with Tekton using the Testkube CLI.

Integrating Testkube with Tekton

We will create a Tekton Task that uses the Testkube CLI Docker image, performs authentication to connect to your Testkube account, and executes the Testkube command we want to run.

Pre-requisites

To follow along, you will need to set up the following:

A Testkube Pro account (free plan is fine)
A Kubernetes cluster - we’re using a local Minikube cluster.
The Testkube Agent is configured on the cluster.
Tekton Pipeline is installed on the cluster.
A Testkube API token with Admin Access rights.

Once the prerequisites are in place, you should have a target Kubernetes cluster ready with a Testkube agent and Tekton configured.

$ kubectl get pods -A
NAMESPACE                    NAME                                                   READY   STATUS    RESTARTS        AGE
kube-system                  coredns-5dd5756b68-5hlw4                               1/1     Running   0               3d23h
…
tekton-pipelines-resolvers   tekton-pipelines-remote-resolvers-9c4dfcb5c-9dnj8      1/1     Running   0               48s
tekton-pipelines             tekton-events-controller-7b68785c4b-q4df7              1/1     Running   0               48s
tekton-pipelines             tekton-pipelines-controller-7d9f595ff9-wtfp8           1/1     Running   0               48s
tekton-pipelines             tekton-pipelines-webhook-544849794c-ldxxz              1/1     Running   0               48s
testkube                     testkube-api-server-5987d56766-wl85n                   1/1     Running   3 (3d23h ago)   3d23h
testkube                     testkube-nats-0                                        2/2     Running   0               3d23h
testkube                     testkube-operator-controller-manager-f4bb85c97-mxg72   2/2     Running   0               3d23h

Create Tekton Task

To integrate Testkube with Tekton, we need to configure the CLI to connect to our Testkube environment. Testkube provides set context CLI command that will set the CLI Context to the expected environment. Once the context is set, we can run any testkube command from the Tekton Pipeline. Here is Tekton Task with Testkube integrated:

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: testkube
spec:
  description: >-
    Run testkube cli commands with tekton
  params:
  - name: tkc-env
    description: testkube cloud environment name
    type: string
  - name: tkc-org
    description: testkube cloud org name
    type: string
  - name: api-key
    description: api key for testkube account
    type: string
  - name: tkc-command
    description: Command to run with testkube
    type: string
  steps:
  - name: run-testkube-cli
    image: "kubeshop/testkube-cli:latest"
    script: |
      #!/bin/sh
      testkube set context -c cloud --env-id $(params.tkc-env) --org-id $(params.tkc-org) -k $(params.api-key)
      testkube $(params.tkc-command)

Let us understand each of the fields we have set in the above Task:

kind: Task: Declares that this YAML defines a Tekton Task
params: Parameter that the Task expects. In the script, we have added the testkube set context command that expects the env-id, org-id, and API key to connect.some text

tkc-env: This is the environment ID. You can find the value in Testkube Dashboard -> Settings.
tkc-org: This is the org ID. You can find the value in Testkube Dashboard -> Settings.
api-key: This is the dedicated API key you created during prerequisites.
tkc-command: In the script, we run the testkube command, which allows you to pass any testkube CLI command or flags.

steps: Set the name of the step along with image to be used to create the pod to run the script. Here we have created a bash script that sets the Testkube CLI Context and runs the Testkube command.

This Task is reusable. You can run any Testkube command using it in a Testkube environment with Admin access. Let us see, with the help of a use case, how we can use this Tekton Task with Testkube integrated to run a Test Workflow.

Test Orchestration with Testkube in Tekton

In our previous blog, we automated acceptance testing in the Testkube Dashboard using the Test Workflows. We will execute the same TestWorkflow from the Tekton CI/CD. This automation will help you understand how seamlessly Testkube integrates the execution of any testing tool in the Tekton Pipeline.

Create a Test Workflow in the Testkube Dashboard

Testkube offers a comprehensive Dashboard where you can manage your Test Workflows and their execution. We have already created the Test Workflow and executed it in the dashboard:

The Robot Framework test for Restful Booker has been executed successfully.

Create a Tekton Task on the cluster

Create the Tekton Task which integrates Testkube on the cluster. We have kept Task in a file tekton-testkube.yaml and deployed it on the cluster.

$ kubectl apply -f tekton-testkube.yaml
task.tekton.dev/testkube created

$ kubectl get tasks
NAME       AGE
testkube   5s

The Tekton Task testkube is successfully created on the cluster.

Create a Tekton Pipeline

A Tekton Pipeline contains the task and the required parameters. Here is the Pipeline we created that includes the testkube Task.

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: testkube-pipeline  # Name of the Pipeline
spec:
  params:                  # Parameters that the Pipeline expects
  - name: tkc-env
    description: testkube cloud environment name
    type: string
  - name: tkc-org
    description: testkube cloud org name
    type: string
  - name: api-key
    description: "The API key for Testkube"
    type: string
  - name: tkc-command
    description: Command to run with testkube
    type: string
  tasks:                   # Tasks to be executed as part of this Pipeline
    - name: testkube-cli
      taskRef:             # Reference to a previously defined Task
        name: testkube     # Name of the Task being referenced is 'testkube'
      params:
      - name: tkc-env
        value: $(params.tkc-env)
      - name: tkc-org
        value: $(params.tkc-org)
      - name: api-key
        value: $(params.api-key)
      - name: tkc-command
        value: $(params.tkc-command)

In the Pipeline, the parameters are defined that are passed in the testkube Task. Save this in a file pipeline-tekton-testkube.yaml and deploy it on the cluster.

$ kubectl apply -f pipeline-tekton-testkube.yaml
pipeline.tekton.dev/testkube-pipeline created

$ kubectl get pipeline
NAME                AGE
testkube-pipeline   9s

The Tekton Pipeline for the testkube Task is successfully created. To instantiate this Pipeline, we create a PipelineRun.

Create a Tekton PipelineRun

A Tekton PipelineRun contains the Pipeline details and the values of the parameters required by the Pipeline. Here is the PipelineRun we have created for execution.

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: testkube-pipeline-run
spec:
  pipelineRef:
    name: testkube-pipeline
  params:
  - name: tkc-env
    value: "tkcenv_34xxxxxxxxxxxxx"
  - name: tkc-org
    value: "tkcorg_b8xxxxxxxxxxxxxx"
  - name: api-key
    value: "tkcapi_58xxxxxxxxxxxxxxx"
  - name: tkc-command
    value: "run testworkflow basic-acceptance-test --watch"

Here, we have added the value to the parameters. You can also make use of Kubernetes Secrets to pass encrypted values. Parameter tkc-command contains the testkube CLI command to be executed on the cluster.

Here we have passed the command to run the testworkflow basic-acceptance-test. This is the testworkflow for the acceptance testing with the Robot Framework created in the Testkube Dashboard. Save the configuration in a file pipelinerun-robottest-tekton-testkube.yaml and deploy it on the cluster.

$ kubectl apply -f pipelinerun-robottest-tekton-testkube.yaml
pipelinerun.tekton.dev/testkube-robotframework-pipeline-run created

$ kubectl get pipelinerun
NAME                                   SUCCEEDED   REASON      STARTTIME   COMPLETIONTIME
testkube-robotframework-pipeline-run   True        Succeeded   2m36s       2m24s

With this, the PipelineRun is successfully created and the Task is executed. In the coming section, we will see how to verify and get more details related to the test execution.

Verify

Tekton provides Tekton CLI which can also be utilized to see the execution of the Task. However this will not show us the complete test execution logs and artifacts-related details. Testkube provides a Dashboard that we can use to verify the execution of the testworkflow from the Task.

Tekton

We installed Tekton CLI on our local machine. Here are the logs related to the Pipeline.

$ tkn pipelinerun logs testkube-robotframework-pipeline-run
task testkube-cli has failed: "step-run-testkube-cli" exited with code 2
[testkube-cli : run-testkube-cli] Your config was updated with new values 🥇
[testkube-cli : run-testkube-cli]
[testkube-cli : run-testkube-cli] Your current context is set to cloud
[testkube-cli : run-testkube-cli]
[testkube-cli : run-testkube-cli]   API Key        : tk*********************************37
[testkube-cli : run-testkube-cli]   API URI        : https://api.testkube.io
[testkube-cli : run-testkube-cli]   Namespace      :
[testkube-cli : run-testkube-cli]   Organization   : SONALI SRIVASTAVA-personal-org (tkcorg_xx)
[testkube-cli : run-testkube-cli]   Environment    : SONALI SRIVASTAVA-personal-env (tkcenv_xx)
[testkube-cli : run-testkube-cli]
[testkube-cli : run-testkube-cli]
[testkube-cli : run-testkube-cli] Context: cloud (1.17.63)   Namespace:    Org: SONALI SRIVASTAVA-personal-org   Env: SONALI SRIVASTAVA-personal-env
[testkube-cli : run-testkube-cli] ------------------------------------------------------------------------------------------------------------------
[testkube-cli : run-testkube-cli] Test Workflow Execution:
[testkube-cli : run-testkube-cli] Name:                 basic-acceptance-test
[testkube-cli : run-testkube-cli] Execution ID:         666c4fa7c3e60220a410f063
[testkube-cli : run-testkube-cli] Execution name:       basic-acceptance-test-8
[testkube-cli : run-testkube-cli] Execution namespace:  testkube
[testkube-cli : run-testkube-cli] Execution number:     8
[testkube-cli : run-testkube-cli] Requested at:         2024-06-14 14:11:51.349926545 +0000 UTC
[testkube-cli : run-testkube-cli] Status:               queued

In the above output, we can see Task name testkube-cli and the step name run-testkube-cli. The Task is running and the execution of the testworkflow is queued. Let us head to the Testkube Dashboard for more details.

Follow a Test Execution in the Testkube Dashboard

In the Testkube Dashboard, we can see the test running as shown below.

Click on Open Details & Logs.

The above screenshot shows that all the tests have successfully passed, and Testkube has uploaded the artifacts.

With this approach, we orchestrated the Robot Framework test from the Tekton Pipeline using Testkube. Not only did Testkube make the integration seamless, but it also offered us a scalable and efficient solution for automating testing processes within your Kubernetes environment.

Conclusion

Integrating Testkube with Tekton Pipelines provides a robust, secure, and scalable solution for test orchestration and execution. In this blog, we created a reusable Tekton Task that integrates with Testkube. Using this Task, we deployed a CI/CD pipeline that executed Robot Framework tests in a Kubernetes native environment. Similarly, you can Bring Your Own Tool(BYOT) and orchestrate tests with Testkube in CI/CD pipelines.
To experience the full potential of testing on Testkube, we invite you to try Testkube today. Witness firsthand how Testkube simplifies and empowers your testing process with its Kubernetes-native test execution capabilities. Join our active Slack community for guidance and support.

DEV Community