DEV Community

Cover image for Your First Workflow in Kraken CI
Michal Nowikowski
Michal Nowikowski

Posted on • Originally published at kraken.ci

Your First Workflow in Kraken CI

Kraken CI is a new Continuous Integration tool. It is a modern, open-source, on-premise CI/CD system that is highly scalable and focused on testing. It is licensed under Apache 2.0 license. Its source code is available on Kraken CI GitHub page.

This tutorial is the second installment of the series of articles about Kraken CI. Part 1, Kraken CI, New Kid on the CI block, presented the installation of Kraken. This time we will cover how to prepare a workflow for a simple Python project. The workflow will:

  1. fetch sources from GitHub,
  2. run tests using Pytest,
  3. build a wheel package,
  4. upload built package to Kraken's artifacts storage.

Here we are using fresh Kraken installation using Docker Compose with built-in Kraken Agent running in Docker container. Installation procedure details can be found in part 1 or in the installation manual from https://kraken.ci project web page.

Project Preparation

Details about a new project setup in Kraken CI can be found in the previous article, part 1 as well.

Here, in short, let's do:

  1. Create a project with a name: Demo
  2. Create a branch with a name: main
  3. Use a pre-created stage and rename it to: Hello World

Stage Hello World

The stage should look as follows:

Branch management page with workflow schema code editor

And the original workflow schema code looks as follows:

def stage(ctx):
    return {
        "parent": "root",
        "triggers": {
            "parent": True,
        },
        "parameters": [],
        "configs": [],
        "jobs": [{
            "name": "hello",
            "steps": [{
                "tool": "shell",
                "cmd": "echo 'hello world'"
            }],
            "environments": [{
                "system": "any",
                "agents_group": "all",
                "config": "default"
            }]
        }]
    }
Enter fullscreen mode Exit fullscreen mode

Now let's try if it works. Click Run CI Flow button.

Logs from the job with echo Hello World

Git Repo Checkout Step

Ok, let's start extending our schema to make it more real life. First, let's add a step for checking out a Git repository:

"steps": [{
    "tool": "git",
    "checkout": "https://github.com/Kraken-CI/sample-project-python.git"
}, {
    "tool": "shell",
    "cmd": "ls -al"
}],
Enter fullscreen mode Exit fullscreen mode

Now run new CI flow. The output should look like this:

Logs from the job with git clone

The logs show that the repository has been cloned and ls -al shows sample-project-python folder.

Run Tests Step

Now let's replace the step with listing the current directory with something more interesting, like running tests. Kraken provides several built-in tools for running tests. In the case of our Python sample project, we will use pytest built-in tool.

"steps": [{
    "tool": "git",
    "checkout": "https://github.com/Kraken-CI/sample-project-python.git"
 }, {
    "tool": "pytest",
    "cwd": "sample-project-python",
    "params": "-vv",
    "pythonpath": "src"
 }],
Enter fullscreen mode Exit fullscreen mode

Change the steps in schema editor, save it and run new CI flow. And the job output should be:

Logs from the job with running Pytest

Building Step

So if tests succeeded we could build a package of our Python project. Add step for building a wheel package using project's setup.py script:

"steps": [{
    "tool": "git",
    "checkout": "https://github.com/Kraken-CI/sample-project-python.git"
 }, {
    "tool": "pytest",
    "cwd": "sample-project-python",
    "params": "-vv",
    "pythonpath": "src"
 }, {
    "tool": "shell",
    "cwd": "sample-project-python",
    "cmd": "python3 setup.py sdist bdist_wheel"
 }],
Enter fullscreen mode Exit fullscreen mode

Now run CI flow again. The output should look as follows:

Logs from the job with packaging step with error

It seems that there is missing a package that provides wheel support for Python. Let's install it.

"steps": [{
    "tool": "git",
    "checkout": "https://github.com/Kraken-CI/sample-project-python.git"
 }, {
    "tool": "pytest",
    "cwd": "sample-project-python",
    "params": "-vv",
    "pythonpath": "src"
 }, {
    "tool": "shell",
    "cmd": "sudo apt update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends python3-wheel python3-setuptools"
 }, {
    "tool": "shell",
    "cwd": "sample-project-python",
    "cmd": "python3 setup.py sdist bdist_wheel"
 }],
Enter fullscreen mode Exit fullscreen mode

New output with building:

Logs from the job with successful deps installation and package building

Storing Artifacts Step

The results of building the sample project are two packages:

  • dist/sampleproject-2.0.0-py3-none-any.whl
  • dist/sampleproject-2.0.0.tar.gz

We could store it Kraken storage and make it available for user for downloading. For that purpose there is another built-in tool: artifacts.

"steps": [{
    "tool": "git",
    "checkout": "https://github.com/Kraken-CI/sample-project-python.git"
}, {
    "tool": "pytest",
    "cwd": "sample-project-python",
    "params": "-vv",
    "pythonpath": "src"
 }, {
    "tool": "shell",
    "cmd": "sudo apt update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y - no-install-recommends python3-wheel python3-setuptools"
 }, {
    "tool": "shell",
    "cwd": "sample-project-python",
    "cmd": "python3 setup.py sdist bdist_wheel"
 }, {
    "tool": "artifacts",
    "action": "upload",
    "cwd": "sample-project-python/dist",
    "source": [
        "sampleproject-2.0.0-py3-none-any.whl",
        "sampleproject-2.0.0.tar.gz"
    ],
    "public": True
 }],
Enter fullscreen mode Exit fullscreen mode

And the job output:

Logs from the job with storing build artifacts

Here the log shows that artifacts have been stored. Now, as they were marked as public in artifacts step, they are available in the UI:

The Run page, Artifacts tab

This is it. In this installment we learn how to clone Git repository, how to run tests using Pytest, how to install dependencies and how to build artifacts and store them in Kraken's storage. More about Kraken CI you can find on https://kraken.ci.

Top comments (9)

Collapse
 
jacksonhead profile image
jacksonhead

It is intereting to see whenever a new ci platform comes out it has a dsl for describing the pipeline. Using a complete script language (like scripted pipelines in Jenkins) can be a security risk, I understand that but nieche requirements are bound to come in. Wouldn't it worth to reserach the possibility to use an interpreter instead of a dsl and make it secure?

Collapse
 
godfryd profile image
Michal Nowikowski

Here in Kraken CI, Python is used to define pipeline. Long term plan is to have Starlark which is a safe subset of Python and it is used in e.g. Bazel, Buck or Caddy.

Collapse
 
jacksonhead profile image
jacksonhead

Oh sorry i have misinterpret the pipeline descriptor. In case i want to extend the current logic with something of my own i can explicitly do that in the 'stage' python function. But where does that function go? Where is the python file containing the function?

Thread Thread
 
godfryd profile image
Michal Nowikowski

You can put the Python code directly in the UI where a stage is defined or store the Python code in a file and put it to Git repo and indicate this in the UI (check the first picture in the article, Stage Hello World section, there is a 'Schema from Repository' tab that allows this).

Collapse
 
melezhik profile image
Alexey Melezhik

Hi. Nice posts on Kraken, thanks. However what is a pros over already existing (billions) ci/cd solutions ?

Collapse
 
godfryd profile image
Michal Nowikowski

Most of them are offered as a service. Kraken CI is an on-premise solution. Beside of that it is focused on testing. When you have thousands of test cases and there is never 100% pass ratio then it is hard to spot regressions or unstable tests (eg. Jenkins does not have good support for such things). All of that and more is present in Kraken CI out of the box. There is much more, hard to list all. Please, check Kraken CI web page and its docs or contact me directly (mike@kraken.ci) or chat here :)

Collapse
 
melezhik profile image
Alexey Melezhik

Thanks for reply. Thousands of test cases? It should be very specific development imho. And even in this case I am not sure what certain problem you are trying to solve here? I guess most of test frameworks provides a user with some sort of reports? Or I miss the point here ...

Thread Thread
 
godfryd profile image
Michal Nowikowski

Medium and big projects can have 10k, 100k or even more test cases. But even for 1k if there is hard to achieve 100% then it is hard to track changes of results over time. E.g. can you say which tests are the least stable in the last month so that you could fix or disable them? Or could you say which tests started failing during the last week? Or if you have some performance tests, lots of them, then which of them degraded? I think that it is very hard to get answers for such questions in Jenkins. I was writing some scripts to do such analysis and generate some reports using Jenkins API but this is a dead end. In the past I wrote similar testing system to Kraken but it was proprietary and it was used only internally. Kraken is an open source version of the concepts that emerged in the original system.

Thread Thread
 
melezhik profile image
Alexey Melezhik

Ok. I get you, thanks for clarification. You might be then interested in tool, I've created - Tomty and see how it deals with tests subgroups - dev.to/melezhik/write-devopsish-te... , see tags section.