Creating An AI Agent For Kubernetes Performance Optimization

#kubernetes #ai #devops #cloud

Instead of the continuous buzz/hype/notion/complete lie that “AI is going to take engineering jobs”, the technical implementors, managers, and leaders need to come together and work on solutions for the next iteration of engineering.

In this blog post, you’ll learn about a quick way to create an Agent that could help you understand performance per region based on where you’re deploying k8s workloads.

Why AI Is Not “Taking Jobs”

When virtualization came out, there were a plethora of engineers that said “it’s going to take our server jobs”.

When PowerShell became very popular in the Windows space, a lot of Sysadmins said “automation is going to take our job”.

When the cloud came out, when DevOps came out, when… well, you get where I’m going with this. The “it’s going to take our jobs” trend has been around for over 30 years.

Every iteration and growth of technology always has a group of people that think it’s going to replace them and guess what? If those engineers don’t keep up, re-train, and progress, it will. They will be phased out for new roles and careers within the tech realm… and that’s how it’s supposed to be. Tech isn’t supposed to remain stagnant as it would quite literally alter the majority of society's progression.

Now that we got that out of the way, let’s push some good energy back into engineering and see how AI can be fun for DevOps engineers and can help them by utilizing Agents.

How Can an Agent Be Used for DevOps

ChatGPT came out in 2022 which really kickstarted the whole AI movement.

💡

AI has been around since the 60’s. It’s not new, it just never had a “hype cycle”.

Since then, there have been a lot of use cases (primarily around chatbots) that have been great, but aside from the data and ML space, there haven’t been a lot of true use cases for DevOps/Platform/Cloud Engineers aside from LLM being implemented in some of the tools that engineers may use.

There may now be a good use case - AI Agents.

There are a lot of use cases for AI Agents, but one use case engineers have been diving into lately has been to perform specific actions for them. You can almost think of it as an “automation 2.0” of sorts.

I started diving into this personally and I’m at the beginning stages of creating an Agent to tell me what region has the best performance and cost to run workloads in Kubernetes. As I’ve been building it out, I thought to myself “This is a great use case for engineers”. Let’s dive into what it looks like.

Defining The Available Libraries

Any time you’re writing software, you always have two choices in front of you:

Hit an API.
Use an existing library/module/package.

Using a library that already exists is usually the path of least resistance as someone else already wrapped the APIs for you. This does, however, lead to you being dependent on that library. In open-source, one unfortunate reality is that sometimes a project stops getting worked on, which means you’re at the mercy of the maintainers unless you fork the project and start working on it yourself.

Luckily in the AI space, there are a lot of projects and although not all of them will get to production, there are a few that will. For example, crewai is a new product, but they’re in direct collaboration with Nvidia when it comes to AI Agents, so there’s a pretty good chance that the libraries will stick around.

Because of that, for the purposes of building an Agent, the following sections in this blog post show how to build one with crewai.

Configuration

For the purposes of this blog post, you’ll see how to create an AI Agent in Python using crewai and gpt-4. It’ll be split up into a few different sections that include how crewai interacts with each Model and how it retrieves the information you’re asking for.

Library and Authentication

The first thing is to ensure you implement the right libraries along with methods and authentication/authorization.

The two primary libraries you'll use are:

crewai
crewai_tools

The crewai_tools library is what you’ll use for the actual search of the data via the Models.

During the process of accessing the data, you’ll need to authenticate to where you’re pulling data from. OpenAI is one of the current leaders in LLMs and Serper is a free Google Search API.

from crewai import Agent, Task, Crew, Process, LLM
from crewai_tools import SerperDevTool, WebsiteSearchTool
import os

os.environ.get('OPENAI_API_KEY')
os.environ.get('SERPER_API_KEY')

One thing to keep in mind is the rate at which the data is fed to you via this application. Chances are you’ll have to spend a few dollars for the OpenAI calls.

💡

I used $5.00 when testing this and it worked great.

Search Algorithm

The next step is to set up the search. It will consist of a few primary pieces:

Searching websites
Searching Google
Choosing a specific model to use for search purposes

And with that information, you use the Agent method from the crewai library.

    serper = SerperDevTool()
    web = WebsiteSearchTool()
    modelType = LLM(model='gpt-4', temperature=0.2)

    search = Agent(
        role="Performance Optimizer",
        goal="Optimize the performance of containerized workloads based on region that they are deployed to.",
        backstory="based on performance and cost, deploying containerized solutions in a specific region may be necessary.",
        tools=[serper, web],
        # if you don't specify a Model, it will default to gpt-4
        llm=modelType,
    )

A good thing to keep in mind here is specifying the tools . The tools section is where you specify how you’re searching for the data. In this case, you’re using both OpenAI and Serper, but you could choose to only use one if you’d like based on what you need to get done.

Job

The job, or rather, the “task” is what’s used to perform the action of the actual search. The goal is to have the task provide all of the information needed for the Agent to do its job properly.

    job = Task(
        description="tell me the best region to deploy Kubernetes workloads to based on performance and cost.",
        expected_output="The best region to deploy Kubernetes workloads to based on performance and cost.",
        agent=search
    )

Notice how the agent value is the Agent workflow from the previous section.

Running The Workload

To run the AI Agent, you’ll use the Crew method.

The kickoff() command is used to start the Agent.

    crew = Crew(
        agents=[search],
        tasks=[job],
        verbose=True,
        process=Process.sequential
    )

    crew.kickoff()

Altogether, the Agent code will look like the below.

from crewai import Agent, Task, Crew, Process, LLM
from crewai_tools import SerperDevTool, WebsiteSearchTool
import os

os.environ.get('OPENAI_API_KEY')
os.environ.get('SERPER_API_KEY')

def main():
    serper = SerperDevTool()
    web = WebsiteSearchTool()
    modelType = LLM(model='gpt-4', temperature=0.2)

    search = Agent(
        role="Performance Optimizer",
        goal="Optimize the performance of containerized workloads based on region that they are deployed to.",
        backstory="based on performance and cost, deploying containerized solutions in a specific region may be necessary.",
        tools=[serper, web],
        # if you don't specify a Model, it will default to gpt-4
        llm=modelType,
    )

    job = Task(
        description="tell me the best region to deploy Kubernetes workloads to based on performance and cost.",
        expected_output="The best region to deploy Kubernetes workloads to based on performance and cost.",
        agent=search
    )

    crew = Crew(
        agents=[search],
        tasks=[job],
        verbose=True,
        process=Process.sequential
    )

    crew.kickoff()


if __name__ == '__main__':
    main()