I usually work on multiple projects simultaneously, and managing all these projects can become very hectic and overwhelming. So, I wanted a solution to reduce my workload.
Considering the significant progress in AI in the past few years, I was specifically looking for an AI tool that can
- Access my GitHub repository.
- Solves any given issue autonomously.
- Tests the code and pushes the fix to the repository.
So, I started my research and stumbled upon quite a few solutions. Here is my experience working with these solutions.
But before that, let’s understand what AI and SWE (Software Engineering) agents are.
What are AI Agents?
AI agents are systems powered by AI models that can autonomously perform tasks, interact with their environment, and make decisions based on their programming and the data they process.
Components of AI Agents
An AI agent has three crucial components.
- Inference Engine: Inference Engines are AI models responsible for planning and decision-making.
- Memory: The memory component allows the agents to manage short-term and long-term information to keep track of the workflows.
- Tools: Tools enable Agents to interact with the external environment. For instance, a GitHub tool allows the agent to extract information from repositories and make necessary changes.
Characteristics of an SWE Agent
So, what makes an AI agent a SWE agent?
A SWE agent is an AI agent that mimics the qualities and characteristics of a human software developer, such as
- Long-term planning and reasoning
- Use standard developer tools.
- Improve code quality through testing and feedback.
- Debug and resolve issues autonomously.
- Collaborate with human counterparts as and when needed.
Simillar to any AI agent, an SWE agent has
- A robust inference engine. Preferably state-of-the-art code models.
- A memory component to keep track of events.
- Specific tools for writing, debugging, and running codes (Isolated code environment, Internet, GitHub, File tools, etc.)
Devin - The AI Software Developer
Devin is the first thing you will find when discussing AI agents that can code. It is an SWE agent from Cognition Labs, which started it all. It garnered a lot of attention from developers, founders, and investors and quickly became the talk of the town.
However, a few things made me turn away from it.
- Closed Source: It is a closed source. So, there is no scope for customization.
- OpenAI only: It exclusively utilizes OpenAI models, considering Llama 3 400b launch. I wanted something more flexible.
- Not Publicly Available: It is still on the waitlist, and few people can access it.
While enterprises may prefer it, I needed something more transparent and flexible.
Open-source Alternatives
The next obvious choice was all the open-source alternatives for Devin. While they are more transparent and flexible in terms of LLM providers. They still didn’t click with me.
Most open-source solutions are
- Bloated: They ship with many things I might never use.
- Complex: Running these tools felt like a job.
- Non-extensible: I can’t extend their capabilities using external tools and integrations.
That being said, they certainly hold their own merits.
SweKit - Simple Extensible Software Engineering Agent Framework
After extensive searching and research, I was leaning towards building it myself until I stumbled upon this framework.
SweKit is an open-source framework from Composio that helps you create simple and extensible SWE agents.
It clicked all the criteria I was looking for.
- Customizable: It is simple and highly customizable. You can use the LLM providers of your choice and change the prompts to suit your requirements.
- Framework Agnostic: It works with all the popular agentic frameworks, such as LangChain and LlamaIndex.
- Extensible: Adding more tools, such as browsing and scrapping, can easily extend the agent's capability. You can add tools from the Composio ecosystem.
- Open-source: Needless to say, the code is open-source, so you can fork and add custom changes.
- Multiple Language Support: SweKit has native support for Python and Typescript. You can go with the language of your choice.
SweKit can access any public or private repository, work on provided issues, and push changes to the repository.
The SWE agent runs in Docker by default for security and isolation. This sandboxes the agent's operations, protecting against unintended consequences of arbitrary code execution. You can also run it locally, but be cautious, as it won't have Docker's protective layer.
Final Words
I have already started using this in my projects, and I cannot stress enough how much time and energy it has saved me.
I spoke with the team building Composio, and I can confidently say this will be a game-changer.
I cannot star them enough, so do me a favour, visit the repository, use the product and give them a star.
Top comments (19)
And not a single example of what it achieved? I really doubt this tool boosting anyone's production by 10 times. And I highly doubt it can pick up any issue and resolve it itself.
This is Dev.to where no one can actually code. Where teenagers think they know more than professionals.
Unfortunately, 10X of 0 is still 0.
New AI Agent in the town.
How does this hold up with tasks like scraping data out of a document, running Python analytics, and then outputting them in a doc file?
I'm looking for something that can do that.
It's great!
You should try it out!
Sure, cloning already. ^^
How much of a vested interest do you have in this particular project? This article is an advertisement. It's nonsense...
You show no proof that it increased your productivity 10x... In fact, you show no code at all, which makes me wonder if you can even write code in the first place. I suppose you could get a 10x improvement on your productivity if you are barely productive in the first place because you're an advertiser/vc and not an engineer.
You know Devin was a farce right? All of the videos they used to show off its "functionality" were proven to be faked. That you even reference it in this article shows your priorities...
If you want to shill the latest AI nonsense then at least put some effort into it. Make it seem like it's got some legitimate use cases outside of stealing other people's code to make surface level garbage...
Interesting that this is the article which gets huge upvotes on Dev.to.
Is it an indication of the future? Probably.
Just something to think about.
You find the article not good?
No, it's not that I don't think the article is good -- and I don't think the article is bad.
I just find it interesting that on a site devoted to Articles About Software Development (and learning software dev) that one of the most highly upvoted articles :
Imagine if I told you, you were going to be a programmer then showed you how to record a macro in MS Word. That's the kind of feel this has.
For me, the article is misleading Nevo, I am sorry. It does not explain what the capabilities offered by an SWE agent are, what are the practical advantages of swekit over competing products, or how the productivity is tenfold enhanced.
This is why I stop using this site.
This article has a very abrupt ending. The author introduces a very promising new tool (thanks for that, of course), and I'm ready/excited to see what it can do, but the article just ends! It would be great if you included some examples of how it improved your productivity exactly. Otherwise, it's very unclear and seems like an ai-generated ad.
As a full-time programmer, I'm of course on a lookout for the new great tools, and I appreciate the heads up like this, but I need more info to decide if I want to spend my very limited time to try it.
I am using Tabnine Pro version for my dev work (primarily PHP now, but adding Python and javascript to the mix soon)
It is a hit-and-miss scenario: sometimes it really helps write some parts of the code faster, but sometimes the suggestions are very inaccurate.
I believe that if you are an experience dev you will use it to do small repetitive things that are tiresome to type, but to create logic... AI has a long way to go, especially in frameworks and complex structures.
So, I don't know if 10x faster will be possible even in the future, AI is creating logic based on predefined parameters, but humans are more flexible and can merge several scenarios to create something that will work in this particular case.
I find AI assistant works very well with migrations and shorter assignments.
Excellent!
What about Pythagora?
Super informative article
Nevo, I think your article is awesome. Thanks for sharing your research results with us. Saving us a lot of time. Please continue with posting your thoughts about tools you find
These comments restore some of my confidence in humanity.
Composio team is doing great work.