DEV Community

Cover image for We made an AI SWE that solved 48.60% of issues on the SWE bench, 100% open-source.
Sunil Kumar Dash for Composio

Posted on • Edited on

We made an AI SWE that solved 48.60% of issues on the SWE bench, 100% open-source.

We at Composio are building the tool infrastructure for AI agents. One of our users' biggest requests was toolkits for building custom coding agents that work. So, we created SWE-Kit, a starter template with all the toolkits for building AI coding agents.

These agents can run locally end-to-end to automate your coding workflows.

To test the efficiency of our tools, we built a comprehensive AI agent complete open-source using LangGraph and tested it on SWE-bench verified, and it got 48.60% and 41% on SWE-bench lite.

SWE bench ranking

SWE-bench is a benchmark with over 2,200 real-world Python issues from repositories such as Django, Flask, Sklearn, SciPy, etc.

SWe-kit on SWE bench GIF

Check out the technical report for more: Tool design is all you need for SOTA AI agents
Complete code for the SWE agent: SWE-kit Agent


The tools you need for running the agent locally

  • Code Analysis Tool: Intelligently retrieves relevant code snippets from the repository.
  • File Tool: Facilitates navigation and updates to files.
  • Shell Tool: Performs shell operations.
  • Git Tool: Handles version control tasks.
  • Composio Docker workspace for isolated code execution.

We optimized the tools for improved function calling accuracy.


What can you build with SWE-kit and Composio?

The code is open-source, and you can even modify it to add external integrations like GitHub, Jira, Linear, Slack, etc., using Composio to build a full-fledged AI software engineer.

You can automate many aspects of your Software development workflows with custom agents such as,

  • Writing codes
  • refactoring code bases
  • testing
  • documentation
  • Project management with Linear or Jira, etc.
  • Communication using Slack and Gmail.

For an architectural explanation of the SWE-Kit agent, check out the SWE-Kit agent blog published on LangChains’ blog.

I am not even kidding. Many companies have raised millions just from this.

Start building your custom local coding agent with SWE-kit now.

Get Started with SWE-kit

Top comments (0)