Debugging Your Crew: Isolating Agents and Tasks in CrewAI

If you're working with CrewAI to build multi-agent AI systems, you know that things can get complex quickly. As your crew grows, debugging and making changes to individual agents or tasks can become quite challenging.

In this post, we will explore how to run agents and tasks independently.

Why Isolate Agents and Tasks?

Isolating parts of your crew is like testing individual components of a larger system. It offers these practical benefits:

• Easier Debugging: When something isn't working right, testing the specific agent or task in isolation helps you quickly pinpoint the problem. You won't have to wade through logs from the entire crew.

• Faster Iteration: If you want to change how an agent behaves or how a task is defined, you can test those changes quickly without running the whole crew each time.

• Focused Performance Tuning: You can profile and optimize individual components more easily when you isolate them from the rest of the system.

Let's look at the main parts:

1. The researcher Agent (agents.yaml):
   

   researcher:
     role: "Senior Research Analyst"
     goal: "Uncover groundbreaking technologies in AI"
     backstory: "A highly skilled researcher with a passion for AI advancements."
     llm: gemini/gemini-1.5-flash # Replace with your desired LLM
     allow_delegation: false
     tools:
       - WebSearchTool
   

2. The research_task (tasks.yaml):
   

   research_task:
     description: "Research the latest developments in AI for 2024."
     expected_output: "A report summarizing the key AI trends."
     agent: researcher
   

3. The IndependentCrew Class (crew.py):
   

   from crewai import Agent, Crew, Process, Task
   from crewai.project import CrewBase, agent, crew, task
   from .tools import WebSearchTool
   
   @CrewBase
   class IndependentCrew():
       """IndependentCrew crew"""
   
       agents_config = 'config/agents.yaml'
       tasks_config = 'config/tasks.yaml'
   
       # If you would like to add tools to your agents, you can learn more about it here:
       # https://docs.crewai.com/concepts/agents#agent-tools
       @agent
       def researcher(self) -> Agent:
           return Agent(
               config=self.agents_config['researcher'],
               verbose=True,
               tools=[WebSearchTool()]
           )
   
       @task
       def research_task(self) -> Task:
           return Task(
               config=self.tasks_config['research_task'],
           )
   
       @crew
       def crew(self) -> Crew:
           """Creates the IndependentCrew crew"""
   
           return Crew(
               agents=self.agents, # Automatically created by the @agent decorator
               tasks=self.tasks, # Automatically created by the @task decorator
               process=Process.sequential,
               verbose=True,
           )
   

4. Running the Agent Independently (run_agent.py):
   

   from crewai import Agent, Task
   from .crew import IndependentCrew
   from .tools import WebSearchTool
   import os
   
   # Create an instance of the IndependentCrew class to access its configurations
   crew_instance = IndependentCrew()
   
   # -- Option 1: Create agent from config and execute a custom task --
   # Access the agents configuration
   agent_config = crew_instance.agents_config['researcher']
   
   # Now you can create your agent using this configuration
   researcher_agent = Agent(
       **agent_config,
       tools=[WebSearchTool()],
       verbose=True
   )
   
   # Create a Task object with description, expected_output, and agent
   task = Task(
       description="What are the latest advancements in AI?",
       expected_output="A report summarizing the latest advancements in AI.",
       agent=researcher_agent
   )
   
   # Execute the task with the agent
   result = researcher_agent.execute_task(task)
   print("Result from custom task:", result)
   
   # -- Option 2: Reuse agent and task from crew definition --
   researcher_agent = crew_instance.researcher()  # Get agent from crew definition
   research_task = crew_instance.research_task()  # Get task from crew definition
   research_result = researcher_agent.execute_task(research_task)
   print("Result from crew-defined task:", research_result)
   

5. Running the Task Independently (run_task.py):
   

   from crewai import Agent, Task
   from .crew import IndependentCrew
   from .tools import WebSearchTool
   from concurrent.futures import Future
   import os
   
   # Create an instance of the IndependentCrew class to access its configurations
   crew_instance = IndependentCrew()
   
   # -- Option 1: Create task from config and execute with a created agent --
   
   # Access the agents configuration
   agent_config = crew_instance.agents_config['researcher']
   
   # Now you can create your agent using this configuration
   researcher_agent = Agent(
       **agent_config,
       tools=[WebSearchTool()],
       verbose=True
   )
   
   # Access the tasks configuration
   research_task_config = crew_instance.tasks_config['research_task']
   
   # Create the task using its configuration
   research_task = Task(
       **research_task_config,
       verbose=True
   )
   
   # Execute the task synchronously
   output = research_task.execute_sync(agent=researcher_agent)
   print("Synchronous Task Output:", output.raw)
   
   # Execute the task asynchronously
   future_output: Future = research_task.execute_async(agent=researcher_agent)
   # Perform other operations...
   
   # Retrieve the result
   output = future_output.result()
   print("Asynchronous Task Output:", output.raw)
   
   custom_context = "Focus on AI in healthcare for the year 2023."
   custom_tools = []  # You can add or remove tools here
   
   output = research_task.execute_sync(
       agent=researcher_agent,
       context=custom_context,
       tools=custom_tools
   )
   print("Task Output with Custom Context:", output.raw)
   
   # -- Option 2: Reuse task from crew definition and provide custom context --
   research_task = crew_instance.research_task()
   result = research_task.execute_sync(context="What are the ethical implications of AI?", agent=researcher_agent)
   print("Task result with custom context:", result)
   

Key Takeaways

• Flexibility: You don't always have to run the entire crew. Isolate and test specific agents or tasks as needed.
• Control: You have fine-grained control over the inputs and context when testing individual components.
• Efficiency: Develop and debug faster by focusing your efforts.

Wrapping Up

Running agents and tasks independently is a helpful technique to have in your CrewAI toolbox. I hope this example project gives you a clear idea of how to implement it in your own projects. Feel free to experiment, adapt the code, and see how it can improve your development process.

If you have any questions, thoughts, or suggestions, feel free to leave a comment below. Let's keep building awesome AI crews!

Links:

• GitHub Repository: https://github.com/apappascs/crewai-independent-execution
• CrewAI Documentation: https://docs.crewai.com