Build Tool Calling Agents with LangGraph and IBM watsonx.ai Flows Engine

Generative AI is evolving quickly, and in the past couple of years, we’ve seen a few clear patterns developer can implement — starting with simple prompting, moving to Retrieval Augmented Generation (RAG), and now Tool Calling and Agents. Building applications around Agents often involves connecting different tools, models, and APIs.

In this tutorial, you’ll learn how to set up and run an AI agent using watsonx.ai Flows Engine and LangGraph, with help from models powered by IBM’s watsonx.ai platform. This guide will take you step by step through installing the tools you need, deploying your tools to Flows Engine, and running a chat application on your own machine.

Tool Calling & Agents

Tool calling allows Large Language Models (LLMs) to work with external tools by following user-defined instructions. The model figures out the needed inputs for a tool, and the tool does the actual work. This is great for turning existing data into structured information or linking tools into workflows. Flows Engine makes this process easier by offering ways to turn your existing data sources such as databases and APIs into tools, and connect to these tools from different frameworks for building AI applications such as LangChain and LangGraph.

Agents are like AI helpers built to handle specific tasks, and to complete these talks they rely on tool calling. This allows them to access external tools, process information, and expand the exsisting knowledge of the LLM to fit the user’s needs. There are several popular frameworks for building agents, either in JavaScript or Python. In this tuturial we'll be using LangGraph, which builds upon LangChain, to create an Agent that is able to call tools that can retrieve information from Google Books and Wikipedia. The tools will be defined in Flows Engine, and connected through the JavaScript SDK as you'll learn in the next section.

Build a Tool Calling Agent

In this section you'll learn how to set up and run an AI agent using watsonx.ai Flows Engine (wxflows) and LangGraph. The agent will be able to call several tools to answer questions about your favorite books or other popular media, using a set of prebuilt community tools. Of course, you can also convert your own data sources (such as databases and APIs) into tools. We'll cover everything you need to know, from installing the tools to deploying and running the app on your machine.

This example includes the following technologies:

• LangGraph SDK (for the agent)
• LangChain SDK watsonx.ai extension (for the models)
• wxflows SDK (for the tools)
• Carbon AI Chat (for the user interface)

We'll be using the watsonx.ai Chat Model from LangChain, but you can use any of the supported chat models.

Continue to the next section to start installing the wxflows CLI, set up your Flows Engine project, and run the agent via a chat application. We’ll use google_books and wikipedia as examples of tools that the agent can call.

Prerequisites

Before you begin, you should create an account for watsonx.ai Flows Engine for this tutorial:

• Sign up for a free watsonx.ai Flows Engine account.
• Sign up for the free watsonx.ai trial to access models. Alternatively, you can use Ollama to run a LLM locally.
• You need to have Python (version 3.8 or higher) installed.

Step 1: Install the wxflows CLI

Start by installing the wxflows CLI, which needs Python (version 3.8 or higher) to be installed on our machine. Follow these steps:

1. Download the CLI from the installation page.

2. Create a new folder on your computer.
   

   mkdir wxflows-project
   cd wxflows-project
   

3. Inside this folder, run:
   

   pip install wxflows_cli-1.0.0rc192-py3-none-any.whl --force-reinstall
   

   Use the exact name of the .whl file you downloaded.

4. Log in to the CLI by following the login instructions.
   

   wxflows login
   

After setting up the CLI for watsonx.ai Flows Engine let's continue to the next section and create an endpoint with the Wikipedia and Google Books tools.

Step 2: Set up a watsonx.ai Flows Engine project

Now, set up a new wxflows project. This will include two tools:

• google_books: Used to search and retrieve information from Google Books.
• wikipedia: Used to query and retrieve information from Wikipedia.

Run the following command to initialize the project and import these tools:

wxflows init --endpoint-name api/wxflows-toolcalling \
--import-name google_books --import-package https://github.com/IBM/wxflows/raw/refs/heads/main/tools/google_books.zip \
--import-tool-name google_books --import-tool-description "Retrieve information from Google Books. Find books by search string, for example to search for Daniel Keyes 'Flowers for Algernon' use q: 'intitle:flowers+inauthor:keyes'" --import-tool-fields "books|book" \
--import-name wikipedia --import-package https://github.com/IBM/wxflows/raw/refs/heads/main/tools/wikipedia.zip \
--import-tool-name wikipedia --import-tool-description "Retrieve information from Wikipedia." --import-tool-fields "search|page"

Here’s what this command does:

• Creates an endpoint: Defines an endpoint called api/wxflows-toolcalling.
• Imports google_books: Adds a tool for retrieving books from Google Books with a description and specific fields like books|book.
• Imports wikipedia: Adds a tool for searching Wikipedia with a description and fields like search|page.

After running the command, you can see a wxflows.toml file appear in the current directory. This file holds the configuration for your tools, which we'll deploy to an endpoint by running:

wxflows deploy

Deploying the endpoint should only take a couple of seconds, and the wxflows SDK will use this endpoint to retrieve and call tools as you'll learn in the next section.

Step 3: Use LangGraph to create an agent

Now that we have a Flows Engine endpoint, we need to create an agent that can use this endpoint for tool calling. We'll be using LangGraph for this, together with the wxflows SDK and the LangChain extension for watsonx.ai:

1. Create a new directory:
   

   mkdir langgraph
   cd langgraph
   

2. In this directory run the following command to set up a new TypeScript project and to install the needed dependencies:
   

   npm init -y
   npm i @langchain/langgraph @langchain/core @langchain/community dotenv typescript @wxflows/sdk@beta
   

   This installs the following packages:

- `@langchain/langgraph`: Build agents, structured workflows and handle tool calling.
- `@langchain/core`: Provides core functionality for building and managing AI-driven workflows.
- `@langchain/community`: Provides access to experimental and community-contributed integrations and tools, such as models on the watsonx.ai platform.
- `dotenv`: Loads environment variables from a `.env` file, making it easier to manage API keys and configuration values.
- `typescript`: Sets up TypeScript, a superset of JavaScript, for type checking and better code quality.
- `@wxflows/sdk@beta`: Includes the SDK for working with watsonx.ai Flows Engine, which allows you to deploy and manage tools.

After running these commands, your project will be ready to start building an agent.

1. Create an .env file with the following values:
   

   # You can get these credentials from https://dataplatform.cloud.ibm.com/developer-access?context=wx
   WATSONX_AI_AUTH_TYPE=iam
   WATSONX_AI_ENDPOINT=https://us-south.ml.cloud.ibm.com
   WATSONX_AI_IDENTITY_SERVER=iam.cloud.ibm.com
   WATSONX_AI_APIKEY=
   WATSONX_AI_PROJECT_ID=
   
   # run the command 'wxflows whoami --apikey'
   WXFLOWS_APIKEY=
   
   # endpoint shows in your terminal after running 'wxflows deploy'
   WXFLOWS_ENDPOINT=
   

2. After creating the .env file, create a new file called index.ts and add the code for LangGraph:
   

   import { AIMessage, BaseMessage, HumanMessage, SystemMessage } from "@langchain/core/messages";
   import { ChatWatsonx } from "@langchain/community/chat_models/ibm";
   import { StateGraph } from "@langchain/langgraph";
   import { MemorySaver, Annotation } from "@langchain/langgraph";
   import { ToolNode } from "@langchain/langgraph/prebuilt";
   import wxflows from "@wxflows/sdk/langchain";
   import "dotenv/config";
   
   (async () => {
       // Define the graph state
       // See here for more info: https://langchain-ai.github.io/langgraphjs/how-tos/define-state/
       const StateAnnotation = Annotation.Root({
           messages: Annotation<BaseMessage[]>({
               reducer: (x, y) => x.concat(y),
           }),
       });
   
       const toolClient = new wxflows({
           endpoint: process.env.WXFLOWS_ENDPOINT,
           apikey: process.env.WXFLOWS_APIKEY,
           traceSession: '...'
       });
   
       const tools = await toolClient.lcTools;
       const toolNode = new ToolNode(tools);
   
       // Connect to the LLM provider 
       const model = new ChatWatsonx({
           model: "mistralai/mistral-large",
           projectId: process.env.WATSONX_AI_PROJECT_ID,
           serviceUrl: process.env.WATSONX_AI_ENDPOINT,
           version: '2024-05-31',
       }).bindTools(tools);
   

   This first part of the code will import the needed dependencies, retrieves the tools from watsonx.ai Flows Engine and sets up the connection to Mistral Large running on the watsonx.ai platform. Let's add the second part of the code for the index.ts file which will handle the creation of the agent:
   

       // Define the function that determines whether to continue or not
       // We can extract the state typing via `StateAnnotation.State`
       function shouldContinue(state: typeof StateAnnotation.State) {
           const messages = state.messages;
           const lastMessage = messages[messages.length - 1] as AIMessage;
   
           // If the LLM makes a tool call, then we route to the "tools" node
           if (lastMessage.tool_calls?.length) {
           console.log('TOOL CALL', lastMessage.tool_calls)
           return "tools";
           }
           // Otherwise, we stop (reply to the user)
           return "__end__";
       }
   
       // Define the function that calls the model
       async function callModel(state: typeof StateAnnotation.State) {
           const messages = state.messages;
           const response = await model.invoke(messages);
   
           // We return a list, because this will get added to the existing list
           return { messages: [response] };
       }
   
       // Define a new graph
       const workflow = new StateGraph(StateAnnotation)
           .addNode("agent", callModel)
           .addNode("tools", toolNode)
           .addEdge("__start__", "agent")
           .addConditionalEdges("agent", shouldContinue)
           .addEdge("tools", "agent");
   
       // Initialize memory to persist state between graph runs
       const checkpointer = new MemorySaver();
   
       // Finally, we compile it!
       // This compiles it into a LangChain Runnable.
       // Note that we're (optionally) passing the memory when compiling the graph
       const app = workflow.compile({ checkpointer });
   
       // Use the Runnable
       const finalState = await app.invoke(
           {
           messages: [
               new SystemMessage(
               "Only use the tools available, don't answer the question based on pre-trained data"
               ),
               new HumanMessage(
               "Search information about the book escape from james patterson"
               ),
           ],
           },
           { configurable: { thread_id: "42" } }
       );
   
       console.log(finalState.messages[finalState.messages.length - 1].content);
       // You can use the `thread_id` to ask follow up questions, the conversation context is retained via the saved state (i.e. stored list of messages):
   })();
   

   In this second part of the code, the agent is created and we pass a message to the agent: "Search information about the book escape from james patterson". This will kick off a series of interactions between the tools running in Flows Engine and the LLM in watsonx.ai to provide you with an answer.

   1. To run the above code, you need to add a start script to the package.json file:

   "scripts": {
       "start": "npx tsx ./index.ts",
       "test": "echo \"Error: no test specified\" && exit 1"
   }
   

   1. To send the message to the agent you can now run npm start from the terminal, which should return more information about the book "Escape" by "James Patterson". Next to the information, the used tool calls will also be printed in your terminal.

   The tool call will look like the following:
   

   TOOL CALL [
     {
       name: 'google_books',
       args: {
         query: '{\n' +
           '  books(q: "intitle:escape+inauthor:patterson") {\n' +
           '    authors\n' +
           '    title\n' +
           '    volumeId\n' +
           '  }\n' +
           '}'
       },
       type: 'tool_call',
       id: 'Zg9jXPOR2'
     }
   ]
   

   watsonx.ai Flows Engine is using GraphQL as the underlying technology to define and call tools.

   And the final response something like this:
   

   ### Information about the book "Escape" by James Patterson
   
   Here is the information about the book "Escape" by James Patterson:
   
   - **Authors:** James Patterson, David Ellis
   - **Title:** Escape
   - **Volume ID:** EFtHEAAAQBAJ
   

   The next step would be to implement a chat application that's using the Flows Engine endpoint and the LangGraph agent, as you'll learn in the next section.

Step 4: Use the agent in a chat application

The final step is to use the agent and the tools endpoint in a chat application, which you can find in this repository.

You can run this application in the browser using the free service StackBlitz:

You can either run the chat application in the browser via StackBlitz or use the "Download" button to download the source code for the application and run it locally.

This application will need the same credentials as in step 3.3, which you can store in a new .env file. Either directly in StackBlitz or locally:

# You can get these credentials from https://dataplatform.cloud.ibm.com/developer-access?context=wx
WATSONX_AI_AUTH_TYPE=iam
WATSONX_AI_ENDPOINT=https://us-south.ml.cloud.ibm.com
WATSONX_AI_IDENTITY_SERVER=iam.cloud.ibm.com
WATSONX_AI_APIKEY=
WATSONX_AI_PROJECT_ID=

# run the command 'wxflows whoami --apikey'
WXFLOWS_APIKEY=

# endpoint shows in your terminal after running 'wxflows deploy'
WXFLOWS_ENDPOINT=

If you want to run the application locally, don't forget to run npm i in the application's directory to install the dependencies. After installing these, you can run npm start to boot up the application.

The chat application is using IBM's Carbon Chat AI component library, and looks like this:

You can now ask any sort of question related to books or other popular media. The agent will use the Google Books or Wikipedia tool from watsonx.ai Flows Engine to answer your questions, as LangGraph keeps memory state you can also ask follow up questions. The next step would be turn your own (enterprise) data into a tool, for which you can have a look at these instructions.

Summary and Next Steps

This tutorial showed you how to set up and run an AI agent using IBM watsonx.ai Flows Engine and LangGraph. You learned how to configure tools for google_books and wikipedia, deploy a Flows Engine project, and run the application locally or in a browser. These tools enable your agent to retrieve real-time data and interact with external APIs.

With these new skills, you now have a strong foundation for building AI-powered applications using wxflows and watsonx.ai. Whether you’re creating simple workflows or more complex integrations, the CLI and SDK make it easy to bring your projects to life.

We’re excited to see what you create! Join our Discord community and share your projects with us!