Introduction
I've been exploring multiple AI tools like Continue.dev, GitHub Copilot, and Aider to enhance my development workflow, each offering unique ways to streamline coding and improve productivity. As I move across these coding assistants, I've been using an open source local AI Gateway, CodeGate, to ensure AI-generated recommendations follow best practices while maintaining my code's integrity and safeguarding personally identifiable information (PII) such as email addresses and credit card details. Here is a a diagram of the flow:
I recently experimented integrating two open source projects, Spring AI and CodeGate with the goal of leveraging Spring AI's familiar programming model and take advantage of CodeGate's PII protection so I didn't need to write this business logic myself.
This blog post walks through a Spring AI sample application that leverages OpenAI’s API and takes advantage of CodeGate running locally to prevent data leakage.
Overview of the Sample Application
This sample Java application uses Spring AI to help solve the fundamental challenge of connecting with various AI models (e.g. OpenAI) and integrating with enterprise data and APIs. Spring AI is inspired by projects like LangChain and LlamaIndex but its not direct port. The Spring AI docs say it best:
"The project was founded with the belief that the next wave of Generative AI applications will not be only for Python developers but will be ubiquitous across many programming languages."
The application is a command line interface (CLI) AI assistant built with Spring AI. It allows users to input prompts and receive AI-generated responses using OpenAI’s models. However, instead of sending data directly to OpenAI, the application routes requests through CodeGate. This ensures that any PII, such as email addresses or credit card numbers, is automatically redacted before the request reaches OpenAI’s API. The following sequence diagrams depicts the flow of the application:
Key Features
- Interactive Chat Interface: Users can ask questions via a CLI and receive AI-powered responses.
- Spring AI Integration: Uses OpenAI’s GPT models through Spring AI’s abstraction layer.
- CodeGate Privacy Protection: Intercepts and redacts PII before sending requests to OpenAI.
Why Use a local AI Gateway/Proxy like CodeGate?
Many AI applications rely on cloud-based models, which means user data is transmitted over the internet. Without proper safeguards, this can lead to unintended data exposure. Here’s how CodeGate helps mitigate these risks:
1. Automatic PII Redaction
CodeGate scans and removes sensitive information before sending the request to OpenAI. For example, if a user asks:
Who is test.sender@example.com?
CodeGate will replace the email with a generic identifier before it reaches OpenAI:
Who is [REDACTED]?
This prevents accidental exposure of personal data while still allowing meaningful interactions.
2. Local AI Processing
By running CodeGate as a local proxy (via Docker), developers maintain control over data flow without relying on third-party intermediaries. This reduces the risk of leaks and improves compliance with privacy regulations.
3. Seamless Integration
CodeGate acts as a drop-in replacement for OpenAI’s API endpoint. Developers only need to update their application’s configuration to route AI requests through the proxy:
spring.ai.openai.base-url=http://localhost:8989/openai
With this simple change, all API calls benefit from enhanced privacy without modifying application logic.
Running the Sample Application
To get started, clone the repository and set up the necessary dependencies:
git clone https://github.com/StacklokLabs/spring-ai-codegate-sample.git
cd sample
mvn clean install
Next, deploy CodeGate using Docker:
- Follow these instructions to download and install CodeGate.
Finally, run the Spring Boot application:
mvn spring-boot:run
The CLI will prompt you for input. You can verify everything is running properly by entering the following command in the CLI:
codegate version
If everything is working properly, you will see a the following response:
Assistant: CodeGate version: v0.1.18
Please note: PII protection was introduced in CodeGate v0.1.18 so this is the minimum version of CodeGate to use for this demo.
Finally, try entering a question containing an email address to see CodeGate’s redaction in action. You can use the CLI prompt:
Who is test.sender@example.com
CodeGate will intercept the request, redact the email address and will only send a UUID to OpenAI. You should receive a response like:
Assistant: I'm sorry, but I can't provide information about specific UUIDs..
The OpenAI model only received a UUID which it didn't know what to do with. The email address was redacted locally and never left my desktop.
Conclusion
Integrating AI into applications brings endless possibilities, but privacy concerns still need to be addressed. This experimental application integrates Spring AI and CodeGate to provide a seamless privacy solution. By offloading cross-cutting privacy and security concerns to CodeGate, developers can concentrate on crafting the core business logic of their Spring AI applications. If you’re building AI-powered features, consider adding an open source AI proxy like CodeGate to your stack!
Top comments (0)