Running Open Source LLMs in Popular AI Clients with Featherless: A Complete Guide

Table of contents

1. Introduction
2. Getting started with Featherless
3. Role-playing client integration
   1. SillyTavern
   2. WyvernChat
   3. VenusAI
   4. JanitorAI
   5. anime.gf
4. Frontend chat client integration
   1. Featherless Phoenix
   2. Typing Mind
5. Security best practices
6. Community & Support

Introduction

Remember the excitement of discovering powerful open-source language models like the latest Qwen or LLama 3 AI models, only to face the daunting challenge of actually running them?

You’re not alone. While these models offer amazing capabilities, deploying them efficiently has been a significant hurdle for developers and enthusiasts alike. Today, we’re changing that with Featherless. In this guide, you’ll learn how to integrate the latest cutting-edge AI models into your favorite chat clients - whether you’re building an ai agent, a coding assistant, roleplaying, or setting up a general chat interface.

I’ll walk you through practical, step-by-step instructions for popular platforms like SillyTavern, WyvernChat and Typing Mind showing you how to leverage our serverless infrastructure. No GPU or server management required, no complex deployments - just powerful language models ready to use in the tools you already know and love.

Getting started with Featherless

Featherless is your gateway to using powerful open-source language models in your favorite applications. As a serverless inference platform, we make it simple to access the latest models without managing any infrastructure.

OpenAI compatibility

Our API is fully OpenAI-compatible. This means any application that works with OpenAI can be easily reconfigured to use Featherless.

Quick Setup API

1. Sign up for a Featherless account
2. Subscribe to a plan that fits your needs
3. Navigate to the API Keys section in your dashboard
4. Create a new API Key

That’s it! You’re ready to use this API Key in your preferred chat client

Choose your plan

Select the plan that fits your needs:

🌱 Featherless Basic ($10/month)

• All models up to 15B parameters
• 2 concurrent requests
• Unlimited monthly usage

⭐ Featherless Premium ($25/month)

• All models up to 72B parameters
• Everything in Basic
• Perfect for power users

🚀 Featherless Scale ($75 per unit/month)

• Everything in Premium
• 2x Premium or 6x Basic model concurrency per unit
• Host private models from Hugging Face

🔒 Privacy First: We never log chats, prompts, or completions. Your conversations stay private.

Ready to begin?

Head to the specific integration guide for your preferred client, or join our Discord for support.

Role-playing client integration

Having spent countless hours lost in AI roleplay conversations, let me tell you - there’s nothing quite like that moment when your character truly comes alive. Whether it’s the witty jokes that feel spontaneous, or those surprising, deep exchanges that make you forget you’re talking to an AI.

From creating more complex characters with a deep lore to vibrant anime personalities, what I’ve learned is that finding the right model isn’t just a technical choice - it’s about finding a perfect way to show off the identity and personality behind your character. That’s why I’m so excited about our growing catalog of open-source models. Each one brings something special and different to your character interactions and creative writing, I’ve seen incredible roleplay scenes emerge from unexpected model choices.

Let me now walk you through integrating Featherless with some of your favorite roleplay clients.

SillyTavern

With SillyTavern it’s pretty easy to create a connection to Featherless. Simply click on the plug icon at the top and make the following selections:

1. API: Chat Completion
2. Chat Completion source: Custom (OpenAI-compatible)
3. Custom Endpoint: https://api.featherless.ai/v1
4. Custom API Key: Your Featherless API key
5. Enter a Model ID: A model chosen from our model catalog (e.g. meta-llama/Meta-Llama-3.1-8B-Instruct)
6. Connect!

Once connected, you’ll see a green status indicator after which you’ll be able to send a message to your characters to ensure everything is working properly. You should receive a response within seconds.

WyvernChat

WyvernChat offers native Featherless integration right out of the box. The built-in support means you’ll spend less time configuring and more time chatting. WyvernChat provides a streamlined setup process. Whether you’re new to AI chat platforms or migrating from another client, you’ll appreciate how seamlessly Featherless meshes with WyvernChat’s clean interface. All you need is your Featherless API Key and following steps:

1. Head on over to https://app.wyvern.chat/
2. At the bottom right of the page click on the plug icon
3. Click on ‘+ Add connection’
4. Select ‘Featherless’ from the Type dropdown
5. Password (API Key)*: Your Featherless API key
6. Select a model from the list at the bottom
7. Scroll down and press ‘Create’

Once your connection is created head over to any of your character chats, then at the bottom right under ‘Settings’ deselect ‘Free Queue’ (this ensures you’re using your Featherless connection). Then simply choose your model from the Connection dropdown. Switching between models is as simple as creating an extra connection. You’ll know everything is working when your character responds using your chosen model - typically within a few seconds.

Venus AI

VenusChat supports OpenAI-compatible APIs out of the box, making our connection process quick and straightforward.

To connect your VenusChat with Featherless, head over to any character chat:

1. Click on the Gear icon on the top right
2. Go to “AI Model Settings”
3. Choose ‘Open AI’ under select an AI Model
4. Select “Reverse Proxy”
5. Pick a model from our model catalog, copy the complete url
6. Paste the url under ‘Open API Reverse Proxy (e.g. https://featherless.ai/models/mistralai/Mistral-Nemo-Instruct-2407)
7. Enter your Featherless API Key under ‘Reverse Proxy Key’

JanitorAI

Head over to chat on any character on JanitorAI and let’s get you set up with just a few steps:

1. Open the dropdown in the topright corner and select ‘API Settings’
2. Go to ‘Proxy’
3. Pick a model from our model catalog
4. Under ‘Model’ choose custom and enter the models name (e.g. meta-llama/Meta-Llama-3.1-8B-Instruct)
5. Other API/proxy URL: https://api.featherless.ai/v1/chat/completions
6. API Key: Your Featherless API Key
7. Check API Key/Model to see everything is working
8. Scroll down and Save Settings

Once you’ve saved your settings, congratulations - you can now chat with your character with any of our compatible models and switching between them is as simple as repeating steps 3-4. Feel free to experiment with different models to find the perfect fit for each character.

anime.gf

Bringing your favorite anime.gf characters to life with Featherless is straightforward. Enhance your interactions by making use of our diverse model catalog. Let’s get you setup with just a few simple steps.

1. Click on the cog in the top right of your screen
2. Head on over to ‘A.I. Settings’ and click on ‘provider’
3. Under API Provider select ‘Proxy’
4. API Key: Your Featherless API Key
5. Base URL: https://api.featherless.ai/v1/
6. Model: Choose a model from our model catalog (e.g. anthracite-org/magnum-v4-72b)
7. Save

Great! Your anime.gf character is now powered by Featherless. Try sending a message to see the integration in action. Feel free to experiment with different models from our catalog to find the perfect match for each character - switching models is as easy as changing the model in your settings.

What’s next?

With your favorite roleplaying client connected to Featherless you’re ready to experiment with a variety of models from our catalog, join our discord to share your experiences and get model recommendations!

Frontend chat client integration

I’ve always find something satisfying about a clean chat interface - it’s like having a dedicated thinking space where you can have a focused conversation. Whether I’m exploring new concepts, in a deep coding session or just brainstorming ideas - platforms like Typing Mind and our own Phoenix have become essential companions. Let me show you how to set up these tools that give you access to our entire model catalog as I think you’ll find them as invaluable as I do.

Featherless Phoenix

If you’re looking for the most straightforward way to start chatting with our models, look no further than Featherless Phoenix. As our native chat interface, it requires zero additional setup - simply login with your Featherless account, choose your preferred model from the menu in the top left corner and start chatting. It’s the perfect starting point for exploring our model catalog and finding the right language model for your needs.

Typing Mind

Typing mind, as a Chat UI frontend allows you to use AI models from our whole catalog. Featherless integration is as easy as going to ‘Models’, then clicking on ‘+ Add Custom Model’ on the top right of your screen followed by these quick steps:

1. Choosing ‘OpenAI Compatible API’ as API Type
2. Endpoint: https://api.featherless.ai/v1/chat/completions
3. Model ID: Pick a model from our model catalog (e.g. mistralai/Mistral-Nemo-Instruct-2407)
4. Choosing a context length (which you can find on the model’s page)
5. Add a custom header with it’s key ‘authorization’ and the key will be Bearer <YOUR_FEATHERLESS_API_KEY>
6. Add a custom body params with a number param `max_tokens' followed by any amount up to your context length. This will be the length of your response.
7. Lastly press Test and if everything went well you can now 'Add Model' That’s it - Your Typing Mind interface is now connected to Featherless! Try chatting to verify if everything is working properly. Response times will vary by model, but you should typically see results within a few seconds.

Security best practices

Your Featherless API Key is your secure gateway to our services and all the models you love - protecting it is crucial for your account’s security and data privacy. Never share your keys publicly and we recommend creating separate API keys for different applications. If you suspect your key might have been exposed, rotate it immediately through your Featherless dashboard.

Community & Support

Our growing community of developers, enthusiasts, and AI practitioners is here to help you get the most out of Featherless:

• Join our Discord community to connect with other users
• Share your experiences with us!
• Get model recommendations for your specific use case
• Stay updated on the latest models that get added

As the world of AI is evolving rapidly, Featherless is committed to evolving with it. As new models emerge and capabilities expand, we’re working to ensure you have seamless access and integration to all the latest advancements. Our mission to make all AI models available for serverless inference remains unchanged.

We’re excited to see what you all create with Featherless. Whether you’re creating engaging characters for roleplay or exploring new applications we haven’t even imagined yet. Ready to get started? Head over to https://featherless.ai/ to create an account, or join our Discord community to connect with other enthusiasts.