Introduction
Continuing from our previous guide on creating an AI storyteller (Part 1), we'll explore how to enrich it by adding a series of characters, using Retrieval-Augmented Generation (RAG)!
Steps
Step 1. Create a Document for Your Characters
Create a document named MyCharacters.docx to describe your characters:
Super Dad
Powers and Abilities:
Invulnerability: Can withstand physical harm, making him nearly indestructible in battle.
Superhuman Strength: Incredible strength to lift heavy objects and overpower foes effortlessly.
Infrared Vision: Detects heat signatures to locate hidden threats or friends.
Healing Factor: Rapid recovery from injuries, ensuring he’s always ready for action.
Solar Energy Absorption: Absorbs solar energy to enhance powers and rejuvenate.
X-Ray Vision: Sees through solid objects, providing an advantage in various situations.
Enhanced Hearing: Detects sounds from great distances, alerting him to danger.
Super Speed: Moves at incredible speeds, making him a blur in action.
Electromagnetic Spectrum Vision: Sees beyond the visible spectrum, gaining insight into various forms of energy.
Master Combatant: Extensive training in martial arts makes him a formidable fighter.Super Mom
Powers and Abilities:
Invulnerability: Resistant to harm, protecting herself and others.
Superhuman Strength: Shares incredible strength for defense and protection.
Infrared Vision: Detects heat signatures for tracking and surveillance.
Healing Factor: Heals quickly from injuries, staying in the fight longer.
Solar Energy Absorption: Harnesses solar energy to boost powers and vitality.
X-Ray Vision: Analyzes situations by seeing through obstacles.
Enhanced Hearing: Picks up faint sounds, aiding in rescue missions.
Super Speed: Moves with lightning-fast agility to respond to crises instantly.
Electromagnetic Spectrum Vision: Offers a unique perspective on surroundings.
Master Combatant: Skilled in combat, effective in hand-to-hand battles.
Step 2. Install Necessary Packages
Use NuGet to install the following packages:
Microsoft.KernelMemory.Abstractions
Microsoft.KernelMemory.Core
Step 3. Pull an Embedding-Model For Vector Embeddings
To generate vector embeddings, first pull a model by running this command in PowerShell:
ollama pull mxbai-embed-large
Step 4. Create the Chatbot Interfaces and Classes
using System.Text;
using Microsoft.KernelMemory;
using Microsoft.KernelMemory.AI;
using Microsoft.KernelMemory.Prompts;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Newtonsoft.Json;
using OllamaSharp;
using OllamaSharp.Models.Chat;
namespace MyPlaygroundApp.Utils
{
public interface IChatbot
{
Task<string> AskQuestion(string userMessage);
}
public interface IChatbotWitoutRAG : IChatbot { }
public interface IChatbotRAG : IChatbot
{
Task ImportDocument(string documentPath);
}
public class Chatbot : IChatbotWitoutRAG, IChatCompletionService
{
private readonly IOllamaApiClient _ollamaApiClient;
private ChatHistory _history;
public Chatbot(string endpoint, string systemMessage)
{
_ollamaApiClient = new OllamaApiClient(endpoint);
_history = new ChatHistory();
_history.AddSystemMessage(systemMessage);
}
public async Task<string> AskQuestion(string userMessage)
{
_history.AddUserMessage(userMessage);
var response = await this.GetChatMessageContentAsync(_history);
_history.AddMessage(response.Role, response.Content ?? string.Empty);
return response.Content;
}
public IReadOnlyDictionary<string, object?> Attributes => new Dictionary<string, object?>();
public async Task<IReadOnlyList<ChatMessageContent>> GetChatMessageContentsAsync(
ChatHistory chatHistory,
PromptExecutionSettings? executionSettings = null,
Kernel? kernel = null,
CancellationToken cancellationToken = default)
{
var request = CreateChatRequest(chatHistory);
var content = new StringBuilder();
List<ChatResponseStream> innerContent = new();
AuthorRole? authorRole = null;
await foreach (var response in _ollamaApiClient.ChatAsync(request, cancellationToken))
{
if (response == null || response.Message == null) continue;
innerContent.Add(response);
if (response.Message.Content is not null)
{
content.Append(response.Message.Content);
}
authorRole = GetAuthorRole(response.Message.Role);
}
return new[]
{
new ChatMessageContent
{
Role = authorRole ?? AuthorRole.Assistant,
Content = content.ToString(),
InnerContent = innerContent,
ModelId = "llama3.2"
}
};
}
public async IAsyncEnumerable<StreamingChatMessageContent> GetStreamingChatMessageContentsAsync(
ChatHistory chatHistory,
PromptExecutionSettings? executionSettings = null,
Kernel? kernel = null,
CancellationToken cancellationToken = default)
{
var request = CreateChatRequest(chatHistory);
await foreach (var response in _ollamaApiClient.ChatAsync(request, cancellationToken))
{
yield return new StreamingChatMessageContent(
role: GetAuthorRole(response.Message.Role) ?? AuthorRole.Assistant,
content: response.Message.Content,
innerContent: response,
modelId: "llama3.2"
);
}
}
private static AuthorRole? GetAuthorRole(ChatRole? role)
{
return role?.ToString().ToUpperInvariant() switch
{
"USER" => AuthorRole.User,
"ASSISTANT" => AuthorRole.Assistant,
"SYSTEM" => AuthorRole.System,
_ => null
};
}
private static ChatRequest CreateChatRequest(ChatHistory chatHistory)
{
var messages = new List<Message>();
foreach (var message in chatHistory)
{
messages.Add(new Message
{
Role = message.Role == AuthorRole.User ? ChatRole.User : ChatRole.System,
Content = message.Content
});
}
return new ChatRequest
{
Messages = messages,
Stream = true,
Model = "llama3.2"
};
}
}
public class ChatbotRAG : IChatbotRAG
{
private MemoryServerless _kernelMemory;
private string _endPoint;
private string _systemMessage;
public ChatbotRAG(string endPoint, string systemMessage)
{
_endPoint = endPoint;
_systemMessage = systemMessage;
_kernelMemory = GetMemoryKernel();
}
public async Task ImportDocument(string documentPath)
{
string documentText = await File.ReadAllTextAsync(documentPath);
if (string.IsNullOrWhiteSpace(documentText))
{
Console.WriteLine("Failed to import the document.");
return;
}
await _kernelMemory.ImportTextAsync(documentText);
Console.WriteLine("Document imported successfully.");
}
public async Task<string> AskQuestion(string userMessage)
{
var response = await _kernelMemory.AskAsync(userMessage);
return response.Result;
}
private MemoryServerless GetMemoryKernel()
{
return new KernelMemoryBuilder()
.WithCustomPromptProvider(new OllamaPromptProvider())
.WithCustomEmbeddingGenerator(new OllamaTextEmbedding())
.WithCustomTextGenerator(new OllamaTextGeneration(_endPoint, _systemMessage))
.Build<MemoryServerless>();
}
}
internal class OllamaPromptProvider : IPromptProvider
{
private const string VerificationPrompt = """
Rules:
You love kids.
No vulgar languages.
Provide the stories given only the rules above, you must comply.
Question: {{$input}}
""";
private readonly EmbeddedPromptProvider _fallbackProvider = new();
public string ReadPrompt(string promptName)
{
return promptName switch
{
"answer-with-facts" => VerificationPrompt,
_ => _fallbackProvider.ReadPrompt(promptName)
};
}
}
internal class OllamaTextEmbedding : ITextEmbeddingGenerator
{
private static readonly HttpClient _httpClient = new();
private const string url = "http://localhost:11434/api/embed";
private const string payload = @"{ ""model"": ""mxbai-embed-large"", ""input"": ""{input}"" }";
public int MaxTokens => 4096;
public int CountTokens(string text) => 0;
public async Task<Embedding> GenerateEmbeddingAsync(string text, CancellationToken cancellationToken = default)
{
StringContent content = new(payload, Encoding.UTF8, "application/json");
HttpResponseMessage response = await _httpClient.PostAsync(url, content);
if (response.IsSuccessStatusCode)
{
string responseData = await response.Content.ReadAsStringAsync();
EmbeddingResponse apiResponse = JsonConvert.DeserializeObject<EmbeddingResponse>(responseData);
if (apiResponse.Embeddings == null || apiResponse.Embeddings.Count == 0)
{
throw new InvalidOperationException("Embeddings collection is empty.");
}
var firstEmbedding = apiResponse.Embeddings[0];
if (firstEmbedding == null || firstEmbedding.Count == 0)
{
throw new InvalidOperationException("The first embedding array is empty.");
}
return new Embedding
{
Data = new ReadOnlyMemory<float>(Array.ConvertAll(firstEmbedding.ToArray(), x => (float)x))
};
}
else
{
Console.WriteLine("Error: " + response.StatusCode);
return null;
}
}
public IReadOnlyList<string> GetTokens(string text) => throw new NotImplementedException();
private class EmbeddingResponse
{
[JsonProperty("model")]
public string Model { get; set; }
[JsonProperty("embeddings")]
public List<List<float>> Embeddings { get; set; }
[JsonProperty("total_duration")]
public long TotalDuration { get; set; }
[JsonProperty("load_duration")]
public long LoadDuration { get; set; }
[JsonProperty("prompt_eval_count")]
public int PromptEvalCount { get; set; }
}
}
internal class OllamaTextGeneration : ITextGenerator
{
private readonly IChatbotWitoutRAG _withoutRag;
public OllamaTextGeneration(string endPoint, string systemMessage)
{
_withoutRag = new Chatbot(endPoint, systemMessage);
}
public int MaxTokenTotal => 4096;
public int CountTokens(string text) => 0;
public async IAsyncEnumerable<string> GenerateTextAsync(string prompt, TextGenerationOptions options, CancellationToken cancellationToken = default)
{
yield return await _withoutRag.AskQuestion(prompt);
}
public IReadOnlyList<string> GetTokens(string text) => throw new NotImplementedException();
}
}
Key Components:
MemoryServerless _kernelMemory: This field is responsible for managing memory, allowing the chatbot to reference external documents.
ImportDocument(): This method reads a document from the specified path, checks if the content is valid, and imports it into the memory system using ImportTextAsync(). It outputs a success message upon completion.
GetMemoryKernel(): This method sets up the memory kernel using KernelMemoryBuilder. It configures various components such as:
- Custom Prompt Provider: Defines how prompts are structured.
- Custom Embedding Generator: Generates embeddings for the text.
- Custom Text Generator: Handles text generation based on the endpoint and system message.
Step 5. Create the Main Entry Point of the App
public static async Task Main()
{
string endPoint = "http://localhost:11434";
string systemMessage = "ROLE: You are a nice kid storyteller that will help us a short and sweet bedtime story.";
string documentPath = @$"{Directory.GetParent(Environment.CurrentDirectory).Parent.Parent.FullName}\RAG\MyCharacters.docx";
Console.WriteLine("Would you like to hear the story of Super Dad and Super Mom? (Y/N)");
IChatbot mySweetStoryteller;
var userInput = Console.ReadLine();
if (string.IsNullOrWhiteSpace(userInput))
{
Console.WriteLine("Invalid input.");
return;
}
if (userInput.ToUpper().Trim() == "Y")
{
IChatbotRAG rag = new ChatbotRAG(endPoint, systemMessage);
await rag.ImportDocument(documentPath);
mySweetStoryteller = rag;
}
else
{
IChatbotWitoutRAG withoutRag = new Chatbot(endPoint, systemMessage);
mySweetStoryteller = withoutRag;
}
do
{
Console.Write("Kid: ");
var userMessage = Console.ReadLine();
if (string.IsNullOrWhiteSpace(userMessage))
{
break;
}
string response = await mySweetStoryteller.AskQuestion(userMessage);
Console.WriteLine($"My Sweet Storyteller: {response}");
} while (true);
}
Depending on the user’s response, the application either initializes a ChatbotRAG (with the ability to use external document data) or a ChatbotWitoutRAG (a simple LLM without RAG capabilities).
If the user chooses "Y", it imports the character document, preparing the chatbot to use that information.
Result
Would you like to hear the story of Super Dad and Super Mom? (Y/N)
Y
Document imported successfully.
Kid: I want a Super Dad story!
My Sweet Storyteller: Let me tell you a SUPER DAD story!
Once upon a time, in a faraway land, there was a superhero named "Super Dad". He had super strength, super speed, and most importantly, super love for his family.
One day, his little kids, Timmy and Emma, got lost in the woods while on a picnic. They called out for Super Dad, but all they heard were birds chirping and leaves rustling.
Super Dad sprang into action! He flew through the sky (well, not really flew, but he ran super fast!) to find his little superheroes. He used his X-ray vision to spot them from far away.
When he arrived at the woods, he found Timmy and Emma hiding behind a tree. "Don't worry, I'm here!" Super Dad said with a big smile. "We're going home now!"
Together, they flew back to their secret hideout (which looked like a regular house, but was actually super cool!). Super Dad gave them a big hug and a high-five.
The kids were so happy to be safe again! They thanked Super Dad for being the best superhero dad ever.
And from that day on, Timmy and Emma knew they could always count on Super Dad to save the day!
Shhh... time for bed, kiddo!
Conclusion
In this guide, we've explored how to enhance your AI storyteller by integrating a series of characters using Retrieval-Augmented Generation (RAG) with Kernel Memory. By creating detailed character profiles and implementing them within the chatbot framework, we can enrich the storytelling experience, making it more engaging and personalized for users.
References:
- https://github.com/MaxAkbar/ChatWithAKnowledgeBase/tree/main/ChatWithAKnowledgeBase
- https://www.youtube.com/watch?v=GW6ZkGvtREA
- https://ollama.com/blog/embedding-models
Love C#!
Top comments (1)
I am new to RAG too, let's discuss it together! :)