Table of Contents
Preface
For those who didn't read previous blogs from the OSD700 series, I highly recommend you to read last two blogs, in order to understand what's written here.
TL;DR
In previous blog I compared file attachment UIs in different messaging and LLM applications. Why I did this? Answer is simple, to implement it for ChatCraft.
Introduction
During previous week me and other maintainers of ChatCraft decided to implement UI for the file attachments.
Full discussion you may find here:
Move `Attach File` icon next to `Start Recording` button
#794
As a user, I want to access Attach Files option without pressing Options button. Moreover, all modern AI chats provide this option next to Text Input Field. It would improve UX. I could move it next to Start Recording button.
Claude
ChatGPT
This new UI feature is follow-up for other new feature implemented by professor Dave. His feature integrates DuckDB to ChatCraft which allows users to manipulate files using chat. Therefore, we need a UI that would help users, first of all, understand and then use it.
Professor's new feature consists of four phases that you may find here:
RFC: Files Table and DuckDB
#788
We've been adding DuckDB support to ChatCraft and it has some powerful features we'd like to use with files. For example, being able to use SQL to query CSV, JSON, or Excel files, do full-text search of large text files, etc. DuckDB-wasm also has its own concept of a file system, which we'd like to connect to our current files feature.
Our current file feature mostly lives in src/hooks/use-file-import.tsx. Users can copy/paste, drag-and-drop, or click Options > Attach Files... to include file content in messages. In other words, file contents (text or extracted text, images as base64 encoded URLs) are included in messages to be sent to the LLM. This works well for small files.
Now that we have DuckDB and the concept of Tables, we want to add Files as well. Files would be separate from messages, and would be stored in binary form in Dexie.
Add a new Table to Dexie, files, with the following layout:
interface ChatCraftFile {
id: string; // unique hash of the file contents, for deduping
name: string; // original file name
type: string; // mime-type of the file (e.g., "text/csv")
size: number; // size of file in bytes
content: Blob; // binary content of file
text?: string; // extracted text of file, base64 encoded version, etc
created: Date; // when the file was created
expires: Date; // when the file can be deleted
metadata?: Record<string, unknown> // extra metadata
}
Modify src/hooks/use-file-import.tsx as follows:
- always store imported files in the
filestable - add logic to decide when to include a file's contents in the message (e.g., based on file type, size). For example, a small PDF would get added as a message but a large one would not
- modify how images work to add as Markdown to human messages when including vs. what we do now with image URLs
- larger files can become available for RAG, DuckDB queries
- Periodic file expiry (e.g., timeout that runs hourly or something and looks at expiry field to compare to current date/time?)
We need to expose files in the current chat (and possibly all files?) to the DuckDB virtual filesystem. This would allow us to do SQL queries against the file data, for example:
- full-text search (RAG) which gets included in chats
- sql queries to obtain info from CSV, Excel, etc
- sql queries to join across multiple files
It would be nice if DuckDB could also write to the Dexie files table, so you can safe the results of things back into the chat.
- Add a files panel/section to show the attached files. This might live above the prompt area, or be available by clicking an icon (e.g., on mobile)
- Allow deleting files
- Allow downloading files (e.g., results of queries, artifacts created by LLM)
- Maybe show all code blocks or other generated source as files in files that you can use or download
- Add preferences for when to include files in messages (e.g., file types or file sizes), with good defaults
- Add preferences/controls for expiring files so they don't take up too much space
- Figure out how to indicate that we want RAG to happen on attached files
- Maybe add some new /slash commands for simple ways to interact with the files in the current chat (e.g.,
/embed <filename>,/delete <filename>,/download <filename>)
We need to decide how to deal with existing imageUrls[] (i.e., base64 encoded image strings) in messages table when we do the next db migration. The table looks like this:
export type ChatCraftMessageTable = {
id: string;
date: Date;
chatId: string;
type: MessageType;
model?: string;
user?: User;
func?: FunctionCallParams | FunctionCallResult;
text: string;
imageUrls?: string[]; // <-- these are big
versions?: { id: string; date: Date; model: string; text: string; imageUrls?: string[] }[];
};
We have a few options:
-
Delete Option
- Pro: Simplest, cleanest approach
- Pro: Reduces database size immediately
- Con: Destroys user data without consent
- Con: No way to recover if needed
-
Archive Option
- Pro: Preserves user data
- Pro: Gives users control over their data
- Con: More complex implementation
- Con: Requires additional UI work
- Con: Delays cleanup of large data
- Con: Not clear if users would know how to recover images from this table
-
Migrate to Files Option
- Pro: Preserves recent images in new format
- Con: Complex async migration
- Con: we don't have filenames
- Con: Risk of migration failure (e.g., user refreshes "hanging" tab)
- Con: Performance impact during migration
- Con: No clear way to handle partial failures
This is a complex change that needs to get done in pieces over many smaller issues/PRs.
-
Phase 1: Basic Files Support
- Add files table
- Implement basic file storage
- Add simple UI for file management
- Choose and implement migration strategy
-
Phase 2: DuckDB Integration
- Connect files to DuckDB filesystem
- Implement query support
- Add basic file querying UI
-
Phase 3: Enhanced Features
- Add RAG support
- Implement file expiry
- Add advanced UI features
- Add slash commands
-
Phase 4: Optimization
- Performance improvements
- Storage optimization
- UI/UX refinements
- Additional file type support
Currently, we are on the phase number one which means that it's only beginning of something huuuuuge, and I am really excited!
UI Implementation
Once I received all requirements in the issue conversation, I proceed to grind.
For the reason, that I am not a "UI guy", I felt very uncomfortable, kinda left comfort zone. It made me understand that I am on the right track!
I divided my implementation onto small stages:
Stage 1:
- Implement raw version that would showcase the whole idea.
- Receive feedback.
Stage 2:
- Based on feedback re-implement what was done with better UI/UX.
- Receive more feedback.
Stage 3:
- Make more changes.
- Receive some more feedback.
I don't want to continue with next stages because every open-source developer knows that communication with maintainers feels like a ping-pong: Contributor commits, receives feedback, makes changes and then receives more feedback, and makes more changes :D
Pull Request
This blog would be endless if I described every single stage that I had to land. However, I decided to walk through first and last stages, so readers could feel the contrast between my raw implementation, and the final one.
Ping-Pong - Round 1 (First Raw Version)
Before I opened a PR, I needed something to be pushed in the new branch. I decided to create a first version of File Attachments UI that includes raw implementation. What does it mean?
Since I started contributing to the open-source community, I developed a plan that would help me to accomplish any work I receive.
Once I receive the requirements, I start implementing things as I understand/see them. I start looking at things from the owner's perspective. Eventually, I open the draft pull-request where I try to explain everything how I understood it.
Therefore, I receive a ton of the change requests, and start working based on them. This cycle repeats itself as long as possible, I call it "ping-pong" until it wasn't merged.
My first UI implementation included:
- Paperclip IconButton with hover effect that was triggering
+icon appearance out of the blue =) - Paperclip icon was disabled if no files attached, but
+icon enabled. -
+icon didn't have a functionality... - Modal window had FileIcons with file names and sizes
- Files could be deleted by pressing trash icon
- Files could be downloaded by pressing download icon
Preview:
I opened a draft Pull Request where maintainers were leaving their comments and change requests:
[UI] File Attachment Added
#804
Closes #794
This PR improves UX in terms of file management. Previously user was only able to see the file in the PromptForm, but now he is able to use paperclip icon to see all files attached in modal window, and manage them:
- Attach files... (Same as in OptionsButton)
- Delete (using
removeFilefromsrc/lib/fs.ts) - Download (using
downloadFilefromsrc/lib/fs.ts)
If user never attached the file(s) paperclip will trigger file attachment process without opening modal window.
Ping-Pong - Final Round (Result)
There were a lot of change requests that I was accepting as new challenges. I loved it!!! Eventually, I came up with the final solution that differs from the first state of the PR.
It is complete version of the UI that consists of all changes requested by maintainers. To be honest, I never thought that I would be able to build UI like this, but I am grateful to professor that he guided me through the entire process.
Final solution includes:
Paperclip icon that triggers:
- Modal window, if number of files attached equals more than zerooo.
- File system to choose file(s) to be attached.
Modal Window:
- Header shows number of files attached.
- All files attached as the cards.
- Footer buttons
Add FilesandRemove All.
File Icon:
- Download button (top-left corner)
-
Xbutton or remove button (top-right corner) - Editable file name textfield (just click on file name)
Preview:
As a result, I have implemented new feature for the ChatCraft. It really makes me happy to work on this project. In advance, expect to read a ton of posts regarding this cool project, I want to spend as much time as possible!
Conclusion
Working on this project makes me feel complete because I learn new language which is TypeScript, I work on something meaningful, I feel myself a crucial part of this project, and I get out of my comfort zone which will result in progress!
For the next sprint I expect to continue my work on ChatCraft. Most probably, I am going to work on follow-ups since it's just phase 1 of DuckDB integration.
That's it for today. Enjoy the process!
Top comments (0)