Building a Short Story Assistant using Cloudflare AI

This is a submission for the Cloudflare AI Challenge.

What I Built

Short Stories Assistant is a simple tool that helps writers generate short stories of about 2000 words using Text Generation models.

An example of story generation flow would be:

• Providing a prompt using an audio file (optional) ;
• Generation a story using one of the available models (e.g. Mistral 7b) ;
• Editing and saving the story (this will automatically generate the story title, summary, and cover image) ;
• Improving the story title and summary ;
• View previously generated stories.

The application acts as a human assistant, and, therefore, allows the user to customize the story, the title, and the summary at any step of the flow.

Demo

The application is live at https://demo.short-stories-assistant.pages.dev/

My Code

Code available at https://github.com/jonathanarnault/short-stories-assistant

Technologies used:

• Remix ;
• TailwindCSS and DaisyUI ;
• Cloudflare Pages ;
• Cloudflare Workers AI ;
• IndexedDB.

Journey

I never used Cloudflare workers before this challenge, but I was looking to test it on a side project since it seemed to be a powerful IaaS. This challenge was an opportunity to test the service on a simple project.

Since this challenge was focused on Workers AI, I wanted to build a Proof of Concept with basic features for a Short Story Assistant. Thus, I decided to only store data client-side and focus on using as many task types as possible that can fit in the app.

For the Short Story Assistant, I used the following task types:

• Speech Recognition (whisper), for prompting the service using audio files ;
• Text Generation (mistral-7b-instruct-v0.1, falcon-7b-instruct, llama-2-13b-chat-awq), for the story generation ;
• Text Summarization (bart-large-cnn) for the story abstract and title ;
• Text to Image (dreamshaper-8-lcm) for the story cover image generation.

During the development, I was impressed by the ease of integration of Workers AI inside a project. I was able in a few hours to implement a fully functional Proof of Concept with many task types. The Cloudflare SDK makes it easy to call an AI service and get results back.

Despite the ease of integration, I faced some technical challenges with the AI services integration:

• First of all, when I tried to implement text generation, the service usually timed out with partial text. I solved this issue using an EventSource to stream data and avoid timeout ;
• The second challenge that I experienced was to find a text generation model that does not ignore my prompts while following system prompts. While Mistral 7b was working well with my prompt, I had to skip Llama 2 7b and Phi-2 as they were not answering my prompts correctly ;
• Finally, while the Remix integration with Cloudflare Pages is great, especially the deployment, I did not find an easy way to call Workers AI in dev mode. To solve this, I simply ran the application in preview mode locally to test the integration.

In conclusion, this was a fun project to implement and I recommend using Cloudflare Workers AI in projects that do not need custom models. I will also probably test more features of Cloudflare Workers in the future as they provide a great set of services to build web applications.