Building an AI Powered Camera for David Bowie

Last year, when Universal Music Group announced they were taking their music off TikTok, I immediately started thinking about ways you could recreate some of the TikTok user experience on the web.

I began with the feed. Pretty simple to emulate the snappy vertical scrolling the app is known for. Then, I set my sights on the camera. Naturally, I’ve built many web based camera concepts over the years but TikTok is known for its AI filters. So, I wanted to try and develop a web based, AI powered camera. The outcome of that (unofficial) experiment is called Heroes Camera and it uses AI to face swap you and David Bowie so you can become a parody of the “Heroes” album cover. Honestly, my focus was less on AI and more on user experience. I wanted the app to feel simple, familiar, and magic. Become a hero, today:

www.heroes.camera

When I originally put this app up, it garnered a few 100 usages and a chuckle from friends and followers. Then, I noticed a swift uptick of uses… 1000, 10000, 50000…? What was happening? I dived into the analytics only to discover that the app went viral in Japan. As it turns out, David Bowie had an incredible relationship with Japan and Japanese style. A lot of this was thanks to his Japanese costume designer Kansai Yamamoto. Apparently the application really resonated with Japanese users and it ended up being covered on Yahoo Japan. Since then, I visited Japan for 3 months and can say I also have a deep affection for this country and its people. I can’t wait to go back. To all of you from Japan who used the app: ありがとうございます!

Read on to learn how this app came together in a day.

Development

This simple app is built on Nuxt and uses a face swap model from Replicate in order to swap David Bowie’s face with your own. All we need to do is capture a new photo of the user's face or allow them to submit an existing photo, then we’ll pass that photo alongside the Heroes cover to swap faces. Like most AI stuff, it is fearfully simple. Let’s start by getting a photo from the user.

Capture Photo or Image Upload

I recently discussed in a dev blog for The Black Keys how I integrate MediaDevices to access a user’s camera and capture a photo of them (or their surroundings.) Check out that dev blog for more info. In order to obtain an existing image from a user’s photo library or computer, we just need a file picker.

<input type=”file” accept=”image/*” @change=”fileSelected” />

And a bit of Javascript. I actually read the image and place it onto an input canvas as I’ll be passing a data url to the prediction function later.

// File selected
// —-------
function fileSelected(e) {
  // File
  const file = e.target.files[0]

  // Create url
  const url = URL.createObjectURL(file)

  // New image
  const img = new Image()

  // On load
  img.onload = () => {
    // Prepare image
    prepareImage(img, img.width, img.height)

  }

  // Image src
  img.src = url

}

// Prepare image
// —-------
function prepareImage(image, width, height) {
  // Create canvas
  const canvas = document.createElement(“canvas”)

  // Resize
  canvas.height = height
  canvas.width = width
  // Context
  const context = canvas.getContext(“2d”)

  // Draw image
  context.drawImage(image, 0, 0)

  // Update input
  input.value = canvas

}

Now that our image has been drawn on a canvas, we can send it through to Replicate so it may predict the face swap.

Using Replicate to Swap Faces

Let’s first set up a Nuxt server route which will run the face swap model on Replicate based on a target image (the Heroes cover) and a source image (the user’s photo.) We’ll integrate the official Replicate Node.js client in order to do this. This code is pretty straightforward but it is worth noting that I store my replicate config details using the Nuxt composable useRuntimeConfig.

import Replicate from “replicate”

export default defineEventHandler(async (event) => {
  // Body
  const body = await readBody(event)

  // Config
  const config = useRuntimeConfig()

  // Client
  const replicate = new Replicate({
    auth: config.replicateApiToken
  })

  // Model
  const model = config.replicateModel

  // Run model
  const output = await replicate.run(model, {
    input: {
      target_image: “heroes.jpg”,
      source_image: body.source_image
    }
  })

  // Return
  return output

})

Finally, in order to call this from the client, we merely need to post the input photo as a Base64 to this new API endpoint.

// Predict
// —-------
async function predict() {
  // Source image as base64
  const sourceImage = input.value.toDataURL(“image/jpeg”)

  // Predict
  const data = await $fetch(“/api/predict”, {
    method: “POST”,
    body: {
      source_image: sourceImage
    }
  })

 // Image
 // data.image

})

We can now decide how we want to display the data.image result back to our user. You can simply throw it up in an <img> tag or generate a reveal video on the fly like I’ve done using Pixi.JS and MediaRecorder. Perhaps a topic for another dev blog.

Conclusion

The web can literally do anything these days and it is a free sandbox to explore and challenge the artist/fan relationship. When it comes to relying on social platforms like TikTok, I give all artists the same advice:

Use everyone.
Trust no one.

Social platforms come and go and typically leave a lot of carnage in their wake. It’s important to grab attention where you can but do not be surprised if that community you’re building disappears overnight. Focus as much attention on channels you own, like email newsletter, sms, past purchasers of your merch store, etc. Only then, will we be kings and queens.