DEV Community

Thai Pangsakulyanont

Posted on Nov 3, 2020 • Updated on Dec 16, 2020 • Originally published at notes.dt.in.th

Building a personal but multi-tenant web page screenshotting service with Puppeteer and Vercel

#puppeteer #vercel

In the past 2 years I found myself having to install and configure Puppeteer to capture webpage screenshots on so many occasions.

So I thought it would be great to have a generic API that captures webpage screenshots that I can reuse across multiple projects. The existence of serverless platforms like Vercel made it all the more easier to do this, even for personal projects.

The need for screenshotting web pages

I regularly find need to programmatically take screenshots of web pages.

In each of these use cases I installed Puppeteer and wrote similar code: Go to the web page, take a screenshot, output the image. But Puppeteer is quite a heavy dependency, weighting over 100 MB in node_modules.

In cases where the code is used in dynamic sites, I would had to set up a Google Cloud function or an AWS Lambda function for that use cases.

Wouldn’t it be great if, instead of setting up Puppeteer every time, there is an API that I can immediately use? Well, there is a plenty of ~~existing~~ offerings, but most of them are paid and the free ones went down. Even some of the paid ones went down (the links that are struck out are now dead).

Since I wanted to re-use it across multiple projects, spanning multiple years to come, I want some degree of control. The service should:

Run on my own domain name. I don't want to have to go through all my projects and migrate to a new service, just because the old service is sunsetted.
Be affordable or free. It’s for personal use and is non-commercial. I don’t want to spend too much money on it.
Personal yet multi-tenant. I may use it on a project that is shared with others. In a rare case that I need to revoke an access to one project, it should not disrupt other projects. Therefore, it should support multiple keys.
Secure. Others should not be able to use the service to screenshot arbitrary web pages without my permission.
Single-shot API. Service consumers should be able to construct the image URL without having to make any extra API requests.

Introducing personal-puppeteer

So this is what I created. Here’s how it works:

First, let’s say I want to generate a social card image for the URL at https://capture.the.spacet.me/. As an API consumer, I would:

Generate a request.
Cryptographically-sign the request into a JWT and construct an image URL.
Send the image URL to client (in a <meta property="og:image"> tag).

The browser would then make a request to the service, which would, on request, take a screenshot
…and return the image back to the browser.

The service maintains a list of tenants which are allowed to use the service. This allows the service to be reused in multiple projects without them having to share the same secret key.

Under the hood

I was able to quickly build the first version of this service thanks to Vercel’s Edge Network and Serverless Functions.

The first time a request is received:

It would enter Vercel’s network.
Since this was the first time the request was processed, it would be a cache MISS.
Vercel would then call the underlying serverless function.
Which in turn validates the request and captures a screenshot of the webpage.

The image is returned from the serverless function with a very aggressive caching header.
Vercel would return the response and put it into its cache.

The next time the request is received, it would be served by Vercel’s cache.

Use cases unlocked

Automatic social image generation for all my web projects

If I create a web project or a blog post, but don’t want to spend the time crafting an og:image for each page, I can just use personal-puppeteer to generate a default og:image from the webpage’s screenshot.

Now when I share my article on Facebook, people would see the webpage’s screenshot.

Although it may not look as good as a handcrafted image, I think this is still way better than using a generic image or a profile picture as a default preview image.

So far, I am doing this for:

https://dt.in.th/ — My personal website
https://notes.dt.in.th/ — This website where I publish notes
https://capture.the.spacet.me/ — personal-puppeteer’s landing page

Sending procedurally generated images in chat rooms

By using data: URLs, an HTML page can be embedded in the JWT.
This can be useful when I want to create a chat bot that can generate infographics.

Adding web page screenshot to `README.md`

The URL can be embedded into other websites to provide an auto-updating screenshot.

Open source

This project is open source, so you can run your own instance too!

dtinth / personal-puppeteer

A personal web page screenshotting service. Basically, it exposes an API that I can use to generate screenshot of any URL.

personal-puppeteer

This is a personal web page screenshotting service. Basically, it exposes an API that I can use to generate screenshot of any URL By personal I mean that only I can use it. It is secured with JWT. But it is open source, and you can run your own instance.

It is deployed to Vercel and is inspired by Vercel’s Open Graph Image as a Service The main difference is that this service is designed to be reusable across many use cases. It can capture arbitrary URLs and run arbitrary code. It is multi-tenant. I can reuse this service without having to share secrets.

→ Read the project introduction post for more info.