Deploying my Astro + Turso + Drizzle project to Cloudflare Pages

Background

I wanted to build a project which utilized map in some way using deck.gl. So I thought what better way than to visualize the users who are visiting the project itself. I knew I would need visitor's location (rough/accurate it did not matter to me that much) and I was aware that Cloudflare allows you to enable location headers from which I could extract the data which I wanted.

This post won't go into detail of the project itself as such. Rather the issues and the flow itself for deploying the app. Below are the links to the project and the repository:

MapViz

flashblaze/map

Tech stack

1. I knew I wanted a fullstack framework to access the headers from the app itself. As such, I went with Astro since I was familiar with it and the docs also explain in great detail on how to deploy Astro site on Cloudflare Pages.
2. For rendering the map, I went with Preact and deck.gl.
3. For storing the location data, I could have gone with Cloudflare's own D1, but I went with Turso and drizzle ORM as my ORM.

Skill issue

drizzle-kit: import.meta is not available

This was my drizzle.config.ts and after running the drizzle-kit generate command I ran into this this error: "import.meta" is not available with the "cjs" output format and will be empty

import { defineConfig } from "drizzle-kit";

export default defineConfig({
  schema: "./src/db/schema.ts",
  out: "./migrations",
  dialect: "turso",
  dbCredentials: {
    url: import.meta.env.TURSO_CONNECTION_URL ?? "",
    authToken: import.meta.env.TURSO_AUTH_TOKEN ?? "",
  },
});

Based on the error message, it felt like it needs the environment variables using process.env format. As such I had to update the config like so:

import { defineConfig } from "drizzle-kit";
import { config } from "dotenv";

// drizzle-kit does not support import.meta.env, so we need to load the .env file manually.
config({ path: "../.env" });

export default defineConfig({
  schema: "./src/db/schema.ts",
  out: "./migrations",
  dialect: "turso",
  dbCredentials: {
    url: process.env.TURSO_CONNECTION_URL ?? "",
    authToken: process.env.TURSO_AUTH_TOKEN ?? "",
  },
});

which fixed the issue.

Using drizzle with Astro

To use drizzle with Astro, I had to export a function in which I was passing the environment variables to the drizzle config. These are the files in question:

src/db/index.ts

import { drizzle } from "drizzle-orm/libsql";
import { createClient } from "@libsql/client";

export function createDB(env: App.Locals["env"]) {
  const client = createClient({
    url: env.TURSO_CONNECTION_URL,
    authToken: env.TURSO_AUTH_TOKEN,
  });

  return drizzle(client);
}

src/env.d.ts

/// <reference path="../.astro/types.d.ts" />

type Runtime = import("@astrojs/cloudflare").Runtime<Env>;

declare namespace App {
  interface Locals extends Runtime {
    env: {
      TURSO_CONNECTION_URL: string;
      TURSO_AUTH_TOKEN: string;
    };
  }
}

interface ImportMetaEnv {
  readonly TURSO_CONNECTION_URL: string;
  readonly TURSO_AUTH_TOKEN: string;
}

interface ImportMeta {
  readonly env: ImportMetaEnv;
}

src/pages/index.astro

---
import { createDB } from "../db";

const db = createDB(Astro.locals.env);
---

Preact compat was not working as expected with deck.gl

I followed the docs and installed the integration using pnpm astro add preact command. Docs also suggested on adding overrides for compat here. I followed the pnpm related docs and added this to my package.json

  "pnpm": {
    "overrides": {
        "react": "npm:@preact/compat@latest",
        "react-dom": "npm:@preact/compat@latest"
    }
  }

However, I ran into these errors

WARN Issues with peer dependencies found
.
├─┬ deck.gl 9.0.33
│ ├── ✕ missing peer react@npm:@preact/compat@latest
│ ├── ✕ missing peer react-dom@npm:@preact/compat@latest
│ └─┬ @deck.gl/react 9.0.33
│   ├── ✕ missing peer react@npm:@preact/compat@latest
│   └── ✕ missing peer react-dom@npm:@preact/compat@latest
├─┬ drizzle-orm 0.34.1
│ └── ✕ missing peer react@npm:@preact/compat@latest
└─┬ react-map-gl 7.1.7
  ├── ✕ missing peer react@npm:@preact/compat@latest
  └── ✕ missing peer react-dom@npm:@preact/compat@latest
✕ Conflicting peer dependencies:
  react      react-dom  

I even tried installing react and react-dom which felt unnecessary but wanted to try it nonetheless. Still the issue persisted.

So what was the solution?

Removing the overrides altogether.

It builds on my machine but not on Cloudflare via GitHub 😔

Try as I may, if I deploy my app using wrangler deploy it used to work, however as soon as I was pushing anything to GitHub, the build used to fail. Reading through the logs and searching online, I figured out that Cloudflare was building my app using Node v18 and pnpm v8 while I was using Node v20 and pnpm v9.

Setting NODE_VERSION to 20.17.0 and PNPM_VERSION to 9.9.0 under Variables and Secrets in the dashboard fixed the build issue.

The website takes too long to load

As of this writing, the website takes about 10 seconds to load when loaded for the first time. I think this is due to the cold start of the Turso DB. One solution is to keep the DB "hot" by making API calls at certain intervals which I don't plan on doing. So I have yet to find an actual solution for this 🥲.

What's next?

Right now, I'm in the process of migrating to Cloudflare D1. Below are the D1 specific links:

D1 MapViz

flashblaze/map (D1 branch)

I will update this section once I have completely migrated and tested the app along with the challenges faced during migration as well, lol. I also need to work on a good CI/CD pipeline using GitHub Actions since the migrations are currently being run manually.