Stefano Magni

Posted on May 26, 2021 • Edited on Jun 4, 2021

Migrating a 150K LOC codebase to Vite and ESBuild: How? (Part 2/3)

#vitejs #esbuild #react #webpack

The meticulous work behind migrating our codebase to Vite, helpful to fail as soon as possible or to succeed the most brilliant way.

This is part of a three-article series about migrating our React+TypeScript codebase from Webpack to Vite. Part 1 is about why we decided to migrate, Part 3 is about post-mortem considerations.

Migrating the codebase

I could summarize the migration with the following steps:

Compatibility: includes studying Vite, playing with it, and simulating our scenario outside the actual codebase.
Feasibility: does our project works under Vite? Let’s migrate the codebase in the fastest way possible.
Benchmarking: is Vite worthwhile? Are our early assumptions correct?
Reproducibility: repeating the migration without messing up the codebase and reducing the required changes.
Stability: being sure that ESLint, TypeScript, and the tests are happy with the updated codebase for Vite and Webpack.
Automation: preparing the Codemods necessary to jump on Vite automatically.
Migration: reaping the benefits of the previous steps.
Collecting feedbacks: does the team like it? What are the limitations once using it regularly?

In the following chapters, I’m going to deepen each step.

1. Compatibility

Probably the easiest step. Vite’s documentation is pretty concise and clear, and you don’t need anything more to start playing with Vite. My goal was to get familiar with the tool and to check out if and how Vite works well with the critical aspects of our project that are:

TypeScript with custom configuration
TypeScript aliases
Import/export types
named exports
aggregated exports
web workers with internal state
Comlink (used to communicate between workers)
React Fast Refresh
Building the project
Browser compatibility
React 17’s JSX transform compatibility

Quick and dirty, just creating a starter project through npm init @vitejs/app, experimenting with it, simulating a scenario with all the abovementioned options, and playing with it.

Honestly, I expected more troubles, but all went fine. The first impact with Vite is super positive 😊.

2. Feasibility

Just one and clear goal for this step: adding Vite to our codebase, no matter how. Seriously, no matter if I break TypeScript, ESLint, .env variables, and the tests, I only want to know if there are technicalities that prevent us from moving the project to Vite.

The reason behind this crazy and blind process is not succeeding the most elegant way but failing as soon as possible. With the least amount of work, I must know if we can move our project to Vite or not.

After reading even the ESBuild’s docs, the most impacting changes for us are

Adding three more settings to the TypeScript configuration (impacts a lot of imports and prevent from using Enums)

"isolatedModules": true,
"esModuleInterop": true,
"allowSyntheticDefaultImports": true

ESBuild requires the first two ones. You can read why in its documentation. Please remember that ESBuild removes type annotations without validating them. allowSyntheticDefaultImports isn’t mandatory but allows us to keep the codebase compatible with both Vite and Webpack (more on this later)

Updating the TypeScript’s aliases: no more @foo aliases but /@foo or @/foo, otherwise, Vite looks for the imported aliases in the node_modules directory.

resolve: {
  alias: {
    '@/defaultIntlV2Messages': '/locales/en/v2.json',
    '@/defaultIntlV3Messages': '/locales/en/v3.json',
    '@/components': '/src/components',
    '@/intl': '/src/intl/index.ts',
    '@/atoms': '/src/atoms/index.ts',
    '@/routing': '/src/routing/index.ts',
    // ...
  },
},

Vite’s automatic JSONs conversion into a named-export module. Consider setting Vite’s JSON.stringify in case of troubles.

That’s all. After that, I proceed by fixing errors the fastest way possible with the sole goal of having the codebase working under Vite.

The most annoying part is the new TypeScript configuration because it requires many manual fixes on

re-exported types that we didn’t migrate earlier (export type { Props } from instead of export { Props } from)
Enums, not supported by ESBuild, replacing them with string unions (UPDATE: const enums aren't supported, thanks Jakub for noticing it)

and then

import * as instead of import for some dependencies
import instead of import * as for the static assets

Other problems come from the dependencies consumed only by the Web Worker because:

Every time the Web Worker imports a dependency, Vite optimizes it and reloads the page. Luckily, Vite exposes an optimizeDeps configuration to handle this situation avoiding a reloading loop.

optimizeDeps: {
  include: [
    'idb',
    'immer',
    'axios',
    // …
  ],
},

If something goes wrong when the Web Worker imports a dependency, you don’t have meaningful hints. That’s a significant pain for me but, once discovered, Evan fixed it swiftly.

In the end, after some hours, our project was running on Vite 🎉 it doesn’t care the amount of dirty and temporary hacks I introduced (~ 40 unordered commits) because I am now 100% sure that our project is fully compatible with Vite 😊

3. Benchmarking

Reaching this step as fast as possible has another advantage: we can measure performances to decide if continuing with Vite or bailing out.

Is Vite faster than Webpack for us? These are my early and empiric measurements.

Tool	yarn start	app loads in	React component hot reload **	web-worker change "hot" reload **
Webpack*	150s	6s	13s	17s
Vite*	6s	10s	1s	13s

* Early benchmark where Webpack runs both ESLint and TypeScript while Vite doesn't
** Means from CTRL+S on a file to when the app is ready

Even if the codebase grows up — we are migrating the whole 250K LOC project to a brand new architecture — these early measurements confirm that betting on Vite makes sense.

Notice: We want to reduce risk. Vite attracts us, Vite is faster, Vite is modern… But we aren’t experts yet. Therefore we keep both Vite and Webpack compatibility. If something goes wrong, we can fall back to Webpack whenever we want.

4. Reproducibility

The takeaways of the Feasibility step is a series of changes the codebase needs to migrate to Vite. Now, I look for confidence: if I start from the master branch and re-do the same changes, everything must work again. This phase allows creating a polished branch with about ten isolated and explicit commits. Explicit commits allow moving whatever I can on master, directly into the standard Webpack-based codebase to ease the final migration steps. I’m talking about:

adding Vite dependencies: by moving them to master, I can keep them updated during the weekly dependencies update (we installed vite, @vitejs/plugin-react-refresh, and vite-plugin-html)
adding Vite types
updating the TypeScript configuration with the aforementioned settings (isolatedModules, esModuleInterop, allowSyntheticDefaultImports) and adapting the codebase accordingly
transform our static-assets directory into Vite’s public one

Once done, the steps to get Vite up and running are an order of magnitude fewer.

5. Stability

Since most of the required changes are already on master, I can concentrate on the finest ones. That’s why this is the right moment to

fix TypeScript (remember, not included in Vite) errors
fix ESLint errors
fix failing tests (mostly due to failing imports)
add Vite’s .env files
add the scripts the team is going to use for starting Vite, building the project with Vite, previewing the build, and clearing Vite’s cache (FYI: Vite’s cache is stored in the local node_modules if you use yarn workspaces)
create the HTML templates
checking that all the Webpack configs have a Vite counterpart

Env variables and files deserve some notes. Our project consumes some process.env-based variables, valorized through Webpack’ Define Plugin. Vite has the same define options and has batteries included for .env files.

I opted for:

Using define for the env variables not dependent on the local/dev/production environment. An example

define: {
  'process.env.uuiVersion': JSON.stringify(packageJson.version),
},

Supporting import.meta (where Vite stores the env variables) for the remaining ones.

According to our decision of supporting both Webpack and Vite, we ended up with the following type definitions (an example)

declare namespace NodeJS {
  export interface ProcessEnv {
    DISABLE_SENTRY: boolean
  }
}
interface ImportMeta {
  env: {
    VITE_DISABLE_SENTRY: boolean
  }
}

and this Frankenstein-like function to consume the env variables

export function getEnvVariables() {
  switch (detectBundler()) {
    case 'vite':
      return {
        // @ts-ignore
        DISABLE_SENTRY: import.meta.env.VITE_DISABLE_SENTRY,
      }
    case 'webpack':
      return {
        DISABLE_SENTRY: process.env.DISABLE_SENTRY,
      }
  }
}

function detectBundler() {
  try {
    // @ts-expect-error import.meta not allowed under webpack
    !!import.meta.env.MODE
    return 'vite'
  } catch {}
  return 'webpack'
}

I wouldn’t say I like the above code, but it’s temporary and limited to a few cases. We can live with it.

The same is valid for importing the Web Worker script

export async function create() {
  switch (detectBundler()) {
    case 'vite':
      return createViteWorker()
    case 'webpack':
      return createWebpackWorker()
  }
}

async function createViteWorker() {
  // TODO: the dynamic import can be replaced by a simpler, static
  // import ViteWorker from './store/store.web-worker.ts?worker'
  // once the double Webpack+Vite compatibility has been removed
  // @ts-ignore
  const module = await import('./store/store.web-worker.ts?worker')
  const ViteWorker = module.default
  // @ts-ignore
  return Comlink.wrap<uui.domain.api.Store>(ViteWorker())
}

async function createWebpackWorker() {
  if (!process.env.serverDataWorker) {
    throw new Error('Missing `process.env.serverDataWorker`')
  }
  // @ts-ignore
  const worker = new Worker('store.web-worker.ts', {
    name: 'server-data',
  })
  return Comlink.wrap<uui.domain.api.Store>(worker)
}

About the scripts: nothing special here, the package.json now includes

"ts:watch": "tsc -p ./tsconfig.json -w",

// launches both Vite and TSC in parallel
"vite:start": "concurrently - names \"VITE,TSC\" -c \"bgMagenta.bold,bgBlue.bold\" \"yarn vite:dev\" \"yarn ts:watch\"",

"vite:dev": "yarn vite",
"vite:build": "yarn ts && vite build",
"vite:build:preview": "vite preview",
"vite:clearcache": "rimraf ./node_modules/.vite"

Last but not least: I didn’t manage to have Vite ignoring the Webpack’s *.tpl.html files. I ended up removing the html extension to avoid Vite validating them.

6. Automation

Thanks to the previous steps, I can migrate the whole codebase with a couple of cherry-picks and some RegExps. Codemod is perfect for creating a migration script and run the RegExps at blazing speed.

I created a script that

remove the node_modules directory
transform the code by updating the TypeScript aliases through Codemod
re-install the dependencies
commit everything

Notice that the script must be idempotent — aka running it once or more times produces the same results — this is crucial when launching the script multiple times and applying it to both the master branch and the open PRs.

Here a small part of the script

# replace aliases pointing to directories (idempotent codemod)

codemod -m -d . - extensions ts,tsx - accept-all \
"'@(resources|components|features|journal)/" \
"'@/\1/"


# replace assets imports (idempotent codemod)

codemod -m -d ./app - extensions ts,tsx - accept-all 'import \* as(.*).(svg|png|jpg|jpeg|json)' 'import\1.\2'


# update some imports (idempotent codemods)

codemod -m -d . - extensions ts,tsx - accept-all 'import \* as tinycolor' 'import tinycolor'

codemod -m -d . - extensions ts,tsx - accept-all 'import \* as classnames' 'import classnames'

codemod -m -d ./apps/route-manager - extensions ts,tsx - accept-all 'import PIXI' 'import * as PIXI'

Here you find the whole script. Again: the more you incorporate changes on master before the final migration, the better.

7. Migration

I designed the script to ease migrating all the open branches, but we opted for closing all the PRs and operate just on master.

Thanks to many prior attempts, and the refinements to the script, migrating the codebase is nothing more than cherry-picking the “special” commit and launching the Codemods.

Pushing the red button

In the end, the 30 hours spent playing with Vite, fixing and refining, paid off: after a couple of minutes, the codebase works both under Vite and Webpack! 🎉🎉🎉

The final vite.config.ts file is the following

import { defineConfig } from 'vite'
import reactRefresh from '@vitejs/plugin-react-refresh'
import { injectHtml } from 'vite-plugin-html'
import packageJson from '../../apps/route-manager/package.json'

// see https://vitejs.dev/config/
export default defineConfig(({ mode }) => {
  return {
    // avoid clearing the bash' output
    clearScreen: false,

    // React 17's JSX transform workaround
    esbuild: { jsxInject: `import * as React from 'react'` },

    define: {
      'process.env.uuiVersion': JSON.stringify(packageJson.version),
    },

    server: {
      port: 3003,
      strictPort: true,
    },

    plugins: [
      reactRefresh(),
      injectHtml({
        injectData: {
          mode,
          title: mode === 'production' ? 'WorkWave RouteManager' : `RM V3 @${packageJson.version}`,
        },
      }),
    ],

    json: {
      // improve JSON performances and avoid transforming them into named exports above all
      stringify: true,
    },

    resolve: {
      alias: {
        '@/defaultIntlV2Messages': '/locales/en/v2.json',
        '@/defaultIntlV3Messages': '/locales/en/v3.json',
        '@/components': '/src/components',
        '@/intl': '/src/intl/index.ts',
        '@/atoms': '/src/atoms/index.ts',
        '@/routing': '/src/routing/index.ts',
        // ...
      },
    },

    // the dependencies consumed by the worker must be early included by Vite's pre-bundling.
    // Otherwise, as soon as the Worker consumes it, Vite reloads the page because detects a new dependency.
    // @see https://vitejs.dev/guide/dep-pre-bundling.html#automatic-dependency-discovery
    optimizeDeps: {
      include: [
        'idb',
        'immer',
        'axios',
        // ...
      ],
    },

    build: {
      target: ['es2019', 'chrome61', 'edge18', 'firefox60', 'safari16'], // default esbuild config with edge18 instead of edge16

      minify: true,
      brotliSize: true,
      chunkSizeWarningLimit: 20000, // allow compressing large files (default is 500) by slowing the build. Please consider that Brotli reduces bundles size by 80%!
      sourcemap: true,

      rollupOptions: {
        output: {
          // having a single vendor chunk doesn't work because pixi access the `window` and it throws an error in server-data.
          // TODO: by splitting axios, everything works but it's luck, not a designed and expected behavior…
          manualChunks: { axios: ['axios'] },
        },
      },
    },
  }
})

Please note that this

esbuild: { jsxInject: `import * as React from 'react'` }

is helpful only if you, like us, have already upgraded your codebase to new React 17’s JSX Transform. The gist of the upgrade is removing import * as React from 'react' from jsx/tsx files. ESBuild doesn’t support new JSX Transform, and React must be injected. Vite exposes jsxInjecton purpose. Alternatively, Alec Larson has just released vite-react-jsx, and it works like a charm.

Last but not least: for now, I can’t leverage vite-tsconfig-paths to avoid hardcoding the TypeScript aliases in Vite’s config yet because, until we support Webpack too, the presence of “public” in the path makes Vite complaining

// Webpack version:
"@/defaultIntlV2Messages": ["./apps/route-manager/public/locales/en/v2.json"]

// Vite version:
'@/defaultIntlV2Messages': '/locales/en/v2.json'

Cypress tests

Unrelated but useful: if you have Cypress-based Component Tests in your codebase, you can jump on Vite without any issue, take a look at this tweet of mine where I explain how to do that.

Benchmarks and conclusions

The final benchmarks confirm the overall speed of Vite

Tool	1st yarn start, app loads in	2nd yarn start, app loads in	browser reload (with cache), app loads in	React component hot reload **	server-data change "hot" reload **
Webpack	185s	182s	7s	10s	18s
Vite	48s	31s *	11s	1s	14s

* Vite has an internal cache that speeds up initial loading
** Means from CTRL+S on a file to when the app is ready

The comparison is merciless, but is it fair? Not really. Vite outperforms Webpack, but, as said earlier, we run TypeScript and ESLint inside Webpack, while Vite doesn’t allow us to do the same.

How does Webpack perform with a lighter configuration? Could we leverage the speed of ESBuild without Vite? Which one offers the best Developer Experience? I address these questions in part 3.

Top comments (4)

Jakub Jastrzębski • Jun 2 '21

Enums, not supported by ESBuild, replacing them with string unions

I'm curious if you mean the regular enums, or const enums. The latter are not supported (yet) but the former should be working without any issues. Just be careful to store them in .ts files rather than .d.ts.