Matanuska ADR 017 - Vitest, Vite, Grabthar, Oh My!

This article is a repost of an ADR from Matanuska BASIC, my attempt to write a BASIC interpreter in TypeScript.

In October, what began as a change in test frameworks snowballed into a complete refactor of Matanuska's builds. These changes were very significant, and yet happened quietly. This ADR intends to remedy that situation, to document what those changes were and why they happened.

Tap and NodeNext

When I started Matanuska, I chose Node Tap as my test framework. Tap was my favorite test framework in Node for a very long time. It's historically had an API that's less "magical" than some of the other frameworks out there, it outputs TAP by default - always a bonus - and it has pretty high-level reporting.

However, in recent years, Tap has started to grow weary. Its API became more complicated as it needed to support promises and async/await. Its features became more complicated and complex, and it became harder to use.

But what ultimately made me disillusioned was encountering bugs and odd behavior over time. For instance, I have a directory called ./test/helpers which contains helper modules for my tests. This is a convention I learned from Nest tests during my time at Procore. Tap absolutely refused to ignore this directory (which had no tests in it), regardless of my efforts to configure it thusly.

What pushed me over the edge was issues with native import syntax in Node.js modules (called "nodenext" in TypeScript parlance). Up to this point, I was using "commonjs" builds, where TypeScript would compile my files to use require. This was mostly fine and good, but it would struggle with modules using native import. Most of my dependencies used commonjs, but one of my development dependencies was using native import - this in part motivated me to make the switch. Unfortunately, Tap struggled with this when I initially made this attempt.

Vitest

I began searching for a new test framework, and at the recommendation of Nuck, I gave Vitest a shot. It's by the developers of Vite, which I loved. I don't do a lot of frontend development, but when I do, Vite is often my choice. Unlike many other solutions to frontend builds I've tried in the past, Vite "just works" and involves minimal baggage (looking at you, Angular).

It turns out that Vitest is incredible, and I made the change incrementally - but also quickly. I started by configuring Vite to build files named *.spec.ts, and having Tap run tests named *.tap.ts. Within a few days, the switch was complete.

Overall, I have been extremely happy with Vitest. It has the good parts of Jest and Chai, but without the stranger baggage. It really is incredible, and I can't recommend it enough.

Native Imports in TSC and SWC

Switching to Vitest fixed native import in the tests, and I was quite happy with that. However, I was not out of the woods when it came to using native import in the main project - that effort was still failing.

The issue I ran into is deep in the weeds. When in module mode, Node likes to have imports specify the extension of the file you're importing, and doesn't like importing directories - you have to spell out ./directory/index.mjs, rather than simply specifying ./directory. tsc (TypeScript's standard compiler) doesn't rewrite these import paths in "nodenext" mode. This alone made things awkward.

But I also had my Vitest build configured to use SWC, the compiler backend I had configured for Vitest, and it had issues of its own. SWC is cool. It's a TypeScript compiler written in Rust that is extremely fast, and - unlike tsc - it mostly handles rewriting import paths just fine. However, I did find that it rewrites index.mjs imports into directory imports.

I also found that SWC's standard command line interface was really immature. This seems to be because it was really intended to run within other build and bundling tools, such as Vite and Nextjs.

By this point, I was finding the impedance mismatch between Vitest's SWC-based build and my project's tsc build to be overwhelming, and I yearned to make the two match. I considered switching Vitest to use tsc. But I also found that SWC was SO much faster (it nearly doubled the speed of my tests) that I couldn't say no. By this point, I was committed to using SWC in my build.

Vite

I realized that the way to use SWC successfully in Matanuska's main build was going to involve a singular bundle. At this point, I started asking if Vite itself could run my main build. After all, it was building my tests with SWC successfully!

As it turns out, Vite is more than capable of doing this, through its server-side rendering functionality (SSR). This is a bit of a misnomer. The motivation is to support server-side rendering of React projects, but the actual feature is bundles for server-side JavaScript runtimes like Node.js.

It's a little limited as compared to its frontend builds - but only a little. For one thing, it can only really handle one SSR entry point. The biggest issue, though, is that Vite's standard dev mode is geared towards hot module replacement of frontend code through a proxy over a server - not something that benefits Matanuska. The ramifications of that, though, were simply that I would need to run Vite in batch build mode - not all that different from the pre-existing build process.

Ultimately, I've been pretty happy with Vite as a build tool. It's blazing fast and does exactly what I need!

Type Checking, SWC and TSC

SWC is a great tool when it comes to compiling TypeScript. But it's a bad tool for type checking TypeScript. This is because part of why it's so fast is that it mostly ignores types completely. This meant that, while SWC was being used for the builds, I still needed tsc in the mix for type checking.

Luckily, tsc is much more flexible with inputs when it comes to type checking than it is with generating compiled output. After all, it doesn't need to concern itself with output at all if it's running with the --noEmit flag.

Unfortunately, this did mean that configuration began to sprawl. At this point, I had configurations not just for Vite (shared with Vitest) and tsc, but also for Prettier, ESLint and even ShellCheck. Many of these files had shared settings that needed to match each other. This was somewhat manageable, until Vite was also in the mix.

Grabthar

My instincts when presented with this configuration sprawl was to begin writing some scripts to generate and update configuration for me. The first draft can be seen in the PR that initially implemented the Vite build. These scripts reflected off a shared JSON file (later YAML) and generated the configurations for the downstream tools. In the case of Vite, this happened through an import, but for other tools, it just wrote JSON to disk.

I began to realize that these scripts were becoming elaborate enough that I wanted to massage them into a proper tool. I created a new package, moved the scripts into it, and named the project grabthar.

This name has a funny background. Many years ago, I was in an IRC conversation with a developer who began describing a build tool he was making. I was a jerk and scoffed at the API, and began sketching out my own build tool. I named it grabthar after my favorite joke from Galaxy Quest. It didn't go anywhere, but I kept the source around. When it came time to write a tool for Matanuska, I decided to reuse the name. But anyway, it turns out I was talking to the author of Grunt, and boy did I look silly.

Either way - Matanuska now has a custom build tool. This tool runs hooks to generate configurations, exports functions for tools using JavaScript configs (ie., Vite and ESLint), and runs the appropriate tools in an opinionated manner.

Make no mistake, grabthar is extremely opinionated. Aside from the shared configurations, it's not all that customizable. Any tools using it would need to support exactly the same underlying build stack as Matanuska. But there are benefits to that, too. I'm currently only using it for Matanuska and a handful of its tools, but may use it outside Matanuska in the future if it ages well.

citree

A brief note on citree. citree is a tool I wrote for generating Matanuska's AST classes. This tool is heavily inspired by the script used in Crafting Interpreters' jlox interpreter. It uses a DSL implemented in ts-parsec that takes a specification for an AST and generates classes implementing a visitor pattern. The DSL is a little janky, but it does exactly what I need for Matanuska.

I considered rewriting citree to run as a step in the Vite build. However, I decided to keep it as a separate code generation step. This is because, while hacking, I need the TypeScript files to exist in order to do type checking - simple enough.

jscc

A final consequence of these refactors was the introduction of jscc.

Matanuska has included build-time code generation from pretty early on. In particular, it uses an environment variable (MATBAS_BUILD) to control whether or not to include certain debugging hooks. During development, good debugging output is extremely desirable. But for a release, it slows things down to an unacceptable level - or, at least, that's the common wisdom.

Initially, I solved this through using nunjucks templates for a constants.ts file and a debug.ts file. Under MATBAS_BUILD=debug, the latter file would contain debug output, including tracing (see ADR 14 for more context here). But under MATBAS_BUILD=release, those hooks would be empty "no-op" functions. This all worked, but was dissatisfying.

JSCC was a simple, general purpose tool for the kind of conditional logic I was looking for. Not only did it have a nice syntax that constituted valid JavaScript; it also had a build plugin that would integrate it into my build for all my files. To me, this was a major win.

Summary

That was a lot, so I wanted to quickly summarize what happened here.

Before

• citree for generating the AST
• tsc for both TypeScript compiling and type checking
• No bundling
• Prettier for formatting
• ESLint for TypeScript linting
• ShellCheck for bash linting
• Node Tap for testing
• Nunjucks for build-time configuration and conditional compiling
• No shared build tool

After

• citree for generating the AST, as before
• tsc for type checking only
• swc for TypeScript compiling
• Vite for bundling
• Prettier, ESLint and ShellCheck used for formatting and linting, as before
• Vitest for testing
• jscc for build-time configuration and conditional compiling
• Custom grabthar build tool