Fixing Package Dependencies

Both Embroider and pnpm ask that packages declare their dependencies correctly: List a dependency (if and only) if it is used.

This is difficult to do when working on a large monorepo (consider an Ember app with many Ember addons and Node packages) that uses yarn@v1. Developers can forget to update the package.json's, because the Ember app can build and run even when a dependency is missing, as long as it gets pulled in from another package.

So neither build nor run can tell us if some package didn't declare its dependencies right. How else can we fix the package.json's so that we can introduce Embroider and pnpm?

1. Static code analysis

Given a file, we can see which dependencies should be present, because we know how JavaScript and Ember work.

For example, were a JavaScript (or TypeScript) file to show,

import { setupIntl } from 'ember-intl/test-support';
import { setupRenderingTest as upstreamSetupRenderingTest } from 'ember-qunit';

export function setupRenderingTest(hooks, options) {
  upstreamSetupRenderingTest(hooks, options);

  // Additional setup for rendering tests can be done here.
  setupIntl(hooks, 'de-de');
}

we would tell from the import statements that the package depends on ember-intl and ember-qunit.

And, if a template file were to show,

{{page-title "My App"}}

<WelcomePage />

{{outlet}}

our knowledge of Ember and its addon ecosystem would direct us to ember-page-title, ember-welcome-page, and ember-source, respectively.

Even when things are implicit (e.g. ambiguity in double curly braces, module resolution, service injection), we can guess the origin of an entity (entities are components, helpers, modifiers, services, etc.) with high accuracy, thanks to Ember's strong conventions.

2. Codemod

Still, we shouldn't check every file in every package manually. That's time-consuming and error-prone.

Instead, we write a codemod (really, a linter) using @codemod-utils. For every package, the codemod parses what's relevant and creates a list of dependencies that should be present ("actual"). It then compares the list to that from package.json ("expected").

To analyze implicit code, there needs to be a list of known entities (a one-time creation), which maps every package that we want to consider to its entities. We can use a Map to record that information.

const KNOWN_ENTITIES = new Map([
  [
    'ember-intl',
    {
      helpers: [
        'format-date',
        'format-list',
        'format-message',
        'format-number',
        'format-relative',
        'format-time',
        't',
      ],
      services: ['intl'],
    },
  ],
  [
    'ember-page-title',
    {
      helpers: ['page-title'],
      services: ['page-title'],
    },
  ],
  [
    'ember-welcome-page',
    {
      components: ['welcome-page'],
    },
  ],
]);

Even explicit code like import statements aren't trivial to analyze. Take the following example:

import Route from '@ember/routing/route';
import fetch from 'fetch';

When we don't provide the right context (i.e. this code is for Ember), the codemod would consider @ember/routing and fetch as dependencies, instead of ember-source and (likely) ember-fetch. The codemod should present its analysis in such a way that we can easily check for false positives and false negatives.

// Results for my-package-37

{
  missingDependencies: [
    'ember-asset-loader',
    'ember-truth-helpers'
  ],
  unusedDependencies: [
    '@babel/core',
    'ember-auto-import',
    'ember-cli-babel'
  ],
  unknowns: [
    'Service - host-router (addon/routes/registration.ts)',
  ]
}

3. Results

The codemod that I had built (in a couple of days) analyzed a production repo with 123 packages in 25 seconds. There were a total of 11,108 files, but the codemod knew to analyze only 5,506 of them (less than half). That's an average of 0.005 seconds/file and 0.20 seconds/package!

To learn more about writing codemods, check out the main tutorial from @codemod-utils.