Maël Nison

Posted on Oct 25, 2021

Yarn 3.1 🎃👻 Corepack, ESM, pnpm, Optional Packages ...

#javascript #npm #node #showdev

Welcome to the release notes for Yarn 3.1! We're quite excited by this release, as it brings various improvements that we've all been looking forward to. Let's dig into that!

As always, keep in mind those are only the highlights, the full changelog is much more comprehensive. And if you just happen to love reading our release posts, here are the past entries 👇

Sponsoring

The Yarn org needs your help to make our work more sustainable! Please take a look at our OpenCollective and GitHub Sponsors pages for more details 😊

Improvements

Node.js Corepack Integration

Did you know that Yarn now ships with Node? This is done via the Node.js Corepack project, which includes both the Yarn and pnpm binaries as shims. By adding the packageManager field to your package.json, you can enforce the use of a specific package manager & package manager version in a completely transparent way:

{
  "packageManager": "yarn@3.1.0"
}

Note that Corepack is available starting from Node.js 16.9, but is currently opt-in. Don't forget to run corepack enable a single time to make sure the shims are globally installed!

We also improved in 3.1 the init command to properly support Corepack: running yarn init -2 will now automatically setup a Yarn Modern project, setting its packageManager field as required 💫

ESM Support

ESM has always been supported when using the node_modules linker, since it's the same old install strategy that Node has always supported. However, with PnP taking ownership of the resolution pipeline, compatibility with ESM wasn't a given and had to be implemented using the Loader Hook API.

While the Loader Hook API isn't entirely stable yet, a large amount of work has been made lately and our team has been able to produce an initial experimental support for ESM modules. It should be enabled automatically if we detect that one of the packages in your dependency tree contains a "type": "module" field, but you can enable or disable it manually through your settings:

pnpEnableEsmLoader: true

Being experimental, it's possible that some bugs may arise or that new Node releases bring some breaking changes around the API. Be sure to report issues on our bug tracker!

New Install Mode: `pnpm`

The pnpm package manager was one of the first tools to advocate for using symlinks when installing packages within the node_modules folder. While we went another way with PnP, we decided that the implementation cost was low enough that it would be worth adding support for this symlink-based install strategy as well.

Starting from Yarn 3.1, you can try out symlink-based installs by adding the following setting to your .yarnrc.yml file:

nodeLinker: pnpm

Conditional Packages

Esbuild and swc are two native packages that gained a lot of attention lately thanks to their impressive performances over their competitors. They recently revamped how their packages are built to avoid complex postinstall scripts, but did so in a way that was less efficient than before for Yarn projects.

Yarn 3.1 features a new optimization that kicks in when a package is listed as optionalDependencies and lists os and/or cpu fields. When that happens, Yarn will skip fetching and installing those packages unless they match the current system parameters.

In case you need to manually configure a strict set of package architectures to support (for example like in a zero-install case, where you want to read from an immutable set of packages), you can use the supportedArchitectures setting:

supportedArchitectures:
  os: [linux, darwin]
  cpu: [x64, arm64]

Smart Changeset Filters

The yarn workspaces foreach and yarn workspaces list commands now ships with brand new --since flags. When set, those commands will only execute against the packages that changed when compared to the main branch (either main or master, depending on the branches in your repository).

This can come in handy if you wish to only run builds in some specific workspaces, or just get a list of the workspaces which changed for scripting purposes:

yarn workspaces foreach --since run eslint .
yarn workspaces list --since

The --since flag also accepts an optional argument (--since=${commit-ish}) to manually define a source from which the changes should be derived.

New Workspace Syntax: `workspace:^`

Workspaces supported a special syntax via workspace:*, with those ranges being replaced at publish-time by exact ranges corresponding to the real version of the target workspace. However, if you wanted to use a caret instead of an exact range, you had to use the verbose workspace:^x.y.z form, which Yarn updated repo-wide after each publish.

Yarn now supports workspace:^ and workspace:~ as well, making it much easier to cross-reference workspaces within a monorepo where most packages are intended to be published, by preventing a good amount of the merge conflicts that used to happen after Yarn updated the verbose ranges.

Additionally, as a special case, this syntax is now allowed in the peerDependencies field as well:

{
  "peerDependencies": {
    "@my/other-package": "workspace:^"
  }
}

Top comments (14)

Zoltan Kochan • Nov 9 '21

I am super grateful that you named the option "pnpm" ❤️

Shinigami • Oct 27 '21

I tried nodeLinker: pnpm instead of nodeLinker: node-modules, but I'm getting many errors like Cannot find module
But the modules are sub dependencies, so why should I require them to have them explicitly set in my own dependencies? I don't want to configure hundreds of dependencies when they get already declared as sub dependencies in the relevant packages

So what can I do to tell yarn that it should read the symlinks correctly?

Devin Rhode • Dec 22 '21

Is there a hard links option that might work? Are you on windows?

Jeremy Gollehon • Dec 17 '21

We mostly use React Native and have used Yarn 1 from the start.
With React Native not supporting PnP, and it appears no one is working on it, is there any benefit to upgrading to 3.x (using node_modules)?

Devin Rhode • Dec 22 '21

I've found 2.x is way faster. I'd expect 3.x is even faster yet.

I hate that yarn needs to use a patched version of typescript. The logs and warnings are not very upfront. I'm waiting for typescript pnp pr to get merged.

The esm support is awesome. I remember rich harris tweeting something like "friends don't let friends use yarn 2". Once this is stable that will be really great, solve a nice batch of issues around pnp.

If you have developers on windows and have a shitty IT department, I've ran into an issue with yarn 2 where we were getting some security popup on windows, and it wasn't worth the effort to try and go through this crazy corporate approval process. I'm sure this is dependent on how you have installed node. Hopefully nvm-windows or Volta.sh would avoid such security prompts.

Devin Rhode • Dec 22 '21

Actually I think we encountered the security prompts when using nvm-windows.

sativ01 • Nov 2 '21 • Edited

This is awesome!
I'm still using yarn 1 for my projects though
I would like to switch to the latest greatest, but not sure how to do the transition
Do you maybe have a tutorial that explains the process for these use-cases:

yarn upgrade for React + Typescript project
yarn upgrade for NodeJS + Typescript project
yarn upgrade for monorepo

I'd like to know what are the new features one can simply use to improve install/build etc. time for the project
Hope I'm not asking too much.
It's just that usually yarn is set up only once in the beginning then it's a part of the app that you simply too scared to touch since it's working.
thanks!

Ayc0 • Oct 25 '21

This new workspace flags are 🔥

Edwin • Oct 25 '21 • Edited

What's the best way to handle these special protocols during local development? I use portal: to get the same behaviour as npm link when I want to test an NPM package that I'm working on, but I don't want to commit my package.json file with the portal:../path/etc stuff still inside of it.

I could probably run a special ESLint task on pre-commit to prevent this, but maybe there is a better way?

Maël Nison • Oct 25 '21

At the moment no, we always use the local package.json as a local source of truth, there's no escape from that. For workspaces it's not a problem (since you can just edit them directly, you don't have to play with portals), but for 3rd-party vendors it'd be nice indeed.

Edwin • Oct 26 '21 • Edited

Alright thanks. Just to clarify, I'm talking specifically about a scenario with Yarn + Lerna (mono-repo with NPM packages). Then I want to test the NPM package inside some other repository so I use portal:.. (or npm link) to set that up. Having all workspaces inside the same repo as you describe would probably work, yes.

Another issue is that Lerna cannot handle Yarn specific protocols (workspace:*), so any Lerna command (lerna version/lerna publish) will fail. This means we cannot use Yarn specific protocols in package.json.