Create a productive environment for your personal nodejs scripts

There are times where I want to automate a boring task using nodejs and I get really excited, because I get to write code for fun. The thing is that half of that excitement goes out the window the moment I have to npm init and then npm install x, those extra steps make me a bit sad. I've never had to that with bash. I want to be able to skip those steps and still have things that just work. How are we going to do that? With the power of bad practices and a few questionable decisions.

The goal

I don't want to get all fancy and mess around too much with node internals, the only thing I want is to have my favorite utility functions and some dependencies preloaded. That's it. I want to just create a something.js and start writing stuff like this.

format_number(74420.5);
// => 74.420,5

Or even better.

sh('node --version');
// => vXX.YY.Z

Without even touching npm. Let us begin.

Step 1: Polute the global scope

Kids, you should never polute the global scope of a node process. Never. However, since this is just for funsies we are going to do just that. I'm going to trust that you, dear reader, are not going to try this in any "production" environment. Only try this at home.

So, the node cli has a handy flag called --require, with it we can tell it to execute any script or module we want before executing the "main" script. It would be something like this.

node --require /path/to/some/script.js ./my-script.js

Let's start with that. Go to that folder where you have all your side projects (I know you have one) and make a new directory (I called jsenv). Next create a main.js or index.js and put this.

function format_number(number) {
  return new Intl.NumberFormat('de-DE').format(number);
}

global['format_number'] = format_number;

Next, create a script in some random location and try to use format_number.

With everything in place try this.

node --require /path/to/jsenv/main.js /tmp/my-script.js

That should have worked. With this simple step we can now "preload" our favorite utilities. But we can take this further.

Step 2: Get your favorite tool(s)

In the jsenv (or whatever you called it) folder run npm init -y and then install something from npm. For this example I will choose arg, this is a library I use to parse command line arguments. If you're going to create a cli tool you'll need one of those, so might as well "preload" that one too.

On jsenv/main.js add this.

global['cli'] = require('arg'); 

On your script add this.

const args = cli({ '--help': String });
console.log(args);

And for the test.

node --require /path/to/jsenv/main.js \
  /tmp/my-script.js --help me

Isn't it cool? Now we can get things from npm ahead of time and not worry about them again. Which leads us to.

Step 3: Get help from the outside

One of the strengths of bash is that we can call just about any tool we have available on our system by just using their name. I know node can do that too, but it's awkward at best. But there is hope, the library execa has a function (execa.command) that can give us a syntax that is more convenient. Before using it in a script I would like to do some adjustments.

const execa = require('execa');

const shell = (options) => (cmd) => execa.command(cmd, options);
const sh = shell({ stdio: 'inherit' });
sh.quiet = shell();
sh.run = (cmd) => sh.quiet(cmd).then(res => res.stdout);
sh.build = shell;
sh.safe = (cmd) =>
  sh(cmd)
    .then((arg) => arg)
    .catch((arg) => arg);

global['sh'] = sh;

I called the variable shell but it's not really a shell. You can't do fancy stuff with it. It's just suppose to work like this.

sh('some-command --an argument --another one');

You can only call commands with its arguments. If you want to get creative you can still call your shell.

sh('bash -c "# fancy stuff goes here"');

I would recommend against this, since is slower and increases that chances of things getting horribly wrong if you pass dynamic values to it.

sh will print the output of the command to stdout. The variant sh.quiet will not do that. sh.safe will not throw an error on fail. And sh.run will keep the result to itself and then it will return the output as a string.

Step 4: Dependencies on demand

As you might have guessed, "preloading" a bunch of libraries can have a negative impact on the startup times of your script. It would be nice if we could "require" a library without npm install everytime for every script. We can do this with the help of the environment variable known as NODE_PATH. With it we can tell node where it can find our dependencies.

We can test this by going to the jsenv folder and installing some tools.

npm install node-fetch form-data cheerio ramda

May I suggest also puppeteer-core, it's the core logic of puppeteer decoupled from the chromium binary. Chances are you already have chrome or chromium in your system, so there is no need to use the puppeteer package.

Now we need some test code.

const fetch = require('node-fetch');
const html = require('cheerio');

(async function () {
  const response = await fetch('http://example.com');
  const $ = html.load(await response.text());

  console.log($('p').text());
})();

Excuse my iife, I was testing this in an old version of node.

We have our tools and our script, now we need to tell node where it can find our packages.

NODE_PATH=/path/to/jsenv/node_modules/ \
node --require /path/to/jsenv/main.js \
/tmp/my-script.js

Now we are getting deep into "unix shell" territory so this might not work on windows if you use cmd.exe or powershell. But it should work on git-bash.

That command should give us this.

This domain is for use in illustrative examples in documents.
You may use this domain in literature without prior
coordination or asking for permission.More information...

We have gain the ability to call libraries located somewhere else. Now we are free from npm init and npm install. We can start hacking on stuff by just creating a single .js file. But we are missing something.

Step 5: Make it convenient

That node command we need to type is not very nice. So, what we would do now is create a script that would call it for us.

#! /usr/bin/env sh

NODE_PATH=/path/to/jsenv/node_modules/ \
  node --require /path/to/jsenv/main.js "$@"

The final step would be putting this somewhere in your PATH, so you can call it like this.

js /tmp/my-script.js

Or make this.

#! /usr/bin/env js

const args = cli({});
const [num] = args._;
console.log(format_number(num));

Assuming you made it executable, it should be possible for you to do this.

/path/to/my-script 12300.4

Extra step: Enable es modules and top-level await

Recent versions of node will allow you to have that but only on .mjs files or if you have a package.json with the property "type": "module". But there is a problem, node ignores the NODE_PATH env variable when using native es modules. Don't be afraid we can still use them, but we need the package esm to enable them.

First step, get the package.

npm install esm

Create an esm.json file and put this.

{
  "cache": false,
  "await": true
}

Modify the node command.

#! /usr/bin/env sh

export ESM_OPTIONS=/path/to/jsenv/esm.json

NODE_PATH=/path/to/jsenv/node_modules/ \
  node --require esm \
       --require /path/to/jsenv/main.js "$@"

Now this should work just fine.

#! /usr/bin/env js

import fetch from 'node-fetch';
import html from 'cheerio';

const response = await fetch('http://example.com');
const $ = html.load(await response.text());

console.log($('p').text());

Show me all the code

I got you fam. It's here, and with some more bells and whistles. But if you're going to use that main.js you might want to delete a few requires, probably won't need all of that.

quick note: all this code was tested against node v10.23.1, and all the specific versions of dependencies are on this package.json.

---

Thank you for your time. If you find this article useful and want to support my efforts, consider leaving a tip in ko-fi.com/vonheikemen.