Chapter 3: images part one

Cover image by M Mahbub A Alahi

Recap

Last time, we configured a local server and previewed our work:

• We installed browser-sync so we can see the contents of dist in a web browser
• Learned about escape characters
• Ran more than one command at the same time
• Learned about the special start command name
• Installed, then removed a package because we changed our mind
• Learned that if something goes wrong with node_modules we can always nuke it and start again

This lesson is all about optimising images. There's a lot to get through here, but I'm going to take it slow.

Fun break

Perhaps right now, you're feeling a bit of imposter syndrome. We are standing on the shoulders of the giants who made all this technology, and yet we are not fit to shine their shoes.

Don't worry: I have a cure for that.

Let's visit a site made by some of the best engineers the world has ever seen. How about Apple? I need you to open the home page in Chrome (sorry, Firefox - I'm cheating on you this one time). Don't dismiss the cookies or click on anything else and open the developer tools. You can do this by either hitting the F12 key or right-clicking on an empty patch of page and selecting Inspect from the pop-up menu.

There's a lot going on in the developer tools but I want you to pay attention to the tabs across the top. They list off the different parts and start Elements Console Sources Network ... I'd like you to select Lighthouse. It might be hidden behind one of these wee chaps: ».

Once the Lighthouse panel has displayed, click the Analyze page load button and wait for the results. Is everything green?

You can scroll down the report and under the Diagnostics heading, you should see a red warning triangle next to a sub-heading which reads Serve images in next-gen formats. We're going to address this problem in the next lesson, but let me ramble on about Lighthouse for a bit first.

Running a Lighthouse audit is mostly a little disheartening. Getting green scores across the board is basically impossible, if you have a marketing department which is determined to drop a 500MB auto-playing video into a popup window. And even if you don't have this problem, improving the score requires many different departments working together.

In the next lesson, we're going to do what Apple failed to do: serve all our images in "next gen" formats.

(and if you're still feeling like a bad engineer, try running Microsoft through Lighthouse)

Compressing images

When it comes to Node packages which can convert images to cutting-edge modern formats, the new hotness is Sharp. It's so bleeding-edge that we need to write some of our own tools to run it.

Sharp is designed to be called from within a Node application and can do all kinds of stuff like adding blurs, adding text, rotating or converting images to different formats. It's just the last bit we want to use.

However, it doesn't just get installed and then called, like the other plugins we've been using. We're going to have to write a few functions to get it to do what we want.

What we want to achieve

An ideal developer user experience would be the following:

1. You start to run your local server
2. You dump a bunch of images into the src/img directory (this doesn't exist yet)
3. Those images get automagically converted into next generation files in the dist/img directory (this doesn't exist either, yet)

You might also want to do any of the following:

• Run a task which converts all images in the src/img directory into their optimised versions in dist/img - for example when you've added some images but the server wasn't running at the time
• Overwrite an existing image in src/img with a new one with the same name, and have the corresponding images in dist/img update automatically
• Use as many different sub-directories in src/img and have those be replicated on dist/img - for example src/img/gallery/open-day which lives inside dist as dist/img/gallery/open-day

That's a lot of features! It's easy to feel overwhelmed when seeing requirements like this. You might mentally try and solve all of them at once. Resist this temptation, if you can! Remember that computers are very, very stupid and need to be told very simple instructions. Most of our job is to take complicated tasks and break them down into very simple steps.

Creating new directories

Let's create some new directories for these images to sit inside. Add some img directories, so your Node project folder looks like this:

🗀 dist
  🗀 css
  🗀 img
🗀 src
  🗀 img
  🗀 scss
.gitattributes
.gitignore
package.json
package-lock.json

Great! Now we just need to write all the code which converts the image from one format to another. So how do we get started?

Installing another package

If you guessed "installing another package from NPM", you have earned a new achievement: "Web Developer Cynicism Level 1". Spend those experience points wisely! The package we need to install is called onchange.

Installing onchange

Inside a terminal, type:

npm i -D onchange

In order to use onchange, we need to edit package.json. Currently, the scripts node looks like this:

"scripts": {
  "sass-dev": "sass --watch --update --style=expanded src/scss:dist/css",
  "sass-prod": "sass --no-source-map --style=compressed src/scss:dist/css",
  "serve": "browser-sync start --server \"dist\" --files \"dist\"",
  "start": "concurrently \"npm run serve\" \"npm run sass-dev\"",
  "prepare": "concurrently \"npm run sass-prod\""
}

We need to change it to look like this:

"scripts": {
  "sass-dev": "sass --watch --update --style=expanded src/scss:dist/css",
  "sass-prod": "sass --no-source-map --style=compressed src/scss:dist/css",
  "serve": "browser-sync start --server \"dist\" --files \"dist\"",
  "start": "concurrently \"npm run serve\" \"npm run sass-dev\"",
  "prepare": "concurrently \"npm run sass-prod\"",
  "log-images": "onchange \"src/img\" -- echo '{{event}} to {{file}}'"
}

Let's break down that new command (the second half of which I took directly from the onchange documentation):

• log-images - this is the name we're calling our new command. It could be anything, as long as it doesn't include spaces
• onchange - when we run our task, the first thing it does is invoke the onchange method which we've just added to node_modules
• \"src/img\" - this is the (escaped) path to the directory which onchange will be watching.
• -- - I can't find what this empty flag means in the documentation. It seems to act as a sort of separator where anything after it is a new Node command which will run, when onchange detects a change
• echo - this allows us to add text to the terminal. It's the same as console.log() in JavaScript
• '{{event}} to {{file}}' - this is a string which is built up from two tokens {{event}} and {{file}}. This string will be echoed to the terminal (this just means that the text will appear in the terminal). These tokens are passed by onchange. What do they mean? Let's find out!

Now from a terminal, run our new command by typing:

npm run log-images

onchange runs constantly - it won't stop until we put the terminal in focus and hold down ctrl and c to cancel out of it. This means that as we update the directories, you can watch the terminal and see it update in real time.

Here's some example images to download. Save the first one to src/img and watch what happens in the terminal. You should see something like this:

"'add" "to" "src\img\h7zTxla.png'"
"'change" "to" "src\img\h7zTxla.png'"

Now save the second one, but overwrite the first. You should see the following:

"'change" "to" "src\img\h7zTxla.png'"
"'change" "to" "src\img\h7zTxla.png'"

Now try renaming the file to example-01. You should see:

"'add" "to" "src\img\example-01.png'"
"'unlink" "to" "src\img\h7zTxla.png'"

OK, so the two {{event}}s we're interested in are add and change - when these occur, we need to ... do something. And that something has to do with the value of the second argument {{file}}.

JS Module mode

First thing we need to do is tell Node that we want to use JS modules. Modules are the most modern way to use JavaScript, where every function lives in its own wee file and has a single purpose (at least that's the idea).

To enable this, we need to add a new node to package.json:

"type": "module"

This needs to sit in the "root" of package.json, as a sibling of other nodes you might recognise, such as name and version.

Something like this:

{
  ...
  "description": "A course on Node.js for front end developers",
  "type": "module",
  "scripts": {
    ...
  }
  ...
}

Passing arguments to a script

We're going to write a script (don't worry - it's a javascript) which will run and gather together all this information. But it's not going to sit inside src.

Create a new directory called tools and put it in the root of the Node application. This means it's a sibling to dist and src. The word "root" can get pretty confusing. Here's three different roots we might encounter:

• The root of the Node application (where package.json lives)
• The root of the local web server (that's the dist directory)
• What Microsoft™ Windows™ regards as the root of the file system (this is probably the C: drive on your computer and we'll try and avoid this root as much as possible)

The directory structure of your project should look like this:

🗀 dist
  🗀 css
  🗀 img
🗀 src
  🗀 img
  🗀 scss
🗀 tools
.gitattributes
.gitignore
package.json
package-lock.json

The new file should be called image-compress.js. It looks like this:

import { argv } from "node:process";

const [node, thisFile, filePath, fileEvent] = argv;

console.log(filePath, fileEvent);

Hmm. There's a lot going on there. Let's break it down.

First thing we do is import a function called argv() from node:process. This is built into Node and allows us to examine arguments, but not in the philosophical sense. We're going to pass image-compress.js an argument in a moment.

Next, we do what is called destructuring, which is a very cool, modern way to assign a bunch of variable at the same time. Because we know that argv() is going to send us data inside an Array in a particular order, we can get ahead of the game and give them nicknames.

Finally, we console.log() two of our favourite arguments to the console. (console.log() is like echo, but in JavaScript)

Couple's therapy (passing arguments on)

Are you still running log-images? Stop that! Type ctrl and c in the terminal to kill that command.

Now in package.json we're going to update that command so it does something more useful. Change:

"log-images": "onchange \"src/img\" -- echo '{{event}} to {{file}}'"

... to:

"watch-images": "onchange \"src/img\" -- node tools/image-compress.js {{file}} {{event}}"

This works exactly the same as before, but instead of echoing to the terminal, we're passing the {{file}} and {{event}} arguments to our script. Let's see if it works.

In the terminal, type

npm run watch-images

Now return to my example image page and download the second image again. Save it to src/img and just leave the name how it is.

You should see this in the terminal:

src\img\rBZtOtY.png add
src\img\rBZtOtY.png change

This might not seem like much progress, but we're using the power of Node to see a live update of what's happening on your hard drive. But this time we can manipulate it with JavaScript.

Hang on, what's doing the image optimisation exactly?

The package we're going to install to do the optimisation is called sharp. It's quite big and complicated, so it might take a little while to finish. Let's install it! Cancel out the watch-images command which is currently running and in the terminal, type:

npm i -D sharp

This should update the devDependencies node in package.json with a reference to sharp.

Import sharp into image-compress.js and see if it works. At the top of the file, add this line:

import sharp from "sharp";

Now let's call sharp and point it as the filePath variable. Your whole file should look like this:

import { argv } from "node:process";
import sharp from "sharp";

const [node, thisFile, filePath, fileEvent] = argv;

console.log(filePath, fileEvent);

sharp(filePath)
  .webp()
  .toFile(`./dist/img/filename.webp`);

---

Chaining:

Some functions can be "chained" together. This means that the output from one is passed to the next. So the Sharp call above could also be laid out like this:

sharp(filePath).webp().toFile(`./dist/img/filename.webp`);

It's a little harder to read so usually we indent the chained functions by one tab stop, like the original example.

---

Let's run the task again. In the terminal, type

npm run watch-images

Now try taking any example image and saving it to src/img. Give it a new name which doesn't currently exist inside the directory.

If everything's worked, you should see a new file inside your dist/img folder called filename.webp. Your first "next generation" image! You should be pleased.

Sensible naming conventions

But we can't call all of our images filename, that would be silly. Luckily, Node comes with a whole suite of tools called path to help us manipulate filepaths and filenames. Let's pull it into the top of image-compress.js like we did with sharp:

import path from "node:path";

Now let's see what we're working with. Change image-compress.js so it looks like this:

import { argv } from "node:process";
import path from "node:path";
import sharp from "sharp";

const [node, thisFile, filePath, fileEvent] = argv;

// The path from the root of the Node application to the filename of the image
const dirName = path.dirname(filePath);
// The image name, plus file extension
const baseName = path.basename(filePath);
// The image file extension
const extName = path.extname(filePath);

console.log('Arguments:', filePath, fileEvent);
console.log('Paths: ', dirName, baseName, extName);

sharp(filePath)
  .webp()
  .toFile(`./dist/img/filename.webp`);

Quick reminder of where we are:

• onchange is passing us the path to a file which has changed, plus an event, which we're not using yet
• We're loading the path into our script file
• We're using Node to split this path into different parts
• Those parts are being console.log()ed to the terminal, along with the original path

Testing the logs

Cancel the watch-images task, then immediately restart it (just in case Node is cacheing an old version).

---

Tip

Your terminal has a sort of history. If you want to re-use a command you've typed in the past, you can press the up arrow key (↑) on your keyboard to cycle through all the old commands you've entered. Press up and down until you find npm run watch-images again, then hit return.

---

Rename an existing image inside src/img to a new name (doesn't matter what). You should see an error:

Error: Input file is missing: src\img\rBZtOtY.png

What's going on here?

Right now, this code is running every time onchange detects a change inside src/img. This error is triggered because sharp can't find an image we're telling it to optimise.

Comment-out sharp, so we can see what's going on. Commenting-out can look like this:

// sharp(filePath)
//   .webp()
//   .toFile(`./dist/img/filename.webp`);

Try renaming the file again and look in the terminal. This is what I saw:

Arguments: src\img\changed.png add
Paths:  src\img changed.png .png
Arguments: src\img\temp.png unlink
Paths:  src\img temp.png .png

Remember, we've got two different console.log()s happening, but they seem to be happening twice. Why is this?

The first set of logs happen during an add event. The second happen during an unlink event. It looks like unlink is the same as deleting a file. So when I renamed a file, what really happened was that the old file was copied and added into the directory under a new name, then the old file was deleted.

We need to only run our code when the right kind of event happens inside src/img. Save another image from the example image page but this time, save it over an existing image which isn't the same image. You should see logs which look something like this:

Arguments: src\img\changed.png change
Paths:  src\img changed.png .png

When we did that, we got a change event. So the two events we need to watch out for are change and add. Let's store them in an Array, for safe keeping:

// White-list of events which should cause Sharp to generate images
const triggerEvents = ['add', 'change'];

Put this variable after the import statements. Now we need to corden off our call to sharp behind an if statement. This should only run if the fileEvent is present in our new triggerEvents Array. So the if statement should look like this:

if (triggerEvents.includes(fileEvent)) {
  sharp(filePath)
    .webp()
    .toFile(`./dist/img/filename.webp`);
}

(it's safe to comment in sharp inside the if statement)

Windows™ woes

I have terrible news for you and I, members of The™ Windows™ Community™: those kids who use Linux, MacOS and so forth giggle at us behind our backs. You see, Windows uses a backslash (\) to represent nested directories on the filesystem. The cool kids use a forward slash (/) instead. If our code is going to run elsewhere, we need to get on board.

Change your dirName variable, so it reads:

// The path from the root of the Node application to the filename of the image
const dirName = path.dirname(filePath).replaceAll('\\', '/');

This looks like it replaces \\ with /, but it doesn't. Remember escape characters? This is another example. The first backslash tells us to ignore the fact that the parser would usually ignore a backslash and to not ignore it. It's a double negative. If you're confused right now, take strength from the thought that this is a normal reaction.

What have we achieved? Let's trigger the script again and find out! Rename an existing file to something new. Your console logs should now look something like this:

Arguments: src\img\example-02.png add
Paths:  src/img example-02.png .png
Arguments: src\img\h7zTxla.png unlink
Paths:  src/img h7zTxla.png .png

Note how the dirName variable now reads src/img. Before, it was src\img. What's nice about this change is that if the same script is run on a Linux or MacOS environment, it won't break because there won't be any back slashes to replace.

Let's get rid of the Arguments console.log(). We've learned all we can from it.

Building up a path to our distribution image

We need to change the path of a changed image - for example src/img/example.png into the path of a converted image - for example dist/img/example.webp. To do this, we need to find the image directory path (img) and the file name (example).

To get the image directory path, let's write a new function!

const trimPath = (thisPath) => {
  const newPath = thisPath.replace('src', '');
  return newPath;
};

This takes a path (for example src/img) and then uses the variable we just created - srcDirectoryName - and finds the first instance of that in the path, before replacing it with ... nothing. So it removes it.

In arrow functions, if we can squeeze the code onto one line and the function returns something, we can omit both the {} and the return statement and the function can automatically return the product. And if our arrow function has exactly one argument, we can dispense with the parenthesis around it. So our trimPath function can also be expressed as:

const trimPath = thisPath => thisPath.replace('src', '');

You might prefer the longer version for clarity and I respect that decision.

Add your favourite version of this function (only one, mind!) to image-compress.js. Let's use this function to create a new variable. Put it after the file extension variable, like this:

// The image file extension
const extName = path.extname(filePath);
// The path to the source image, minus the `src` bit
const subPath = trimPath(dirName);

To get the image filename without the file extension, do a similar trick:

// The path to the source image, minus the `src` bit
const subPath = trimPath(dirName);
// The name of the image, without the file extension
const fileName = baseName.replace(extName, '');

This takes the filename (example.png) and replaces the file extension (.png) with nothing, leaving us just the name (example).

Finally, we need to stitch together a couple of strings which we'll use more than once:

const distPath = `./dist${subPath}`;

This will give us dist/img.

OK, we've got enough scraps of string to knot together into a proper path now. Change your call to sharp so it looks like this:

sharp(filePath)
  .webp()
  .toFile(`${distPath}/${fileName}.webp`);

---

Template literals

Using the backtick character to build up string like this more flexible than the legacy method, which looks like this:

sharp(filePath)
  .webp()
  .toFile(distPath + '/' + fileName + '.webp');

Another advantage of template literals is that they can include whitespace, so you can use whole chunks of JSON data spread out down your code, if you need to. You can switch between strings and simple JavaScript expressions by encapsulating JavaScript inside ${}.

---

Get rid of your console.log() in image-compress.js. In terms of what order variables, imports and functions should appear in your file, the imports should always appear first. You shouldn't call on a variable before you've declared it. Some developers say you should declare all variables at the top of your file, but this isn't always practical (what about variables which occur inside of functions?).

I personally think that if your code includes an if statement, then moving what variables you can to inside the statement helps lower the memory footprint (as they're never put into memory, if the code isn't parsed).

This would leave us with the following code:

import { argv } from "node:process";
import path from "node:path";
import sharp from "sharp";

// Destructuring the Array from Node which includes data we need
const [node, thisFile, filePath, fileEvent] = argv;
// White-list of events which should cause Sharp to generate images
const triggerEvents = ['add', 'change'];

// If the wrong kind of event triggers this script, do nothing
if (triggerEvents.includes(fileEvent)) {

  const trimPath = thisPath => thisPath.replace('src', '');

  // The path from the root of the Node application to the filename of the image
  const dirName = path.dirname(filePath).replaceAll('\\', '/');
  // The image name, plus file extension
  const baseName = path.basename(filePath);
  // The image file extension
  const extName = path.extname(filePath);
  // The path to the source image, minus the `src` bit
  const subPath = trimPath(dirName);
  // The name of the image, without the file extension
  const fileName = baseName.replace(extName, '');
  const distPath = `./dist${subPath}`;

  sharp(filePath)
    .webp()
    .toFile(`${distPath}/${fileName}.webp`);

}

We've still plenty more to do, but let's recap what we've learned:

• We can use the onchange package to pass events from the file system to our JavaScript
• We can run arbitrary JavaScript files directly in Node - no web browser required
• Node gives us access to different methods of accessing path and filename information
• Sharp is this season's must-have image optimisation package (although perhaps that's changed now, if you are reading this in the future)

View Chapter 3 code snapshot on GitHub

Quiz