Chapter 4: images part two

Cover image by Yura Forrat

Recap

Last time, we started to look at optimising images:

• We used the onchange package to pass events from the file system to our JavaScript
• We ran arbitrary JavaScript files directly in Node - no web browser required
• We saw that Node gave us access to different methods of accessing path and filename information
• We learned that Sharp is this season's must-have image optimisation package

There's still a lot to do before we meet our goals set out at the start of the previous lesson. Here's the features we wanted to build:

• Drop an image into the src/img directory and it automatically gets converted into a next-gen format in the dist/img directory
• Run a task which converts all images in the src/img directory into their optimised versions in dist/img - for example when you've added some images but the server wasn't running at the time
• Overwrite an existing image in src/img with a new one with the same name, and have the corresponding images in dist/img update automatically
• Use as many different sub-directories in src/img and have those be replicated on dist/img - for example src/img/gallery/open-day which lives inside dist as dist/img/gallery/open-day

Let's continue the build!

Nested directories

Our script works perfect, but only if the image in question is inside src/img. What about if we wanted to create sub-directories inside src/img, such as src/img/gallery?

Let's see what happens. In a terminal, run:

npm run watch-images

Now Create a new folder inside src/img and call it gallery. Copy one of the images and paste it into there.

I got an error in the terminal which read:

Error: ./dist/img/gallery/example-01.webp: unable to open for write
windows error: The system cannot find the file specified.

sharp tried to write the optimised image into a new folder called gallery which lives inside dist/img - exactly as we'd hoped. But it couldn't find the directory. Let's give the script the power to create directories.

For this, we'll need to lean on another part of Node, the fs module. fs stands for "file system" and not a rude exclamation you might use when your code doesn't work.

import fs from 'node:fs';

Just like before, we're going to use two functions built into fs. The first, existsSync(), takes a parameter which is a path. It returns true or false depending if the path exists or not.

The second function is called mkdirSync and allows us to make one or more directories in the path we supply it.

We can use these two functions inside your if statement, between where the variables are initialised and the calls to sharp, like this:

if (!fs.existsSync(distPath)) {
  fs.mkdirSync(distPath, {recursive: true});
}

Guess what: there's a shortened form of if too, as long as the JavaScript fits onto one line. It uses the double ampersand and the cool name for it is short-circuit evaluation.

It looks like this:

true && alert("Truth detected!");

This would cause a JavaScript alert with the string Truth detected!.

However this would do nothing:

false && alert("Can you hear this?");

How this works is the JavaScript parsing engine works from left to right. It will continue until it finds something false and then skip the rest of the statement and work further down the file. So we could shorten our if statement to:

!fs.existsSync(distPath) && fs.mkdirSync(distPath, {recursive: true});

Don't go hog-wild with this technique. Remember that JavaScript needs to be readable as well as using the most up-to-date slang.

OK, but what does this statement do? First, we check if the distPath we want to use exists or not. If it doesn't exist, we call mkdirSync on it, while passing a recursive boolean in the option object. This will recursively call mkdirSync over and over again, until all the directories are created.

Testing out nested directories

Let's try this out. Cancel the watch-images task and create some nested folders inside src/img, for example src/gallery/1/2/3. Restart the watch-images task and drop an image inside the 3 directory.

Now check what's inside the dist/img directory. You should see a mirror of the src directory structure, except now your example file is a webp rather than a png. Great! We can cross off this requirement:

• Use as many different sub-directories in src/img and have those be replicated on dist/img - for example src/img/gallery/open-day which lives inside dist as dist/img/gallery/open-day

Adding watch-images to our start task

Cancel the watch-images task using ctrl + c. Let's edit our old pal package.json and introduce watch-images to the start krew:

{
  ...
  "scripts": {
    ...
    "start": "concurrently \"npm run serve\" \"npm run sass-dev\" \"npm run watch-images\"",
    ...
  }
}

Lovely.

Check if the image changes

From a terminal, run:

npm start

Your example web page should open and accuse you of being a worm. Let's add an image.

Open the file dist/index.html. Add a paragraph before the closing body tag so that the whole file looks like this:

<!doctype html>
<html class="no-js" lang="en-GB">

<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>Hello Worm</title>
  <link rel="stylesheet" href="/css/main.css">
</head>

<body>
  <p>Hello Worm</p>
  <p><img src="/img/example-01.webp" alt="An animal, yesterday">></p>
</body>

</html>

You will hopefully have a few different images inside dist/img by this point, none of which might be called example-01.webp so update the img tag so it points to an image which actually exists. Save the file.

Let's update the image! Visit the example image page and download a different image from the one which is currently displayed on your page. Use this image to overwrite example-01.png (or whatever) in your src/img directory. You should see your image-compress.js script kick into action. And once it's finished, you should see the browser refresh and the displayed image should update.

This happens "automagically" because we've already configured browser-sync to watch the dist directory and to refresh when something happens in there.

Missing task

Just like with the CSS, our hypothetical second developer will inherit this codebase and an empty dist/img directory.

We need to write a task which will go through all the images in src/img and optimise them. As the site grows in complexity, this isn't something we'd want to do every time we start the local server, because that would take ages. But it should run at least once, when a new developer downloads the project.

Let's start by grabbing a list of all the src images.

Get all files

We need to write a function which pulls a list of file names from an arbitrary directory. Inside the tools folder, create a new file called get-files.js. It should look like this:

// Import the "file system" function, which is built into Node.js
import fs from 'fs';

export default function getFiles(dir, files = []) {
  // Get an array of all files and directories in the passed directory using fs.readdirSync
  const fileList = fs.readdirSync(dir);
  // Create the full path of the file/directory by concatenating the passed directory and file/directory name
  for (const file of fileList) {
    const name = `${dir}/${file}`;
    // Check if the current file/directory is a directory using fs.statSync
    if (fs.statSync(name).isDirectory()) {
      // If it is a directory, recursively call the getFiles function with the directory path and the files array
      getFiles(name, files);
    } else {
      // If it is a file, push the full path to the files array
      files.push(name);
    }
  }
  return files;
};

I stole this from Learn with Param. Does this make me a great artist? That's not for me to say.

Use getFiles() to pull in a list of the images

Create a third file in our new tools directory called compress-all-images.js. It should look like this:

import getFiles from "./get-files.js";

console.log(getFiles('./src/img'));

This does exactly two things:

1. Imports the function get-files.js from the same directory
2. Console-logs the output of that function, if it's pointed at ./src/img

Save any missing images from the example page into src/img, just so we have a good supply.

In the terminal type:

node tools/compress-all-images.js

You should see an output something like this inside your terminal:

[
  './src/img/example-01.png',
  './src/img/example-02.png',
  './src/img/example-03.png',
  './src/img/example-04.png',
  './src/img/example-05.png',
  './src/img/gallery/1/2/3/example-01.png']

Those square brackets mean it's an Array. A nice way to work with Arrays is to call the map method on them. This means that map goes through each item in an Array and runs the same function on it (a function we're just about to define). Let's store that data in an variable, then iterate over it with map():

import getFiles from "./get-files.js";

const allImagePaths = getFiles('./src/img');

allImagePaths.map((path) => {
  console.log(path);
});

Our new function is passed as a parameter to map and we're writing it inline, as an arrow function. The function receives an automatic parameter which I've chosen to call path. In the example above, the first value of path will be ./src/img/example-01.png.

Call the JavaScript file from the terminal a second time, using the up arrow, if you're in a hurry.

As expected, this console.log()s the same data, but as individual lines. Now we just need to replicate a bunch of code from image-compress.js in this file, right?

Let's see if we can split this reused code off into different modules, so we don't write the same code twice.

Making image-compress.js more modular

A lot of image-compress.js is concerned with getting the right dist path and extracting the filename from a path. Let's pull all of that code into a new file inside tools called get-dist-path.js:

It should look like this:

import path from "node:path";

export default function getDistPath(filePath) {

  const trimPath = thisPath => thisPath.replace('src', '');

  // The path from the root of the Node application to the filename of the image
  const dirName = path.dirname(filePath).replaceAll('\\', '/');
  // The image name, plus file extension
  const baseName = path.basename(filePath);
  // The image file extension
  const extName = path.extname(filePath);
  // The path to the source image, minus the `src` bit
  const subPath = trimPath(dirName);
  // The name of the image, without the file extension
  const fileName = baseName.replace(extName, '');
  const distPath = `./dist${subPath}`;
  return {
    distPath,
    fileName
  }
};

---

Shorthand property and method names in JavaScript

You might have notices some weird return syntax there:

return {
  distPath,
  fileName
}

This is a shorthand form of an object in JavaScript. It's the same as this:

return {
  "distPath": distPath,
  "fileName": fileName
}

Where the name and the value are the same (even if one is a variable), you can skip the value part in JavaScript (not in JSON!)

---

The getDistPath function is passed a src path and uses this to work out what the equivalent dist path should be.

But there's still code in image-compress.js which just calls Sharp, which would be common to compress-all-images.js. So let's split that out into a new file too. Inside the tools directory, create a new file called write-images.js. It should look like this:

import fs from 'node:fs';
import sharp from "sharp";
import getDistPath from "./get-dist-path.js";

export default function writeImages(filePath) {
  const { distPath, fileName } = getDistPath(filePath);

  !fs.existsSync(distPath) && fs.mkdirSync(distPath, {recursive: true});

  sharp(filePath)
    .webp()
    .toFile(`${distPath}/${fileName}.webp`);

};

This means we can import the function writeImages into image-compress.js and then get rid of a lot of code:

import { argv } from "node:process";
import writeImages from "./write-images.js";

// Destructuring the Array from Node which includes data we need
const [node, thisFile, filePath, fileEvent] = argv;
// White-list of events which should cause Sharp to generate images
const triggerEvents = ['add', 'change'];

// If the wrong kind of event triggers this script, do nothing
if (triggerEvents.includes(fileEvent)) {
  writeImages(filePath);
}

Because the only thing which differs between compress-all-images.js and image-compress.js is how they are called and if the web browser is running or not.

---

Breaking down code

It might feel a bit as if we've been shuffling code about between a whole lot of files in a pretty arbitrary way here. Indeed other programmers might choose to slice this code up differently. Splitting a long piece of code into different modules is an example of separation of concerns and is quite a challenging topic to teach!

The approach we've stumbled into here had us writing one big script which did everything, then realising we'd need to write a second script which did 90% the same thing.

We could, of course, just duplicate the first script and make some changes to it. But breaking it into modules will help us in all kinds of ways in the future.

One way to work out how to split up a long script up is to add comments before each line explaining what it does. Then see if groups of those comments use the same words, suggesting different "concerns".

Another way is to look at inputs and outputs. In our example, we had a lot of code which just edited strings to normalise them and translate them from the src to the dist directories. Not only does this code belong together, it's easy to see how useful it might be in the future for other purposes.

The approach we've taken here where we write the code first, then refactor it later is normal. Don't expect everything to be perfect right away. We can always improve things later.

---

Finally, let's import writeImages.js into compress-all-images.js. We need to slightly trim the paths, to remove the ./ at the start:

import getFiles from "./get-files.js";
import writeImages from "./write-images.js";

const allImagePaths = getFiles('./src/img');

allImagePaths.map((path) => {
  const trimPath = path.replace('./', '');
  writeImages(trimPath);
});

Generating out other image formats

Now that write-images.js is common to both image-compress.js and compress-all-images.js, let's make it generate three times as many images! We won't use these other formats yet (that's coming soon) but there's no harm in outputting them ahead of time.

Open write-images.js and add more Sharp calls like this:

import fs from 'node:fs';
import sharp from "sharp";
import getDistPath from "./get-dist-path.js";

export default function writeImages(filePath) {
  const { distPath, fileName } = getDistPath(filePath);

  !fs.existsSync(distPath) && fs.mkdirSync(distPath, {recursive: true});

  sharp(filePath)
    .avif()
    .toFile(`${distPath}/${fileName}.avif`);

  sharp(filePath)
    .webp()
    .toFile(`${distPath}/${fileName}.webp`);

  sharp(filePath)
    .jpeg()
    .toFile(`${distPath}/${fileName}.jpg`)

};

Now we're generating an avif, a webp and a boring old jpg file for each image dropped into src/img.

Adding a new task to compress all images

Finally, let's add another task to our prepare command, so that all the images are generated when our second developer (we really ought to give this imaginary person a name) runs npm install.

Open up package.json and edit your prepare task, so it runs compress-all-images.js:

...
"scripts": {
    "prepare": "concurrently \"npm run sass-prod\" \"node tools/compress-all-images.js\"",
    ...
}
...

Testing out optimising all images

Delete dist/img then run npm install from the terminal. Once it's finished, you should see your img directory reappear as if by magic and be populated by each image in your src/img directory, in a mouth-watering choice of three flavours.

• Run a task which converts all images in the src/img directory into their optimised versions in dist/img - for example when you've added some images but the server wasn't running at the time

Let's review what we learned:

• Used the fs module from Node to elegantly handle nested directories inside src/img
• Learned about short-circuit evaluation in JavaScript
• Refreshed the browser automatically
• Stole a utility function from Param Harrison to get a list of files and folders in Node
• How we might split up a long, complicated script into modules
• The shorthand which can be used in JavaScript if the name and property are the same

View Chapter 4 code snapshot on GitHub

Quiz