Building an Express web App for File Uploads and Dynamic Image Processing on the fly

Guide: Building an Express web app for File Uploads and Dynamic Image Processing

In this tutorial, we will show you how to build a server with Express.js that handles file uploads and performs dynamic image processing like resizing, format conversion, and quality adjustments using Sharp.

Prerequisites

Before we begin, ensure that you have Node.js and npm installed. We will use the following libraries in this tutorial:

1. Express.js - for setting up the server.
2. Multer - for handling file uploads.
3. Sharp - for image processing.
4. CORS - to allow cross-origin requests.

---

Step 1: Setting Up the Project

Start by creating a new directory for your project:

mkdir image-upload-server
cd image-upload-server
npm init -y

This will create a new project folder and initialize a package.json file.

You can install all dependencies by running:

npm install express multer sharp cors 

Create the necessary directories

We will need two directories:

• original-image to store the original uploaded images.
• transform-image to store the processed images.

Create these directories by running:

mkdir original-image transform-image

---

Step 2: Set Up the Express Server

Now, let's set up the basic server using Express.js. Create a file called index.js in the root of your project and add the following code to set up the server:

const express = require('express');
const cors = require('cors');
const multer = require('multer');
const path = require('path');
const sharp = require('sharp');
const fs = require('fs');

const app = express();

// Middleware for CORS and JSON parsing
app.use(cors());
app.use(express.json());
app.use(express.urlencoded({ extended: true }));

This basic setup includes:

• CORS to allow cross-origin requests.
• express.json() and express.urlencoded() to parse incoming request data.

---

Step 3: Configure Multer for File Uploads

We will use Multer to handle file uploads. Multer allows us to store uploaded files in a specified directory.

Add the following code to configure Multer:

// Configure multer for file storage
const storage = multer.diskStorage({
  destination: function (req, file, cb) {
    cb(null, 'original-image'); // Ensure the 'original-image' directory exists
  },
  filename: function (req, file, cb) {
    const uniqueSuffix = Date.now() + '-' + Math.round(Math.random() * 1E9);
    cb(null, file.fieldname + '-' + uniqueSuffix + path.extname(file.originalname));
  }
});

const upload = multer({ storage: storage });

This setup ensures that:

• The uploaded files are stored in the original-image folder.
• Each file gets a unique name based on the current timestamp and a random number.

---

Step 4: Create the File Upload Endpoint

Next, create a POST endpoint for file uploads. The user will send a file to the server, and the server will store the file in the original-image directory.

Add the following code to handle the file upload:

// File upload endpoint
app.post('/upload', upload.single('file'), (req, res) => {
  const file = req.file;
  if (!file) {
    return res.status(400).send({ message: 'Please select a file.' });
  }
  const url = `http://localhost:3000/${file.filename}`;

  // Store file path with original filename as the key
  db.set(file.filename, file.path);

  res.json({
    message: 'File uploaded successfully.',
    url: url
  });
});

This endpoint does the following:

• Receives a single file upload (with the field name file).
• Returns the URL of the uploaded file.

---

Step 5: Serve the Uploaded Files

Now, let's create a GET endpoint to serve the uploaded files. If any query parameters are provided (for example, resizing, format conversion), the server will process the image accordingly.

Add the following code to serve the uploaded files:

// In-memory storage for file paths
const db = new Map();
const processed = new Map();

// Ensure the transform-image directory exists
const transformedDir = path.join(__dirname, 'transform-image');
if (!fs.existsSync(transformedDir)) {
  fs.mkdirSync(transformedDir);
}

app.get('/:filename', async (req, res) => {
  const filename = req.params.filename;
  const { h, w, f, q } = req.query;
  const filePath = db.get(filename);

  if (!filePath) {
    return res.status(404).send({ message: 'File not found.' });
  }

  // Generate a unique key for the processed image based on the parameters
  const formateUrl = `http://localhost:3000/${filename}?h=${h}&w=${w}&f=${f}&q=${q}`;
  let editPath = processed.get(formateUrl);

  if (editPath) {
    // Serve cached processed image if it exists
    return res.sendFile(path.resolve(editPath));
  } else if (h || w || f || q) {
    // Process image if resizing or format/quality adjustments are specified
    editPath = await processImage(filePath, h, w, f, q);
    if (editPath) {
      processed.set(formateUrl, editPath);
      return res.sendFile(path.resolve(editPath));
    }
  }

  // Serve the original file if no processing is required
  res.sendFile(path.resolve(filePath));
});

This endpoint:

• Retrieves the file from the db map based on the filename.
• Processes the image if resizing, format conversion, or quality adjustments are specified.
• Caches the processed images to improve performance.

---

Step 6: Process Images with Sharp

The Sharp library will allow us to perform various transformations on the images, such as resizing, format conversion, and quality adjustments.

Add the processImage function that handles these transformations:

async function processImage(filePath, h, w, f, q) {
  try {
    const transformer = sharp(filePath);

    // Apply resizing only if `h` or `w` is provided
    const resizeOptions = {};
    if (h) resizeOptions.height = parseInt(h);
    if (w) resizeOptions.width = parseInt(w);
    if (Object.keys(resizeOptions).length > 0) {
      transformer.resize(resizeOptions);
    }

    // Apply format conversion if `f` is provided and supported
    if (f) {
      switch (f.toLowerCase()) {
        case 'jpeg':
        case 'jpg':
          transformer.jpeg({ quality: q ? parseInt(q) : 80 });
          break;
        case 'png':
          transformer.png({ quality: q ? parseInt(q) : 80 });
          break;
        case 'webp':
          transformer.webp({ quality: q ? parseInt(q) : 80 });
          break;
        case 'gif':
          transformer.gif(); // GIF format doesn’t support quality adjustment
          break;
        case 'tiff':
          transformer.tiff({ quality: q ? parseInt(q) : 80 });
          break;
        case 'avif':
          transformer.avif({ quality: q ? parseInt(q) : 80 });
          break;
        default:
          throw new Error('Unsupported format');
      }
    }

    // Save processed file to the `transform-image` directory
    const extension = f ? `.${f}` : path.extname(filePath);
    const processedFilePath = path.join(
      transformedDir,
      `processed-${Date.now()}${extension}`
    );
    await transformer.toFile(processedFilePath);

    return processedFilePath;
  } catch (error) {
    console.error('Error processing image:', error);
    return null;
  }
}

This function:

• Resizes the image based on the h (height) and w (width) parameters.
• Converts the image format based on the f parameter (JPEG, PNG, WebP, etc.).
• Adjusts the image quality based on the q parameter (optional).
• Saves the processed image in the transform-image folder.

---

Step 7: Start the Server

Finally, start the server by adding the following code:

app.listen(3000, () => {
  console.log('Server is running on port 3000');
});

This will start the server on port 3000.

---

Step 8: Testing the Server

1. Testing File Upload with Postman

To test the file upload functionality using Postman, follow these steps:

1.1 Open Postman

Launch Postman on your computer. If you don't have Postman installed, you can download it here.

1.2 Create a POST Request

• Set the request type to POST.
• In the URL field, enter: http://localhost:3000/upload.

1.3 Add the File in the Body

• Select the Body tab.
• Choose the form-data option.
• In the form, set the key to file (this must match the field name in your multer configuration).
• Click the Choose Files button and select an image file from your computer.

1.4 Send the Request

• Click Send.
• If the upload is successful, you should receive a response with the URL of the uploaded image.

Example Response:

{
  "message": "File uploaded successfully.",
  "url": "http://localhost:3000/somefile-123456789.jpg"
}

---

2. Testing Image Retrieval and Processing via Browser

Now, let's test retrieving the image with transformations using the Browser.

2.1 Get the Uploaded Image

To retrieve the image, simply open your browser and navigate to the URL you received after uploading the file. For example, if the response URL was:

"http://localhost:3000/somefile-123456789.jpg"

Just type this URL in your browser's address bar and hit Enter. You should see the original image displayed.

---

3. Testing Image Transformations with Query Parameters

Now, let's test dynamic image transformations by appending query parameters for resizing, format conversion, and quality adjustment.

3.1 Add Query Parameters for Transformation

In your browser, append query parameters to the image URL to test transformations. Here are some examples:

• Resize the image to width 200px and height 300px:

  http://localhost:3000/somefile-123456789.jpg?h=300&w=200

• Convert the image to PNG format:

  http://localhost:3000/somefile-123456789.jpg?f=png

• Convert the image to WebP format with 90% quality:

  http://localhost:3000/somefile-123456789.jpg?f=webp&q=90

• Resize the image to width 400px, height 500px, and convert to JPEG with 80% quality:

  http://localhost:3000/somefile-123456789.jpg?h=500&w=400&f=jpeg&q=80

3.2 Expected Behavior

• When you access any of the URLs with the query parameters, the server will process the image accordingly.
  • If the image has been processed before with the same parameters, it will serve the cached version.
  • If it hasn’t been processed yet, it will process the image (resize, convert format, adjust quality) and save it in the transform-image folder for future requests.

The browser will display the processed image, and you can confirm if the transformation has been applied correctly.

---

Example Workflow

1. Upload an image via Postman.
2. Retrieve the uploaded image in the browser using the URL provided by Postman.
3. Modify the URL in the browser by adding query parameters like ?h=300&w=200 to see resizing in action or ?f=webp&q=90 for format conversion.

---

Conclusion

This image upload and processing server provides a robust solution for handling image uploads, transformations, and retrievals. Using Multer for file handling and Sharp for image processing, it supports resizing, format conversion, and quality adjustments through query parameters. The system efficiently caches processed images to optimize performance, ensuring fast and responsive image delivery. This approach simplifies image management for applications requiring dynamic image transformations, making it a versatile tool for developers.