DEV Community

Cover image for User Interactive background color remover using HTML Canvas
swimmingkiim
swimmingkiim

Posted on • Edited on

User Interactive background color remover using HTML Canvas

#html #canvas #remove #background

Intro

For my personal project, I needed to implement frontend side image background remover and color remover using javascript. In this post, I would like to share how I did it.
You can see a full demo in below link.

https://swimmingkiim.github.io/color-background-remover/

Image description

index.html settings

First, you need to set your index.html and import necessery packages from unpkg.

<body>
  Color Remover
  <button id="drawing-mode">remove</button>
  <button id="remove-background">remove background</button>
  <div id="color-box"></div>
  <canvas id="canvas"></canvas>
  <script src="https://unpkg.com/@tensorflow/tfjs-core@3.3.0/dist/tf-core.js"></script>
  <script src="https://unpkg.com/@tensorflow/tfjs-converter@3.3.0/dist/tf-converter.js"></script>
  <script src="https://unpkg.com/@tensorflow/tfjs-backend-webgl@3.3.0/dist/tf-backend-webgl.js"></script>
  <script src="https://unpkg.com/@tensorflow-models/deeplab@0.2.1/dist/deeplab.js"></script>
  <script src="script.js"></script>
</body>
Enter fullscreen mode Exit fullscreen mode
  1. #drawing-mode → mode changing button
  2. #remove-background → remove image’s background button
  3. #color-box → display the color that current mouse cursor points
  4. tfjs packages → need for removing image’s background
  5. deeplab package → need for using ready-to-use model for masking image

1. Remove image background with deeplab(tfjs)

This part, I got a lot of help from the first link in References. I didn’t changed much. So I’ll just explain one by one.

1. set variables 


let model;
const segmentImageButton = document.getElementById("remove-background");
segmentImageButton.style.display = "none";
const canvas = document.getElementById("canvas");
const originalImageCanvas = document.createElement("canvas");
const maskCanvas = document.createElement("canvas");
const ctx = canvas.getContext('2d');
const originalCtx = originalImageCanvas.getContext('2d');
const maskCtx = maskCanvas.getContext('2d');
Enter fullscreen mode Exit fullscreen mode

In order to remove image’s background, you need a model from deeplab. Deeplab is providing a model to segment image. Also, you need to prepare a mask canvas that which is not displayed on the screen, but helper canvas to record a background data.

2. load ready-to-use model from deeplab 

async function loadModel(modelName) {
    model = await deeplab.load({ "base": modelName, "quantizationBytes": 2 });
    segmentImageButton.style.display = "inline-block";
}
Enter fullscreen mode Exit fullscreen mode

In deeplab, there’re three possible mode for image segmentation. For this case, I would suggest you a ‘pascal’ mode. The model name is ‘pascal’ in all lowcase. Once you load a model, then you can display remove background botton. You need to do it like this because if the user trying to remove background before model is loaded, there’ll be an error.

3. predict(segment image to get the mask) 

async function predict() {
    let prediction = await model.segment(image);
    renderPrediction(prediction);
}

function renderPrediction(prediction) {
    const { height, width, segmentationMap } = prediction;

    const segmentationMapData = new ImageData(segmentationMap, width, height);
    canvas.width = width;
    canvas.height = height;
    ctx.drawImage(image, 0, 0, width, height);
    maskCanvas.width = width;
    maskCanvas.height = height;
    maskCtx.putImageData(segmentationMapData, 0, 0);
    removeBackground([0,0,0], width, height);
}
Enter fullscreen mode Exit fullscreen mode

Now, you can segment your image. The type of image is HTMLImage. The segment function returns a result data. In that data, you need width, height and segmentationMap fields. Width and height are the size of segmented image. The segmentationMap refers to the image data of segmented image. If you draw this by using putImageData method on a canvas, you’ll see a combinations of colors representing different objects. In pascal mode, background is black color. So you can use this to make a mask. First, draw a segmentationMap on the maskCanvas.

4. remove background using segment data 

function removeBackground(color, width, height){
  image.width = width;
  image.height = height;
  originalImage.width = width;
  originalImage.height = height;
  canvas.width =width;
  canvas.height =height;
  originalImageCanvas.width =width;
  originalImageCanvas.height =height;
  ctx.drawImage(image, 0,0,width,height);
    var canvasData = ctx.getImageData(0, 0, width, height),
        pix = canvasData.data;
  var maskCanvasData = maskCtx.getImageData(0, 0, width, height),
        maskPix = maskCanvasData.data;

    for (var i = 0, n = maskPix.length; i <n; i += 4) {
        if(maskPix[i] === color[0] && maskPix[i+1] === color[1] && maskPix[i+2] === color[2]){
             maskPix[i+3] = 0;   
             pix[i+3] = 0;
        }
    }

    ctx.putImageData(canvasData, 0, 0);
    maskCtx.putImageData(maskCanvasData, 0, 0);
    const base64 = canvas.toDataURL();
    image.src = base64
    originalImage.src = originalImage.src;
}
Enter fullscreen mode Exit fullscreen mode

In this part, I got help from second link in the References.

First, draw a current image on the displayed canvas, original image to other canvas. Since the size of an image will be changed after segmentation, you need to redraw your original(for backup) image to update.

Second, get Image data from canvas and mask canvas. And loop all mask canvas’s pixels. If you find black pixel on a mask canvas make it transparent. the pixel data array is an array of rgba number in each pixel, so you need to jump 4 at a time.

Third, redraw updated image on the canvas and mask canvas. Retrieve data url from canvas and update current image’s src.

2. Select color part according to mouse position

Now, you need to implement interactive color remover.

1. set variables & image 

const canvas = document.getElementById("canvas");
const originalImageCanvas = document.createElement("canvas");
const modeButton = document.getElementById("drawing-mode");
const ctx = canvas.getContext('2d');
const originalCtx = originalImageCanvas.getContext('2d');
const colorBox = document.getElementById("color-box");
let mode = "remove";
colorBox.style.width = "20px";
colorBox.style.height = "20px";
colorBox.style.backgroundColor = "black";
const selectColorData = [19, 163, 188];
const removeColorData = [255,255,255, 0];
let originalImage;
let image;
let imageSize = {
  width: 500,
  height: 500,
}
const brushSize = 20;
let circle = new Path2D();
circle.stroke = `rgb(${selectColorData.join(",")})`;
const imageSrc = "https://images.unsplash.com/photo-1648199840917-9e4749a5047e?ixlib=rb-1.2.1&ixid=MnwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8&auto=format&fit=crop&w=687&q=80";

const canvasOffset = {
  x: canvas.getBoundingClientRect().left,
  y: canvas.getBoundingClientRect().top,
}
Enter fullscreen mode Exit fullscreen mode
  1. colorBox → for displaying current color of mouse cursor points at.
  2. selectColorData → single rgb value array for display selected area.
  3. removeColorData → single rgba value array with alpha to 0. (for removing background)
  4. imageSize → Maxium size of initial image size.
  5. brushSize → used when you remove or recover(heal) original area
  6. cicle → for brush
  7. canvasOffset → important when calculating cursor position relative to canvas itself. 
const setCanvas = () => {
  const _image = new Image();
  _image.onload = () => {
    image = _image.cloneNode();
    image.onload = null;
    originalImage = _image.cloneNode();
    imageSize = {
      width: Math.min(_image.width, imageSize.width),
      height: Math.min(_image.width, imageSize.width) * (_image.height /_image.width)
    }
    canvas.width = imageSize.width;
    canvas.height = imageSize.height;
    originalImageCanvas.width = imageSize.width;
    originalImageCanvas.height = imageSize.height;
    image.width = imageSize.width;
    image.height = imageSize.height;
    _image.width = imageSize.width;
    _image.height = imageSize.height;
    originalImage = _image.cloneNode();
    ctx.drawImage(image, 0,0, image.width, image.height);
    originalCtx.drawImage(_image, 0,0, _image.width, _image.height);
    console.log(originalImageCanvas)
  }
  _image.crossOrigin = "anonymous";
  _image.src = imageSrc;
}
Enter fullscreen mode Exit fullscreen mode

When your image is loaded, there’re some works to be done. You need to initialize canvas and original canvas(not change after this, it’s a reference for recovering image) and their src(HTML Image element). Finally, draw the image on canvas and original canvas.

3. Remove certain color from HTML Canvas 

const isInColorRange = (targetColor, compareTo, i) => {
  return (
    targetColor[i] >= compareTo[0] - 10
    && targetColor[i] <= compareTo[0] + 10
    && targetColor[i+1] >= compareTo[1] - 10
    && targetColor[i+1] <= compareTo[1] + 10
    && targetColor[i+2] >= compareTo[2] - 10
    && targetColor[i+2] <= compareTo[2] + 10
  )
}

const selectColor = (colorData) => {
    let canvasData = ctx.getImageData(0, 0, canvas.width, canvas.height),
        pix = canvasData.data; 
    for (let i = 0, n = pix.length; i <n; i += 4) {
        if(isInColorRange(pix, colorData, i)){ 
          pix[i] = selectColorData[0];
          pix[i+1] = selectColorData[1];
          pix[i+2] = selectColorData[2];
        }
    }
    ctx.putImageData(canvasData, 0, 0);
}

const removeColor = (colorData) => {
    let canvasData = ctx.getImageData(0, 0, canvas.width, canvas.height),
        pix = canvasData.data;  
    for (let i = 0, n = pix.length; i <n; i += 4) {
        if(isInColorRange(pix, colorData, i)){ 
             pix[i] = removeColorData[0];
             pix[i+1] = removeColorData[1];
             pix[i+2] = removeColorData[2];
             pix[i+3] = removeColorData[3];
        }
    }
  ctx.putImageData(canvasData, 0, 0);
}
Enter fullscreen mode Exit fullscreen mode

It’s just like removing background with tfjs(deeplab). Get pixel data and loop, find, change them. Note that isInColorRange is customizable you can change 10 to other number and adjust for you project.

4. Recover original image to HTML Canvas 

const healArea = (position) => {
  ctx.clearRect(0,0, image.width, image.height);
  ctx.drawImage(originalImage, 0,0, originalImage.width, originalImage.height);
  ctx.globalCompositeOperation = "destination-in";
  ctx.moveTo(position.x, position.y);
  circle.moveTo(position.x, position.y);
  circle.arc(position.x, position.y, brushSize/2, 0, 2 * Math.PI);
  ctx.fill(circle);
  ctx.globalCompositeOperation = "source-over";
  ctx.drawImage(image, 0,0, image.width, image.height);
}

const removeArea = (position) => {
  ctx.clearRect(0,0, image.width, image.height);
  ctx.drawImage(image, 0,0, image.width, image.height);
  ctx.globalCompositeOperation = "destination-out";
  ctx.moveTo(position.x, position.y);
  circle.moveTo(position.x, position.y);
  circle.arc(position.x, position.y, brushSize/2, 0, 2 * Math.PI);
  ctx.fill(circle);
  ctx.globalCompositeOperation = "source-over";
}
Enter fullscreen mode Exit fullscreen mode

Also, above two fuction’s logic is very simular too. The difference is that there’re using different globalcCompositionOperation. Removing area is using “destination-out” to remove the area of its own, and healing area is using “destination-in” to draw only area of circle itself and remove else.

5. Register mouse events

Finally, combine all those functions properly in mouse move, donw and up events. I won’t go on detail. I don’t think the code below contains any special new approch.

1. utilities 

const detectColor = (position) => {
  const colorData = ctx.getImageData(
    position.x, 
    position.y,
    1,
    1
  ).data;
  return colorData;
}

const changeColorBoxColor = (colorData) => {
  const rgba = `rgba(${colorData.join(",")})`;
  colorBox.style.backgroundColor = rgba;
}
Enter fullscreen mode Exit fullscreen mode

2. onMouseMove(without mouse down) 

const onMouseMoveSelectColor = (e) => {
    ctx.drawImage(image, 0,0, image.width, image.height);
    const position = {
      x: e.clientX - canvasOffset.x,
      y: e.clientY - canvasOffset.y,
    }
    const color = detectColor(position);
    changeColorBoxColor(color);
    if (mode === "remove by color") {
      selectColor(
        color, 
        position.x, 
        position.y
      );
    } else {
      selectArea(
        position.x, 
        position.y
      );
    }
  }
Enter fullscreen mode Exit fullscreen mode

3. onMouseMove(with mouse down) 

const onMouseDownAndMoveRemoveColor = (e) => {
    const callback  = (e) => {
    const position = {
      x: e.clientX - canvasOffset.x,
      y: e.clientY - canvasOffset.y,
    }
    const color = detectColor(position);
  if (mode === "remove by color") {
    removeColor(
      color, 
      position.x, 
      position.y
    )
  } else if (mode === "heal area") {
    healColor(
    position,
  );
  } else {
    removeArea(position);
  }
  }
    canvas.onmousemove = callback;
    callback(e);
}
Enter fullscreen mode Exit fullscreen mode

4. Register listeners 

const toggleMode = (e) => {
  if (e.target.innerText === "remove by color") {
    mode = "heal area";
  } else if (e.target.innerText === "heal area") {
    mode = "remove area";
  } else {
    mode = "remove by color";
  }
  e.target.innerText = mode;
}

const registerListener = () => {
  canvas.onmousemove = onMouseMoveSelectColor;
  canvas.onmousedown = onMouseDownAndMoveRemoveColor;
  canvas.onmouseup = (e) => {
    canvas.onmousemove = null;
    canvas.onmousemove = onMouseMoveSelectColor;
    image.src = canvas.toDataURL();
  };
  canvas.onmouseleave = (e) => {
    canvas.onmousemove = null;
    canvas.onmousemove = onMouseMoveSelectColor;
    ctx.drawImage(image, 0,0, image.width, image.height);
  }
  modeButton.onclick = toggleMode;
}
Enter fullscreen mode Exit fullscreen mode

Conclusion

I’m planning to implement these in my image editor project. If anyone interested to see my project implementation, please checkout below link.

https://online-image-maker.com/

Image description

Cheers!

Buy Me A Coffee

References

Top comments (0)

Read next

florianrappl profile image

Sticky Transitioning Veggie Hamburger 🍔

Florian Rappl -

bogdanfromkyiv profile image

7 Old-School Practices in HTML Should Be Avoided

Bogdan Bendziukov -

shieldstring profile image

Elements and Attributes: Understanding HTML elements and their attributes

Agbo, Daniel Onuoha -

ridoy_hasan profile image

Best Practices for Writing CSS – Clean, Scalable, and Maintainable Code

Ridoy Hasan -