Home > HTML5 > Google Hangout Effects using HTML5 and JavaScript facedetection

Google Hangout Effects using HTML5 and JavaScript facedetection

Google Effects

Google hangouts very popular because of Google effects like face masks and background effects.

I tried to implement Google Effects using HTML5 Canvas,HTML5 getUserMedia API via JavaScript face detection library by  Liuliu.

To simplify I divided this tutorial into two parts First getUserMedia basics and Second Face detection & Google Effects.

Please go through the getUserMedia basics then you will understand the logic clearly.

Continuous to the previous post, we learnt how to take a snapshot using getUsermedia and Canvas. Here is the Demo

button.addEventListener('click',snapshot, false);

function snapshot() {
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
ctx.drawImage(video, 0, 0);
}

 

Just remove capture button and replace onclick event with setInterval Method as shown below

setInterval(snapshot, 30);

 

The Logic is simple I am injecting local media stream video into HTML5 Canvas.For every 30msec one latest image snapshot will be drawn on the canvas. As the Time Interval is very less both video and canvas look like exactly same. Here is the Demo.

Now we will understand face detection JavaScript by LiuLiu. The library contains two js file ccv.js and face.js.

 Go to the A Not-so-slow JavaScript Face Detector, upload your image it takes some time and then an Orange color circle will be drawn on image.here is the sample demo

Face Detection Using javascript
Face Detection Using javascript
It took almost 1537ms to detect the faces.The Library is capable of detection multiple faces. I have gone through the page source actually it’s HTML5 canvas element.
  •  First Image is drawn in Canvas.
  • And then the library has one method ccv.detect_objects() which takes canvas(which contains image) element as argument and it returns array of  coordinates of faces.
  • Finally looping through the coordinates,drawing circle on canvas using context.arc method.

The code behind is

var canvas = document.getElementById("output");
var ctx = canvas.getContext("2d");
ctx.drawImage(image, 0, 0);
var comp = ccv.detect_objects(
{ 
"canvas" : ccv.grayscale(ccv.pre(image)),/*Here we can pass canvas also instead of ccv.grayscale(ccv.pre(image))*/
"cascade" : cascade,
"interval" : 5,
"min_neighbors" : 1 
});

ctx.lineWidth = 2;
ctx.strokeStyle = 'rgba(230,87,0,0.8)';
/* draw detected area */
for (var i = 0; i < comp.length; i++)
{
 ctx.beginPath();
 ctx.arc((comp[i].x + comp[i].width * 0.5) * scale, (comp[i].y + comp[i].height * 0.5) * scale,(comp[i].width + comp[i].height) * 0.25 * scale * 1.2, 0, Math.PI * 2);
 ctx.stroke();
}

 

Also Read : basics of LocalStorage API

Detect_objecta take so many arguments like cascade,interval,min_neighbours I don’t know about these arguments I have not explored much about it.I am using this detech_objects alone to detect face.

I modified the snapshot method as shown below

setInterval(snapshot, 800);
function snapshot() {
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
ctx.drawImage(video, 0, 0);
var position = ccv.detect_objects(
{ 
"canvas": canvas,
"cascade": cascade,
"interval": 2,
"min_neighbors": 1
});

 

The Logic here is

  • whenever snapshot called a new image drawn onto the canvas.
  • I am passing that canvas to detect_objects method it will return coordinates of faces.
  • And I increase interval time to 800 because detect_objects taking 1500ms to detect the face coordinates.
  • And Finally instead of drawing circle on our face we will draw Mask Image(I am using three images batman mask and Glasses with Mask,PirateCaptain).

I modified HTML as shown below

<div align="center" class="normaldiv">
    <div style="width: 90%" class="normaldiv">
        <div style="float: left;">
             <h3>
              <img src="GoogleEffects.png" alt="GoogleEffects" />
              Google Effects By Arunkumar Gudelli
             </h3>
         </div>
         <div>
                <p>
                    <input id="GlassesWithMask" onclick="selectmask(this.id)" type="button" style="background: url(glassesIcon.png);
                        background-repeat: no-repeat; width: 110px; height: 110px; cursor: pointer" />
                    <input id="BatMan" onclick="selectmask(this.id)" type="button" style="background: url(batmanIcon.png);
                        background-repeat: no-repeat; width: 110px; height: 110px; cursor: pointer" />
                </p>
            </div>
        </div>
        <video id="webcam" autoplay>
        </video>
        <canvas id="screenshot-canvas">
        </canvas>
    </div>

 

I created two buttons with Id’s GlassesWithMask and Batman which are two effects in our Google Effects Web App. Onclick we are calling selectMask method. And the Javascript is

var MaskImage = new Image();

function selectmask(source) 
{
 MaskImage.src = source + ".png";        
}

setInterval(snapshot, 30);

function snapshot() 
{
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
ctx.drawImage(video, 0, 0);

var position = ccv.detect_objects({ "canvas": canvas,
                    "cascade": cascade,
                    "interval": 2,
                    "min_neighbors": 1
                });
for (var i = 0; i < position.length; i++) 
{
 ctx.drawImage(MaskImage, position[i].x - 90, position[i].y - 150, position [i].width + 200, position[i].height + 200);
}
}
  • I declared one variable MaskImage of type Image()
  • In selectmask() method I am assigning MaskImage with id of the button.
  • And in Snapshot() method I am replacing ctx.arc with ctx.drawImage as explained above.
  • In the snapshot method, I am drawing MaksImage on each face (based on selected mask you have added or substract positions the mentioned metrics will work for most of the Masks)
  • If two people are in Webcam they will be masked with selected feature.
  • I am adding “.png” at the end. i.e., id of buttons and Images should be same so that onclick we can easily change the MaskImage
  • Refer below Image The Mentioned Images are our two Google Effects.
TheLogicBehindSelectingFeature
TheLogicBehindSelectingFeature

I Added some CSS code to style our Google Effects page.Here is the Final HTML Code.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <title>Google Effects By By Arunkumar Gudelli</title>
    <script src="http://code.jquery.com/jquery-1.10.2.min.js" type="text/javascript"></script>
    <script src="ccv.js" type="text/javascript"></script>
    <script src="face.js" type="text/javascript"></script>
    <script>

        var MaskImage = new Image();

        function selectmask(source) {
            console.log(source);
            MaskImage.src = source + ".png";
        }

        function onFailure(err) {
            alert("The following error occured: " + err.name);
        }

        jQuery(document).ready(function () {

            var video = document.querySelector('#webcam');
            var button = document.querySelector('#screenshot-button');
            var canvas = document.querySelector('#screenshot-canvas');
            var ctx = canvas.getContext('2d');

            navigator.getUserMedia = (navigator.getUserMedia ||
                            navigator.webkitGetUserMedia ||
                            navigator.mozGetUserMedia ||
                            navigator.msGetUserMedia);
            if (navigator.getUserMedia) {
                navigator.getUserMedia
                            (
                              { video: true },
                              function (localMediaStream) {
                                  video.src = window.URL.createObjectURL(localMediaStream);
                              }, onFailure);
            }
            else {
                onFailure();
            }
            //button.addEventListener('click', snapshot, false);
            setInterval(snapshot, 30);
            function snapshot() {
                canvas.width = video.videoWidth;
                canvas.height = video.videoHeight;
                ctx.drawImage(video, 0, 0);
                var position = ccv.detect_objects({ "canvas": canvas,
                    "cascade": cascade,
                    "interval": 2,
                    "min_neighbors": 1
                });
                for (var i = 0; i < position.length; i++) {
                    ctx.drawImage(MaskImage, position[i].x - 90, position[i].y - 150, position[i].width + 200, position[i].height + 200);
                }
            }
        });

    </script>
</head>
<body>
    <div align="center" class="normaldiv">
        <div style="width: 90%" class="normaldiv">
            <div style="float: left;">
                <h3>
                    <img src="GoogleEffects.png" alt="GoogleEffects" />
                    Google Effects By Arunkumar Gudelli</h3>
            </div>
            <div>
                <p>
                    <input id="GlassesWithMask" onclick="selectmask(this.id)" type="button" style="background: url(glassesIcon.png);
                        background-repeat: no-repeat; width: 110px; height: 110px; cursor: pointer" />
                    <input id="BatMan" onclick="selectmask(this.id)" type="button" style="background: url(batmanIcon.png);
                        background-repeat: no-repeat; width: 110px; height: 110px; cursor: pointer" />
<input id="PirateCaptain" onclick="selectmask(this.id)" type="button" style="background: url(PirateCaptainIcon.png);
                        background-repeat: no-repeat; width: 110px; height: 110px; cursor: pointer" />

                </p>
            </div>
        </div>
        <video id="webcam" autoplay>
        </video>
        <canvas id="screenshot-canvas">
        </canvas>
    </div>
</body>
</html>

 

Here is the Final Google Effects Demo.

You have to enable getUserMedia in your browser. That I explained in previous post If you have any queries feel free to comment.

Remember getUserMedia does’not work on file:// URL you have host html in IIS or Apache.Use Chrome or Mozilla or Opera

NOTE:

It’s not perfect. Keep Your Face straight and Close to Webcam(with Clean Background) and Remove your Eye Glasses if you have.

And Your suggestions are always welcome Please Add your valuable feedback to improve this App.

GoogleEffectsDemo
GoogleEffectsDemo

I hope you’ve enjoyed this article and that it gives you more ideas face detection and getUserMedia API.If so share this post with your friends and also join our mailing list I will be sharing my learning’s and experiments with HTML5.

NOTE: getUserMedia has been removed from the Web standards.

Wait before leaving. why can’t you follow me on twitter or be a friend on Facebook or googlePlus or linkedn to get in touch me. or join our mailing list

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Arunkumar Gudelli
I am “One among a million” Software engineers of India. I write beautiful markup.I make the Web useful. You can connect me via @twitter or @facebook or Google+ or e-mail.
http://www.arungudelli.com

One thought on “Google Hangout Effects using HTML5 and JavaScript facedetection

Lets have chat