In this tutorial you'll learn how to use BodyPix, through Handsfree.js, to start and stop a YouTube video by raising up your hand ✋
The Setup
Let's start by adding our dependencies:
<head>
<!-- Handsfree dependencies -->
<link rel="stylesheet" href="https://unpkg.com/handsfree@6.1.3/dist/handsfree.css" />
<script src="https://unpkg.com/handsfree@6.1.3/dist/handsfree.js"></script>
<!-- YouTube dependencies -->
<script defer src="https://www.youtube.com/iframe_api"></script>
</head>
<body>
<div id="player"></div>
</body>
We'll then use the YouTube iframe API to instantiate a video:
/**
* This method is automatically called by the YouTube API
* @see https://developers.google.com/youtube/iframe_api_reference
*/
let player
function onYouTubeIframeAPIReady () {
player = new YT.Player('player', {
// I picked this video because it's ad free
videoId: 'AFKktZURpJ8'
})
}
Setting up BodyPix through Handsfree.js
The next step is to instantiate Handsfree with the bodypix model enabled, and then create a plugin to watch for when the hand is raised. Handsfree.js plugins are simply callbacks that run after a model does inference.
// Create one instance for each camera you plan to use. By default it uses the main webcam (the selfie camera on mobile devices)
const handsfree = new Handsfree({
models: {
// The head model is enabled by default, so lets disable it
head: {enabled: false},
bodypix: {enabled: true}
}
})
/**
* Create a plugin that stops the video when the hand is up
*/
Handsfree.use('hand.stopVideo', {
// Whether we have entered "hands have been up" state
isBeingHeld: false,
/**
* This method is called once per webcam frame by Handsfree.js
* @param {Object} The handsfree instance. Here we destructure it to just get instance.body
*/
onFrame ({body}) {
// only continue if one of the wrists are visible
if (!body.pose || !body.pose.keypoints
|| (body.leftWrist.score < 0.6 && body.rightWrist.score < 0.6)) return
// Check if one wrist is level with that shoulder
if (body.leftWrist.y <= body.leftShoulder.y || body.rightWrist.y <= body.rightShoulder.y) {
if (!this.isBeingHeld) {
this.toggleVideo()
this.isBeingHeld = true
}
} else {
this.isBeingHeld = false
}
},
/**
* Toggle the video
*/
toggleVideo () {
const state = player.getPlayerState()
if (state === 1) {
player.pauseVideo()
} else {
player.playVideo()
}
}
})
That's all there is too it!
This tutorial quickly covered how to instantiate a YouTube video using the YouTube Iframe API. We then learned how to instantiate Handsfree with the bodypix model, and check for when the hand is raised...but there's so much more you can do!
Here are some other links you might find useful:
- The bodypix helpers that Handsfree.js provides
- Handsfree.js GitHub
- My Twitter @HeyOzRamos
- The Demo
- Demo Source
Thanks for reading and happy coding 👋
Top comments (0)