Playing Air Drums in JavaScript

#javascript #iot #creativecoding #node

Last year, as I was going on some work weekend trip, I was talking to an ex-colleague about how much I've always wanted to learn to play the drums, but never actually did because it takes a lot of space and drum sets are quite expensive.

Out of nowhere, he reached into his bag and pulled out the Freedrum sensors. I had never heard of them before but he told me about these motion sensors you attach to drum sticks and your shoes, to play air drums. You connect them to your phone or laptop via bluetooth and you just play.

It was such a coincidence that he just happened to have them in his bag and I was so excited! Not only would it give me the possibility to have some kind of a cheaper and portable drum kit but knowing it connected via bluetooth meant I had to try and hack something with it. I tried during that weekend but didn't manage to make it work.

A few weeks ago, I bought my own Freedrum kit and last weekend, I spent some time hacking away with it AND IT WORKED!! 🎉🎉🎉

If you wanna skip the rest and just look at the code, feel free to check freedrum.js.

Demo

How does it work

I've built a few projects with JavaScript, bluetooth and hardware before, so I knew that in Node.js, I could use the noble module, and directly in the browser, I could use the Web Bluetooth API.

Before starting, I did some research to see if anybody had done something like this before, or if at least, I could find some of the bluetooth specifications for the Freedrum sensors. Luckily, I found this gist but that was kind of it…

I started by checking that the info in the gist was still correct by trying to connect to the bluetooth services mentioned.

Once I saw that the uuids indicated in the gist were working, I started working on the 2 versions, the one in Node.js and the one using the Web Bluetooth API.

Web bluetooth API

The code is not much different from any Web Bluetooth API example:

const bluetoothLEMidi = '03b80e5a-ede8-4b33-a751-6ce34ec4c700';
const bleMidiCharacteristic = '7772e5db-3868-4112-a1a9-f2669d106bf3';
const uuid = "XrnS1FRG/q/kM7ecsfErcg==";

const options = {
  "filters": [
      {name: uuid},
      {services: [bluetoothLEMidi]}
  ],
};
return navigator.bluetooth.requestDevice(options)
.then(device => device.gatt.connect())
.then(server => server.getPrimaryService(bluetoothLEMidi)
.then(service => service.getCharacteristic(bleMidiCharacteristic)
.then(characteristic => characteristic.startNotifications())
.then(characteristic => {
  characteristic.addEventListener('characteristicvaluechanged', function(e){
  let data = event.target.value;
  });
 })

We start by requesting a device, in our case one of the Freedrum sensors; we connect to it, request one of its services and characteristics to get the BLE MIDI data, and subscribe to notifications so we receive the data in real time.

Node.js version

Same with the Node.js version, the code is similar to a basic noble example:

let state = {};
const sensorId = "XrnS1FRG/q/kM7ecsfErcg==";

function FreedrumsController(sensorId){
    noble.on('stateChange', function(state) {
        state === 'poweredOn' ? noble.startScanning() : noble.stopScanning();
      });

      noble.on('discover', function(peripheral){
        if(peripheral.id === sensorId){
          noble.stopScanning();
        }

        explore(peripheral);
      });

      function explore(peripheral){
        peripheral.connect(function(){
          peripheral.discoverSomeServicesAndCharacteristics([bluetoothLEMidi], [], function(error, services, characteristics){
              characteristics[0].on("read", function(event, isNotification){
                let data = event.toJSON().data;
                state = data;
                onStateChangeCallback(state);
              })
          })
        });
      }

      function onStateChangeCallback(e){
        return e;
      }

      return {
        onStateChange: function ( callback ) {
          onStateChangeCallback = callback;
        }
    }
}

We start by scanning for nearby devices, connect to the one with the right uuid, connect to the service and characteristic provided and listen to live data.

The BLE MIDI data

Once you are subscribed and receive data from the sensors, it comes back as an array of 5 integers, for example [128,128,153,60,90].

The first 2 numbers represent the header and timestamp byte. They seem to always come back with a value of 128 which is fine because we don't use them anyway. What matters to us is the 3 last values.

The 3rd value of the array represents the status byte or MIDI command. Basically, 153 means noteOn when a note should be played and 137 is noteOff when a note isn't played.

The 4th value is the MIDI note itself. Depending on where you're hitting with your drum stick, a different number (note) will come back and you can use that to trigger different sounds.

The last value represents the velocity, between 0 and 127, that can be used to set the volume.

And that's pretty much it! Now that you get this data in JavaScript you can trigger whatever sound and visualization you want!

Improvements

At the moment, I'm only reading the data from the sensors but I know you can also write to it if you want.

I've also only worked with the BLE MIDI service but there is also a service to get raw motion data that could be interesting to play with!

That's it! If you want to check the code, here's the repo!

Hope it helps! 💜

Top comments (19)

Frederik 👨‍💻➡️🌐 Creemers • Jul 24 '19

How did you find out the uuid for the device and service and characteristic for the device? I've thought about getting into web bluetooth, but I haven't found a device with these details well documented.

Gérôme Grignon • Aug 6 '19 • Edited

For the uuid, i made a uuid finder : bluetooth-uuid-finder.stackblitz.io/ (you can see code here if you want to see a simple implementation of web bluetooth : stackblitz.com/edit/bluetooth-uuid...)

For services and characteristic, you can use chrome://bluetooth-internals (but it's raw data).