LEAP Dev Kit Initial Impressions

I was thrilled to get my LEAP dev kit today, and even more excited that the simple code I had whipped up “just worked”.  If you’ve not followed the news on this, it is a motion capture device that tracks hands and fingers, similar to a Kinect.  Unlike the Kinect, this is designed for laptop and PC use, and is about the size of a pack of gum.

The device I received, Rev 6, is sleek brushed aluminum with a black top and bottom.  A USB cable connects it to your PC or Mac.  It’s a very nice 1st gen release, and is the same form factor that will be available at Best Buy soon.  That said, you do get the idea that this could be integrated into laptops etc. instead of or in addition to trackpads, etc. 

From a developer perspective, the provided SDK is extremely easy to work with.  I chose to focus on the JavaScript SDK for now, and have found ramp up super easy.  LEAP has abstracted away all the complex tracking and math, leaving the developer with a rich, but not overwhelming API.  Interestingly, it works by exposing a WebSocket that you can consume from your page to stream in tracking data.  Fortunately, LEAP hides all the WebSocket goo and makes this easy to implement. To work with it in JavaScript, one just has to include the leap client js, then register to receive “frames”:

 Leap.loop(function (frame) {
                    latestFrame = frame
                    self.hands(frame.hands);
                    self.fingers(frame.fingers);
                    self.tools(frame.tools);
                });

6 lines of code for 3D motion capture is pretty sweet.  Some developers might find this lacking.  Videos show off point clouds and implied other lower-level abstractions are possible.  If you’re expecting to use one of these to capture 3D objects, for example, you’ll be disappointed at least in the 0.7.1 version of the SDK. 

Fortunately, I’m more interested in the higher-level abstractions  for now anyway.  I like the idea of being able to just wave my hands and _do stuff_.   Who that has seen Star Wars is not interested in that?  To test the waters, I decided to publish the sample JavaScript, but with a very simple main menu that allows me to choose a demo by holding up a number of fingers":

function chooseLinkByFingerCount() {
    if (lastFingerCount == 1) window.document.location.href = '@Url.Content("~/Scripts/leapjs-master/examples/dumper.html")';
    if (lastFingerCount == 2) window.document.location.href = '@Url.Content("~/Scripts/leapjs-master/examples/camera.html")';
    if (lastFingerCount == 3) window.document.location.href = '@Url.Content("~/Scripts/leapjs-master/examples/visualizer.html")';
    if (lastFingerCount == 4) window.document.location.href = '@Url.Action("Ko")';
}

var lastFingerCount = 0
var selectionTimeout = -1;
Leap.loop(function (frame) {
    latestFrame = frame

    if (latestFrame.fingers.length != lastFingerCount) {
        lastFingerCount = frame.fingers.length;
        document.getElementById('uiFingerCount').innerHTML = lastFingerCount.toString();
        if (selectionTimeout > -1) window.clearTimeout(selectionTimeout);
        selectionTimeout = window.setTimeout(chooseLinkByFingerCount, 1000);
    }
});

If you hold up 2 fingers for a second, then it jumps to the second demo.  In practice, this may not be the best UX (it only goes up to 10 menu items and took longer than mousing!), but it couldn’t have been much easier to implement, and I could feel the Force growing stronger within me as I navigated by simply holding up fingers.

Since then, I’ve also developed a quick demo to display tracking data using KnockoutJS so that I can start to get my head around gesture detection with this.

All in all, I’m really impressed with the kit and look forward to developing more with it.  I have some fun ideas for this thing, so keep an eye out here and on my test site: http://leapthing.azurewebsites.net/