Developer Guide

This is a terse guide to the classes of the Leap Motion JavaScript API. The Leap object is a singleton library, like jQuery's $ object. It is your bridge to the stream of data received from the Leap detector.

The only class you should manually instantiate is the Controller. (And for most applications you'll only want one controller instance.) None of the other classes should be manually created by you -- they should be retrieved by methods of the controller -- or methods of objects retrieved by controller methods ... and so on.

Getting Frame Data from your Leap Motion Controller

In order to get information from your Leap Motion controller you need:

  1. The Leap Motion controller hardware attached to your computer
  2. The Leap Motion software installed

Once this is established, you can interact with the data coming from the Leap Motion system. This article covers the LeapJS JavaScript API library.

There are three ways to access the Leap Motion data in your JavaScript applications:

  1. Call Leap.loop()
  2. Create a controller and listen for frame events
  3. Create a controller and poll the controller.frame() function when you want motion data

Event listening from the static Leap.loop() function

Leap.loop(handler) creates a Controller object and connects to the Leap Motion service automatically. You just need to supply a function to handle frames of data:

Leap.loop(function(frameInstance){
  ...
});

Event listening from a Controller

You can create your own Controller object and add an event listener for frame events. Each and every frame will be broadcast from the Leap Motion controller to your handler:

  var my_controller = new Leap.Controller({enableGestures: true});
  // see Controller documentation for option details
  my_controller.on('frame', function(frame_instance){ ... });
  my_controller.connect();

Polling a Leap.Controller

You can create a Controller object without adding a frame event listener. Instead, call the Controller.frame() method, whenever and as often or rarely as you wish:

  var my_controller = new Leap.Controller({enableGestures: true});

  my_controller.on('connect', function(){
    setInterval(function(){
      var frame = my_controller.frame();
    }, 500);
  });

  my_controller.connect();

This has several potential advantages:

  1. You can control the number of measurements yourself; you might only need 10 frames of information per second, so why do six times as much processing? Tying your polling to your rendering is more efficient since there may be no value to getting more than one frame per render cycle.
  2. You can get the best sample out of several measurements. You can sample the last 8 frames, and get the one with the most pointers/fingers/hands and use that to drive your application.
  3. You can choose whether or not to take signal data. After the user pushes a virtual button, you can stop listening while you resolve an action, then pick up listening later on.

Units of Measurement

All distances are expressed in millimeters, as floating point numbers. If you want to get relative (percent) measurements, use the InteractionBox.normalizePoint method. See the Leap.InteractionBox documentation for axis orientation. Coordinates for positions and directions are expressed as array objects containing three values ([x, y, z]). You can use the glMatrix library to perform vector and matrix math with these arrays.

All angles are measured in Radians. To convert to degrees, multiply by 180/Math.PI.

All time/timestamp measurements are given in microseconds; 1,000,000 microseconds = 1 second. Timestamps are relative measures of time since the Leap Motion software initialized. Subtract one timestamp from another to calculate the time that has elapsed between them.

Frame indexes reflect the LIFO nature of the Leap.Controller's frames collection.

  • my_controller.frames(0) == my_controller.frames() == the most recent frame.
  • my_controller.frames(1) is the previous frame
  • my_controller.frames(2) is two frames ago

and so on. Keep in mind that frames are constantly being pushed onto the stack, so my_controller.frames(2) will be a different frame in a few milliseconds. When in doubt use frame.id to identify/compare frames.

All positions and directions are returned as an array of three coordinates (see above). See Leap.InteractionBox for methods on normalizing and denormalizing coordinates.

scaleFactor

Leap.Hands and Leap.Frames have scaleFactor functions. Scale factor indicates a change in distance between things over time. For hands, this means the change in distance between the fingers. The function has a parameter: a baseline frame.

Scale factor for hands

Jazz_hands

Hand scale factor reflects the relative "Jazz handiness" of your fingers, compared with the baseline spread in the reference frame.

  • If you pinch all your fingers together since the reference frame, your hand's scaleFactor will be < 1.
  • If you expand ("jazz hands") your fingers farther apart, your hands' scaleFactor will be > 1.

Scale factor for frames

Jazz_arms

When frame.scaleFactor(old_frame) is called, it reflects the relative distance between the hands -- the "fish measuring factor".

  • If you move your hands apart, the scale factor will be > 1
  • If you move your hands together, the scale factor will be < 1

If Leap cannot relate the baseline reference with the current frame/hand data, the scaleFactor will be 1.

Valid and Invalid function return results

Any function which returns an instance object (Pointable, Frame, Hand,...) will ALWAYS return an object, even under conditions where returning an object is impossible (bad parameters, your hands aren't in front of the detector, etc.). Instead of returning false, null, or throwing an error, an invalid instance will be returned.

This lets you access child objects, such as fingers, without first checking that the frame exists, and that the hand exists in the frame, and so on, but you should examine the valid property of the final, target object before using its properties and methods. The valid property is a boolean and all Leap instances (except Gesture objects) have one.

Leap Classes

Leap.Frame

A collection of state information fed back from the Leap Motion hardware. A frame is the "root" data unit; it contains all the positional data that streams from the Leap Motion detector, Fingers/Tools/Pointables at a given instant in time.

See Getting Frame Data from your Leap Motion controller for documentation on getting frames.

A note on Frame.Fingers, Frame.pointables, and Frame.tools:

Pointables

frame.fingers, frame.tools and frame.pointables are different collections of instances which are all instances of the Leap.Pointable base class. (there is no special Leap.Tools or Leap.Fingers class -- just Leap.Pointable.)

  • The pointables collections contains all the pointables also found in tools and fingers.
  • The tools collections and the fingers collections are exclusive; there is no pointer which can be found in both the fingers collection and the tools collection
  • All pointables in these root collections can be found in the frame.hands properties.
  • Any of these collections can be empty, if the user's hands/fingers/tools aren't picked up by the Leap Motion detector.
  • All fingers/tools/pointables from both hands are stored with no ordering in these root level collections.

Frame Properties:

Frame Functions:


For more information, see Frame in the API reference.

Leap.Pointable

The Pointable class represents detected finger or tool **. Both fingers and tools are classified as Pointable objects. Use the Pointable.tool property to determine whether a Pointable object represents a tool or finger Note that Pointable objects can be invalid, which means that they do not contain valid tracking data and do not correspond to a physical entity. Invalid Pointable objects can be the result of asking for a Pointable object using an ID from an earlier frame when no Pointable objects with that ID exist in the current frame.

** The Leap classifies a detected entity as a tool when it is thinner, straighter, and longer than a typical finger.

Pointable Properties:

Pointable Functions:


For more information, see Pointable in the API reference.

Leap.Controller

Represents the connection to a single Leap detector. You can have multiple controllers in a single run time. Note that Leap.Controller is a JavaScript class; whereas the Leap Motion controller refers to an actual piece of hardware -- the box.

The code below creates a Leap.Controller, with an option parameter object specifying to enable gestures and to use the browser animation loop:

  var controller = new Leap.Controller({
  enableGestures: true,
  frameEventName: 'animationFrame'
});

Any or all these properties are optional; you can also create a controller with a simpler parameter set:

  var controller = new Leap.Controller({enableGestures: true});

Controller Events

connect
The client is connected to the WebSocket server.
frame
An iteration of the frame loop is starting. This event is either driven by the animationFrame or the deviceFrame event, depending on how the Leap.Controller was created. The latest frame is passed as an argument to the event handler. This frame may or may not be a new frame.
gesture
Dispatched for each gesture object in a new frame of data received from the WebSocket.
disconnect
The client disconnects from the WebSocket server.
focus
The browser received focus.
blur
The browser loses focus.
deviceConnected
A Leap device has been connected.
deviceDisconnected
A Leap device has been disconnected.
protocol
The protocol has been selected for the connection. The protocol object is passed as an argument to the event handler.

Here is a typical initialization cycle for interacting with a Leap Motion controller:

var controller = new Leap.Controller();

controller.on('connect', function() {
  console.log("Successfully connected.");
});

controller.on('deviceConnected', function() {
  console.log("A Leap device has been connected.");
});

controller.on('deviceDisconnected', function() {
  console.log("A Leap device has been disconnected.");
});

controller.connect();

Controller Properties:

Controller Functions:


For more information, see Controller in the API reference.

Leap.Hand

The Hand class reports the physical characteristics of a detected hand. Hands are accessed as properties of the Leap.Frame's hand array; at this point, up to two hands can be simultaneously tracked and returned with frame data. Hand tracking data includes a palm position and velocity; vectors for the palm normal and direction to the fingers; properties of a sphere fit to the hand; and lists of the attached fingers and tools.

See the Leap.Frame class documentation for the difference between Pointables, Fingers and Tools.

The below illustration shows the hand's palmNormal and direction vectors emanating from the palmPosition:

Palm Vectors

Hand Properties:

Hand Functions:


For more information, see Hand in the API reference.

Leap.InteractionBox

A representation of the "airspace" in which the Leap Motion controller can measure/see your hands and fingers. Note that the range of the interaction box may change with hardware or software advances. Use the interaction boxes' properties to scale your measurements -- don't hard code these values in your JavaScript.

The Axes' orientation

Leap_axes_annotated

  • The x-axis goes to the right. 0 on the x axis is the middle of the Leap Motion controller.
  • The y-axis goes up, from the Leap Motion controller; you will likely have to negate it to make it consistent with screen/DOM graphic coordinates. 0 on the y axis is very close to the Leap Motion detector.
  • The z-axis points towards you. 0 on the z axis is right above the detector. Your monitor will be at -Z, your head will have a +Z measurement.

InteractionBox Properties:

InteractionBox Functions:


For more information, see InteractionBox in the API reference.

Gesture

The Gesture class represents a recognized movement by the user. The Leap watches the activity within its field of view for certain movement patterns typical of a user gesture or command. For example, a movement from side to side with the hand can indicate a swipe gesture, while a finger poking forward can indicate a screen tap gesture. When the Leap recognizes a gesture, it assigns an ID and adds a Gesture object to the frame gesture list. For continuous gestures, which occur over many frames, the Leap updates the gesture by adding a Gesture object having the same ID and updated properties in each subsequent frame. Important: Recognition for each type of gesture must be enabled; otherwise no gestures are recognized or reported. Subclasses of Gesture define the properties for the specific movement patterns recognized by the Leap.

The Gesture subclasses for include:

CircleGesture
A circular movement by a finger
SwipeGesture
A straight line movement by the hand with fingers extended.
ScreenTapGesture
A forward tapping movement by a finger.
KeyTapGesture
A downward tapping movement by a finger.

Number of gestures produced

Circle and swipe gestures are continuous and these objects can have a state of start, update, and stop.

The screen tap gesture is a discrete gesture. The Leap only creates a single ScreenTapGesture object appears for each tap and it always has a stop state.

Getting Gesture instances from a Frame object.

You can get a list of gestures from the Frame gestures array. You can also use the Frame gesture() method to find a gesture in the current frame using an ID value obtained in a previous frame. Gesture objects can be invalid.

For more information, see the following classes in the API reference: