Hands

Hands are the main entity tracked by the Leap Motion controller. The controller maintains an inner model of the human hand and validates the data from its sensors against this model. This allows the controller to track finger positions even when a finger is not completely visible. Note that it is possible for movement or changes in position to be lost when a finger is behind or directly in front of the hand (from the point of view of the controller). The Leap Motion software matches the internal model against the existing data. In some cases, the software can make an incorrect match – for example, identifying a right hand as a left hand.

The Hand class represents a physical hand detected by the Leap. A Hand object provides access to lists of its pointables as well as attributes describing the hand position, orientation, and movement.

Hands can be identified by left versus right handedness. In addition, each hand is assigned an ID value when the hand is first detected. If you remove a hand from the Leap Motion field of view and then re-insert it, a new ID is assigned (use the timeVisible attribute to tell whether a hand is newly detected or not). New hand IDs are also assigned if the Leap Motion software decides it has misclassified a hand and must change its type from right to left or vice versa.

Getting Hands

Get Hand objects from a Frame:

frame = controller.frame() # controller is a Leap.Controller object
hands = frame.hands
first_hand = hands[0]

Or, if you know the ID from a previous frame:

known_hand = frame.hand(hand_ID)

You can also get hands by their relative positions in the frame:

frame = controller.frame() # controller is a Leap.Controller object
hands = frame.hands

leftmost = hands.leftmost
rightmost = hands.rightmost
frontmost = hands.frontmost

Note that the the leftmost() and rightmost() functions only identify which hand is farthest to the left or right. Use the Hand isLeft or isRight attribute to tell if a hand object represents a left or a right hand.

Getting the Hand Characteristics

A hand is described by its handedness, position, orientation, posture, and motion:

  • isRight, isLeft — Whether the hand is a left or a right hand.
  • Palm Position — The center of the palm measured in millimeters from the Leap Motion origin.
  • Palm Velocity — The speed and movement direction of the palm in millimeters per second.
  • Palm Normal — A vector perpendicular to the plane formed by the palm of the hand. The vector points downward out of the palm.
  • Direction — A vector pointing from the center of the palm toward the fingers.
  • grabStrength, pinchStrength — Describe the posture of the hand.
  • Motion factors — Provide relative scale, rotation, and translation factors for movement between two frames.

The hand’s position is given by its palm position attribute, which provides a vector containing the 3-dimensional coordinates of the palm center point in millimeters from the Leap Motion origin. The hand’s orientation is given by two vectors: the direction, which points from the palm center towards the fingers, and the palm normal, which points out of the palm, perpendicular to the plane of the hand.

The movement of the hand is given by the velocity attribute, which is a vector providing the instantaneous motion of the hand in mm/s. You can also get motion factors that translate how a hand has moved between two given frames into translation, rotation, and scaling values.

The following code snippet illustrates how to get a Hand object from a frame and access its basic attributes:

hand = frame.hands.rightmost
position = hand.palm_position
velocity = hand.palm_velocity
direction = hand.direction

Getting the Fingers

You can get the fingers associated with a hand as a list or individually using an ID obtained in a previous frame.

By list:

# hand is a Leap.Hand object
pointables = hand.pointables
fingers = hand.fingers

By ID from a previous frame:

known_pointable = hand.pointable(pointable_ID)

To get a finger by relative position within the Leap field of view, use the right-, left- and frontmost functions of the matching list class:

# hand is a Leap.Hand object
left_pointable = hand.pointables.leftmost
right_finger = hand.fingers.rightmost
front_finger = hand.fingers.frontmost

Note that these functions are relative to the Leap Motion origin, not to the hand itself. To get the fingers relative to the hand, you can use the Leap Matrix class to transform the finger positions into the hands frame of reference.

Computing the Hand Orientation

You can compute the hand orientation angles using the Hand direction and normal vectors.

https://di4564baj7skl.cloudfront.net/documentation/images/Leap_Palm_Vectors.png

The normal vector points perpendicularly out of the hand; the direction vector points forward.

The Vector class defines functions for getting the pitch (angle around the x-axis), yaw (angle around the y-axis), and roll (angle around the z-axis):

pitch = hand.direction.pitch
yaw = hand.direction.yaw
roll = hand.palm_normal.roll

Note that the roll function only provides the expected angle when used with a normal vector.

Transforming Finger Coordinates into the Hand’s Frame of Reference

Sometimes it is useful to obtain the coordinates of the fingers of a hand with respect to the hand’s frame of reference. This lets you sort the fingers spatially and can simplify analysis of finger positions. You can create a transform matrix using the Leap Matrix class to transform the finger position and direction coordinates. The hand frame of reference can be usefully defined by the hand’s basis and palm_position. The basis orients the x-axis sideways across the hand, the z-axis pointing forward, and the y-axis parallel with the palm normal. The origin of the transform is the palm_position.

frame = controller.frame()
for hand in frame.hands:
    hand_x_basis = hand.basis.x_basis
    hand_y_basis = hand.basis.y_basis
    hand_z_basis = hand.basis.z_basis
    hand_origin = hand.palm_position
    hand_transform = Leap.Matrix(hand_x_basis, hand_y_basis, hand_z_basis, hand_origin)
    hand_transform = hand_transform.rigid_inverse()

    for finger in hand.fingers:
        transformed_position = hand_transform.transform_point(finger.tip_position)
        transformed_direction = hand_transform.transform_direction(finger.direction)
        # Do something with the transformed fingers

Hand Lists

The HandList class acts like a vector-style array and supports iterators. You cannot remove or alter the member objects of a hand lists received from the Leap Motion API, but you can combine lists of the same object type.

To use an iterator with a list class:

for hand in handList:
    print hand

The HandList class defines additional functions for getting a member of the list based on its relative position within the Leap coordinate system. These functions include leftmost(), rightmost(), and frontmost(). The following snippet illustrates how to get the hand object furthest to the right:

furthest_right = frame.hands.rightmost