 |
Leap Motion C API
4.1.0
The API to the LeapC library.
|
|
Leap Motion C API
4.1.0
The API to the LeapC library.
|
The Leap Motion system employs a right-handed Cartesian coordinate system. The origin is centered at the top of the Leap Motion Controller. The x- and z-axes lie in the plane of the camera sensors, with the x-axis running along the camera baseline. The y-axis is vertical, with positive values increasing upwards (in contrast to the downward orientation of most computer graphics coordinate systems). The z-axis has positive values increasing toward the user.
The Leap Motion API measures physical quantities with the following units:
A Frame is the collection of tracking data derived from a single stereo-pair of images. The Leap Motion service/daemon software analyzes each pair of stereo images from the device sensors to detect the presense, posture, and motion of hands. This data is transmitted as a LEAP_TRACKING_EVENT struct. The images used to create the frame of data can be requested separately.
The Leap Motion software analyzes the images and fits an internal model of the human hand to the observed scene. If parts of a hand are occluded or out of view, the software estimates the data based on what is visible, past observations, and model constraints.
You get LEAP_TRACKING_EVENT objects by calling LeapPollConnection() or LeapInterpolateFrame().
Frame interpolation uses the nearest frame data to synthesize a new frame at a specified time. If this specified time is between two past frames, the data of these two frames is interpolated to synthesize an intermediate frame. If the specified time is more recent than the latest frame, then the new frame is synthesized by extrapolating the most recent data. Extrapolation can reduce latency between the motion of the hand and the corresponding change in the graphical display – especially on systems that double or triple buffer rendering frames. However, extrapolating too far in the future can lead to jerky motion and other artifacts.
Request an interpolated frame with LeapInterpolateFrame(). Past tracking data is only kept for a few frames. If you attempt to interpolate too far in the past, then this function will return an error.
See Interpolating Frames for more information and Interpolated Frames Example for code samples.
When requesting interpolated frames, you must specify the time for the desired frame. This time must be provided in terms of the internal clock used by the Leap Motion software. You can use LeapGetNow() to get the current Leap time, but if you are synchronizing frames with an application clock, you can use clock rebasing to synchronize the aLeap and application clocks. When the two clocks are synchronized, you can use the function LeapRebaseClock() to translate the application clock to the Leap Motion clock.
To use clock rebasing, create a LEAP_CLOCK_REBASER object with LeapCreateClockRebaser(). Maintain synchronization between the clocks by calling LeapUpdateRebase() as often as possible, ideally every time you request a frame of data.
See Interpolated Frames Example for code samples.
Hands are the primary focus of the Leap Motion tracking data. The LEAP_HAND struct is the parent of the physical tracking data measured for an individual hand. In addition to its own position and orientation, the hand provides access to the objects representing the digits and arm attached to that hand.
Get hand data from a LEAP_TRACKING_EVENT object.
Fingers are represented by the LEAP_DIGIT struct. Fingers are made up of four bones: metacarpal, proximal phalange, intermediate phalange, and distal phalange, in that order, from hand to tip. One thing to note, is that the thumb uses the same bones as the other fingers, even though an anatomical thumb has no intermediate phalange. To compensate, the Leap Motion model of the thumb has a zero-length metacarpal bone.
The segments of the fingers are represented by LEAP_BONE structs. There are four bones per finger:
A bone has two endpoints, a quaternion representing its orientation, and a width. Other physical quantities, such as length and direction, can be calculated from these values.
The arm is represented by the LEAP_BONE struct. Given the centrality of the hand to the Leap Motion hand tracking system, the arm is a child of the hand, rather than vice versa.
The LEAP_VECTOR struct is a three-element vector used to store positions, directions, velocities, and other coordinates. The LEAP_VECTOR struct has no functions for performing vector math.
The LEAP_QUATERNION struct is a four-element vector used to store rotations. The LEAP_QUATERNION struct has no functions for performing quaternion operations.
The Leap Motion system currently uses two monochrome, IR cameras as its primary sensors. You can request images by setting the Images Policy flag. When the images are received, the LEAP_IMAGE_EVENT struct is provided by the LeapPollConnection() function. The image event also contains image properties, such as width, height, and the distortion map.
The image buffer is filled in with images from both cameras. The left image is first; followed by the second. Current devices use one byte per pixel, representing the recorded brightness value at that pixel. Future devices may provide different pixel formats. Image resolution can change dynamically and varies by device type, settings, and operating modes.
Different Leap Motion hardware can support different image formats – although all currently available devices support only the monochrome, 1-byte-per-pixel format. Future devices may support different formats. The format and number of bytes per pixel of an image are provided by the LEAP_IMAGE_PROPERTIES struct.
See Using Images for more information.
The distortion map can be used to rectify an image so that the brightness value of a pixel represents the true value entering the camera after correcting for lens distortions. The distortion map is a 64x64 grid of 32bit floating point coordinates. Each point in the map has an x and y coordinate that represents the location in the image data to find the true brightness value. For points in between the grid centers, you can use bilinear interpolation. In fact, the distortion map is defined this way to take advantage of GPU texture map interpolation. Rectifying an image without a GPU is generally too slow for real-time systems. However, you can use the LeapPixelToRectilinear() function to rectify individual points.
See Image Distortion for more information.
Policies are Leap Motion system settings that can affect not just your own application, but any running Leap-enabled application. There are currently three defined policies (enumerated by the eLeapPolicyFlag enum):
Setting and getting policies is asynchronous and may fail. When you call LeapSetPolicyFlags to read or write a policy, LeapPollConnection() will provide a LEAP_POLICY_EVENT when the policy change has been processed. Compare the desired policy flags with the active policy flags in the LEAP_POLICY_EVENT struct to determine whether the policy change succeeded.
See Controlling Policies for information about setting and checking policies.
The Leap Motion software has a number of configuration settings. Some settings reflect the user options in the control panel, other settings reflect default, but changeable configuration values assigned in the config.json files read at system startup, still others are experimental software switches intended for internal use only. Only use documented settings in your applications. Leap Motion may remove or change non-documented settings without notice.
Setting and reading configuration values is asynchronous and may fail. When you call LeapRequestConfigValue() to read a setting, LeapPollConnection() will provide a LEAP_CONFIG_RESPONSE_EVENT when the value is available. Similarly, when you write a config value with LeapSaveConfigValue(), LeapPollConnection() provides a LEAP_CONFIG_CHANGE_EVENT, which reports whether the change was successful.
See Getting and Setting Configuration Values and Configuration for more information.
The LEAP_DEVICE_INFO struct provides information about the capabilities of the attached Leap Motion hardware. Rather than hard-coding the capabilities based on your current device, you can use the device info struct to provide a measure of reiliance against future changes. The key capabilities include:
See Reading Device Properties for information on accessing device information.
The primary Leap Motion software operates as a service (on Windows) or a daemon (on Mac, Linux, and Android). The LeapC library connects to this service/daemon when you create and open a connection.
See Getting Tracking Frames and Other Event Messages for more inormation about creating and opening connections. See Examples for code samples.
As events occur in the Leap Motion service/daemon, they are added to an event queue. LeapPollConnection() reads this queue and populates the LEAP_CONNECTION_MESSAGE struct type relevant to the event. For example, when a tracking frame is generated by the service, LeapPollConnection() provides a LEAP_TRACKING_EVENT embedded in the LEAP_CONNECTION_MESSAGE struct. If you don't poll the connection fast enough, messages will be discarded.
See Getting Tracking Frames and Other Event Messages for more inormation about accessing tracking events. See Examples for code samples.
1.8.11 
