The Leap Motion API provides information that you can use to implement touch emulation in your application. Touch information is provided by the Pointable class.
The Leap defines an adaptive touch surface that you can use to orchestrate interaction with 2D elements of your application. This surface is oriented roughly parallel to the x-y plane, but adapts to the user’s finger and hand position. As the user reaches forward with a finger or tool, the Leap reports whether that pointable object is close-to or touching this imaginary surface. The API reports touches with respect to the surface with two values: the zone and the distance to the touch plane.
The touch zone identifies whether the Leap Motion software considers a Pointable as hovering near the touch surface, as penetrating the touch surface, or as too far from the surface (or pointing in the wrong direction). The zones are “hovering,” “touching,” and “none.” The transition between zones tends to lag behind the touch distance. This lag is used to prevent abrupt and repeated transitions. If you are implementing touch interaction within an application, you may not need to consider the touch zone very often.
The touch distance is valid only when a Pointable is within the hovering or touching zones. The distance is a normalized value in the range [+1..-1]. When a Pointable first enters the hovering zone, the touch distance is +1.0 and the distance decreases toward 0 as the Pointable nears the touch surface. When the Pointable penetrates the surface, the distance is 0. As the Pointable pushes deeper into the touch zone, the distance approaches, but never exceeds, -1.
You can use the zone value to decide when to update UI elements based on hover or touch. You can use the distance to further modify UI elements based on proximity to the touch plane. For example, you can show the highlight state of a control when a finger is over the control and in the hovering zone and change the cursor based on distance to provide feedback about how close the user is to touching the control.
As part of the touch emulation API the Leap Motion provides a stabilized position for Pointable objects in addition to the standard position. The Leap Motion software stabilizes the position using an adaptive filter that smooths and slows the motion to make it easier for the user to interact with small regions on the screen (like buttons and links). The smoothing is greater when the movement is slow so that the user can zero in and touch a particular point more easily.
The touch zone is reported by the touchZone attribute of the Pointable class. The zones are identified using the Zone enumeration, which defines the following states:
The following code snippet illustrates how to retrieve the zone of the forward-most finger:
zone = pointable.touch_zone
The touch distance is reported by the touchDistance attribute of the Pointable class. The distance ranges from +1 to -1 as the finger moves to and through the virtual touch surface. The distance does not represent a physical quantity, but rather how close to touching the Leap Software considers the pointable.
The following code snippet illustrates how to retrieve the touch distance of the forward-most finger:
distance = pointable.touch_distance
The stabilized position is reported by the stabilizedTipPosition attribute of the Pointable class. This position is reported in reference to the standard Leap Motion coordinate system, but has a context-sensitive amount of filtering and stabilization.
The following code snippet illustrates how to retrieve the stabilized position of the forward-most finger:
frame = controller.frame() pointable = frame.pointables.frontmost stabilizedPosition = pointable.stabilized_tip_position
When implementing touch emulation, you must map the Leap Motion coordinate space to the screen space of your application. To make this mapping easier, the Leap Motion API provides the InteractionBox class. The InteractionBox represents a rectilinear volume within the Leap Motion field of view. The class provides a function that normalizes positions within this volume to coordinates in the range [0..1]. You can normalize a position and then scale the resulting coordinate by the application dimensions to get a point in application coordinates.
For example, if you have a window with a client-area size represented by the variables windowWidth and windowHeight, you can get the 2D pixel coordinates of a touch point within this window using the following code:
frame = controller.frame() finger = frame.fingers.frontmost stabilizedPosition = finger.stabilized_tip_position interactionBox = controller.frame().interaction_box normalizedPosition = interactionBox.normalize_point(stabilizedPosition) x = normalizedPosition.x * windowWidth y = windowHeight - normalizedPosition.y * windowHeight
The following example uses the touch emulation APIs to display the positions of all detected Pointable objects in an application window. The example uses the touch zone to set the color of the points and uses the touch distance to set the alpha value. The stabilized tip positions are mapped to the application window using the InteractionBox class.
from Tkinter import Frame, Canvas, YES, BOTH import Leap class TouchPointListener(Leap.Listener): def on_init(self, controller): print "Initialized" def on_connect(self, controller): print "Connected" def on_frame(self, controller): self.paintCanvas.delete("all") frame = controller.frame() interactionBox = frame.interaction_box for pointable in frame.pointables: normalizedPosition = interactionBox.normalize_point(pointable.tip_position) if(pointable.touch_distance > 0 and pointable.touch_zone != Leap.Pointable.ZONE_NONE): color = self.rgb_to_hex((0, 255 - 255 * pointable.touch_distance, 0)) elif(pointable.touch_distance <= 0): color = self.rgb_to_hex((-255 * pointable.touch_distance, 0, 0)) #color = self.rgb_to_hex((255,0,0)) else: color = self.rgb_to_hex((0,0,200)) self.draw(normalizedPosition.x * 800, 600 - normalizedPosition.y * 600, 40, 40, color) def draw(self, x, y, width, height, color): self.paintCanvas.create_oval( x, y, x + width, y + height, fill = color, outline = "") def set_canvas(self, canvas): self.paintCanvas = canvas def rgb_to_hex(self, rgb): return '#%02x%02x%02x' % rgb class PaintBox(Frame): def __init__( self ): Frame.__init__( self ) self.leap = Leap.Controller() self.painter = TouchPointListener() self.leap.add_listener(self.painter) self.pack( expand = YES, fill = BOTH ) self.master.title( "Touch Points" ) self.master.geometry( "800x600" ) # create Canvas component self.paintCanvas = Canvas( self, width = "800", height = "600" ) self.paintCanvas.pack() self.painter.set_canvas(self.paintCanvas) def main(): PaintBox().mainloop() if __name__ == "__main__": main()
The example uses Tkinter, but the Leap Motion-related code is applicable to all Python projects.