Systems and methods for remapping three-dimensional gestures onto a finite-size two-dimensional surface

1. A real-time gesture based interactive system comprising:

a processor;

an acquisition system configured to capture a sequence of frames of data for constructing a depth map of a field of view of the acquisition system;

memory storing an interaction medium tracking application, the interaction medium tracking application configuring the processor to:

obtain the data from the acquisition system;

compare successive frames of the frames of data for portions that change from one frame to the next;

determine whether any of the portions that changed are part of an interaction medium detected in the sequence of frames of data;

define an inner 3D interaction zone relative to an initial position of the part of the interaction medium detected in the sequence of frames of data, where the inner 3D interaction zone corresponds to a bounded region that is less than the frame of data and that contains the part of the interaction medium detected in the sequence of frames of data;

track a movement of the interaction medium in the data to generate a plurality of 3D positions of the interaction medium;

detect movement of the interaction medium from inside to outside the inner 3D interaction zone at a boundary 3D position;

shift the inner 3D interaction zone relative to the boundary 3D position;

compute a plurality of computed positions based on the 3D positions; and

supply the computed positions to control an application.

2. The system of claim 1 , wherein each of the computed positions includes an x component, a y component, and a z component.

3. The system of claim 1 , wherein the acquisition system comprises at least one of:

a plurality of cameras in a stereo arrangement having overlapping fields of view;

an infrared camera;

a color visible light camera; and

an illumination source configured to generate at least one of visible light, infrared light, ultrasonic waves, and electromagnetic signals.

4. The system of claim 1 , wherein the memory further stores instructions of the interaction medium tracking application to configure the processor to:

compute a 3D velocity of the interaction medium within the inner 3D interaction zone based on the 3D positions; and

compute two-dimensional movement data corresponding to the 3D positions and the 3D velocity, differences in the two-dimensional movement data being non-linear with respect to differences in the 3D positions.

5. The system of claim 1 , wherein the interaction medium is a portion of a human body.

6. The system of claim 1 , wherein the memory further stores instructions of the interaction medium tracking application to configure the processor to detect the movement of the interaction medium from the inside to the outside of the inner 3D interaction zone based on the 3D positions.

7. The system of claim 1 , wherein the memory further stores instructions of the interaction medium tracking application to configure the processor to detect the movement of the interaction medium from the inside to the outside of the inner 3D interaction zone by detecting a coarse movement of the interaction medium within an entire portion of the field of view of the acquisition system.

8. The system of claim 1 , wherein the memory further stores instructions of the interaction medium tracking application to configure the processor to:

detect a disengagement event; and

stop tracking the movement of the interaction medium in response to the disengagement event.

9. The system of claim 8 , wherein the memory further stores instructions of the interaction medium tracking application to configure the processor to detect the disengagement event by:

comparing the interaction medium detected in the frames of data to a target configuration to generate a compatibility confidence level; and

detecting the disengagement event when the confidence level is below a threshold level.

10. The system of claim 8 , wherein the memory further stores instructions of the interaction medium tracking application to configure the processor to detect the disengagement event by:

comparing the interaction medium detected in the frames of data to a disengagement configuration; and

detecting the disengagement event when the disengagement configuration is detected.

11. The system of claim 1 , wherein the memory further stores instructions of the interaction medium tracking application to configure the processor to:

track the movement of the interaction medium along a first direction toward a boundary of the inner interaction zone and concurrently update the plurality of computed positions in accordance with the movement along the first direction;

shift the inner interaction zone along the first direction and concurrently stop updating the plurality of computed positions in accordance with movement along the first direction; and

track the movement of the interaction medium along a second direction opposite the first direction and concurrently start updating the plurality of computed positions in accordance with the movement along the second direction.

12. A real-time gesture based interactive system comprising:

a processor;

an acquisition system configured to capture a sequence of frames of data for constructing a depth map of a field of view of the acquisition system;

memory storing an interaction medium tracking application, the interaction medium tracking application configuring the processor to:

obtain the data from the acquisition system;

compare successive frames of the frames of data for portions that change from one frame to the next;

determine whether any of the portions that changed are part of an interaction medium detected in the sequence of frames of data;

define an inner 3D interaction zone relative to an initial position of the part of the interaction medium detected in the sequence of frames of data, where the inner 3D interaction zone corresponds to a bounded region that is less than the frame of data and that contains the part of the interaction medium detected in the sequence of frames of data;

track a movement of the interaction medium in the data to generate a plurality of 3D positions of the interaction medium;

compute a 3D velocity of the interaction medium within the inner 3D interaction zone based on the 3D positions;

compute a plurality of computed positions based on the 3D positions and the 3D velocity, differences in the computed positions being non-linear with respect to differences in the 3D positions; and

supply the computed positions to control an application.

13. The system of claim 12 , wherein each of the computed positions includes an x component, a y component, and a z component.

14. The system of claim 12 , wherein the 3D velocity comprises a horizontal component v x and a vertical component v y and wherein the 3D velocity is thresholded by a minimum threshold T m and a maximum threshold T M to compute thresholded 3D velocities a and b, where:

a =min(max(| v x t |,T m ), T M )

b =min(max(| v y t |,T m ), T M ).

15. The system of claim 14 , wherein the real-time gesture based interactive system further comprises a display device having S w columns and S h rows,

wherein the plurality of computed positions comprises column data c and row data r corresponding to coordinates of the display device, and

wherein the 3D velocity and the thresholded 3D velocities are mapped onto the column data c and the row data r, where:

c t =min(max( c t-1 −v x t *K*a, 0), S w )

r t =min(max( r t-1 −v y t *K*b, 0), S h ),

and where K is a sensitivity parameter.

16. A method for operating a real-time gesture based interactive system, the method comprising:

obtaining a sequence of frames of data from an acquisition system configured to capture data for constructing a depth map of a field of view of the acquisition system;

comparing, by a processor, successive frames the frames of data for portions that change from one frame to the next;

determining, by the processor, whether any of the portions that changed are part of an interaction medium detected in the sequence of frames of data;

defining, by the processor, an inner 3D interaction zone relative to an initial position of the part of the interaction medium detected in the sequence of frames of data, where the inner 3D interaction zone corresponds to a bounded region that is less than the frame of data and that contains the part of the interaction medium detected in the sequence of frames of data;

tracking, by the processor, a movement of the interaction medium in the data to generate a plurality of 3D positions of the interaction medium;

detecting, by the processor, movement of the interaction medium from inside to outside the inner 3D interaction zone at a boundary 3D position;

shifting, by the processor, the inner 3D interaction zone relative to the boundary 3D position;

computing, by the processor, a plurality of computed positions based on the 3D positions; and

supplying the computed positions to control an application.

17. The method of claim 16 , wherein each of the computed positions includes an x component, a y component, and a z component.

18. The method of claim 16 , wherein the acquisition system comprises at least one of:

a plurality of cameras in a stereo arrangement having overlapping fields of view;

an infrared camera;

a color visible light camera; and

an illumination source configured to generate at least one of visible light, infrared light, ultrasonic waves, and electromagnetic signals.

19. The method of claim 16 , further comprising:

computing a 3D velocity of the interaction medium within the inner 3D interaction zone based on the 3D positions; and

computing two-dimensional movement data corresponding to the 3D positions and the 3D velocity, differences in the two-dimensional movement data being non-linear with respect to differences in the 3D positions.

20. The method of claim 16 , wherein the interaction medium is a portion of a human body.

21. The method of claim 16 , further comprising detecting the movement of the interaction medium from the inside to the outside of the inner 3D interaction zone based on the 3D positions.

22. The method of claim 16 , further comprising detecting the movement of the interaction medium from the inside to the outside of the inner 3D interaction zone by detecting a coarse movement of the interaction medium within an entire portion of the field of view of the acquisition system.

23. The method of claim 16 , further comprising:

detecting a disengagement event; and

stopping tracking the movement of the interaction medium in response to the disengagement event.

24. The method of claim 23 , wherein the detecting the disengagement event comprises:

comparing the interaction medium detected in the frames of data to a target configuration to generate a compatibility confidence level; and

detecting the disengagement event when the confidence level is below a threshold level.

25. The method of claim 23 , wherein the detecting the disengagement event comprises:

comparing the interaction medium detected in the frames of data to a disengagement configuration; and

detecting the disengagement event when the disengagement configuration is detected.

26. The method of claim 16 , further comprising:

tracking the movement of the interaction medium along a first direction toward a boundary of the inner interaction zone and concurrently update the plurality of computed positions in accordance with the movement along the first direction;

shifting the inner interaction zone along the first direction and concurrently stop updating the plurality of computed positions in accordance with movement along the first direction; and

tracking the movement of the interaction medium along a second direction opposite the first direction and concurrently start updating the plurality of computed positions in accordance with the movement along the second direction.

27. A method for tracking a gesture comprising:

obtaining a sequence of frames of data from an acquisition system configured to capture data for constructing a depth map of a field of view of the acquisition system;

comparing, by a processor, successive frames of the frames of data for portions that change from one frame to the next;

determining, by the processor, whether any of the portions that changed are part of an interaction medium detected in the sequence of frames of data;

defining, by the processor, an inner 3D interaction zone relative to an initial position of the part of the interaction medium detected in the sequence of frames of data, where the inner 3D interaction zone corresponds to a bounded region that is less than the frame of data and that contains the part of the interaction medium detected in the sequence of frames of data;

tracking, by the processor, a movement of the interaction medium in the data to generate a plurality of 3D positions of the interaction medium;

computing, by the processor, a 3D velocity of the interaction medium within the inner 3D interaction zone based on the 3D positions;

computing, by the processor, a plurality of computed positions based on the 3D positions and the 3D velocity, differences in the computed positions being non-linear with respect to differences in the 3D positions; and

supplying, by the processor, the computed positions to control an application.

28. The method of claim 27 , wherein each of the computed positions includes an x component, a y component, and a z component.

29. The method of claim 27 , wherein the 3D velocity comprises a horizontal component v x and a vertical component v y and wherein the 3D velocity is thresholded by a minimum threshold T m and a maximum threshold T M to compute thresholded 3D velocities a and b, where:

a =min(max(| v x t |,T m ), T M )

b =min(max(| v y t |,T m ), T M ).

30. The method of claim 29 , wherein the application includes a user interface displayed on a display device having S w columns and S h rows,

wherein the plurality of computed positions comprises column data c and row data r corresponding to coordinates of the display device, and

wherein the 3D velocity and the thresholded 3D velocities are mapped onto the column data c and the row data r, where:

c t =min(max( c t-1 −v x t *K*a, 0), S w )

r t =min(max( r t-1 −v y t *K*b, 0), S h ),

and where K is a sensitivity parameter.