IP Library Granted Patent US 10,719,170
Granted Patent B2
US 10,719,170 · App. 15/274,968 · Granted Jul 21, 2020

Method and device for detecting a touch between a first object and a second object

Inventor: Daniel Kurz (Munich, DE)
Assignee: Apple Inc.
G06F3/0425G06F3/011G06F3/017G06T19/006H04N5/33
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,719,170
App. No.
15/274,968
Granted
Jul 21, 2020
Kind
B2
Abstract

The present disclosure is related to a method and device for detecting a touch between at least part of a first object and at least part of a second object, wherein the at least part of the first object has a different temperature than the at least part of the second object. The method includes providing at least one thermal image of a portion of the second object, determining in at least part of the at least one thermal image a pattern which is indicative of a particular value or range of temperature or a particular value or range of temperature change, and using the determined pattern for detecting a touch between the at least part of the first object and the at least part of the second object.

Claims (43)

1. A method of detecting a touch between at least part of a first object and at least part of a second object, wherein the at least part of the first object has a different temperature than the at least part of the second object, comprising the steps of:

receiving a thermal image of a portion of the second object;

determining temperature measurements based on different locations in the thermal image;

determining a derivative of temperature with respect to position in the thermal image based on the temperature measurements, the temperature measurements corresponding to a sample line in the thermal image; and

determining a touch between the at least part of the first object and the at least part of the second object has occurred in response to a determination that the derivative of temperature with respect to position in the thermal image indicates:

a first section of the sample line is associated with a first rate of change in temperature with respect to position;

a second section of the sample line adjacent to the first section is associated with a second rate of change in temperature with respect to position greater than the first rate of change; and

a third section of the sample line adjacent to the second section is associated with a third rate of change in temperature with respect to position greater than the second rate of change.

2. The method according to claim 1 , wherein the first object corresponds to a portion of a human body.

3. The method according to claim 1 , further comprising determining a position, a size, an orientation, a direction, a trajectory, or a shape of the touch in the thermal image.

4. The method according to claim 1 , further comprising outputting an indication of the touch to a machine interface program, wherein the machine interface program changes states based on the indication.

5. The method according to claim 1 , wherein detecting the touch further comprises determining one or more differences between temperatures measured at different locations in the thermal image.

6. The method according to claim 1 , wherein the at least part of the second object is not shown in the thermal image.

7. The method according to claim 1 , wherein the at least part of the first object is not shown in the thermal image.

8. The method according to claim 1 , wherein the thermal image corresponds to a moment during which the at least part of the first object and the at least part of the second object are touching.

9. The method according to claim 1 , wherein the thermal image corresponds to a moment after which the at least part of the first object and the at least part of the second object have touched.

10. The method according to claim 1 , wherein the derivative corresponds to a first derivative.

11. The method according to claim 1 , wherein the derivative corresponds to a second derivative.

12. The method according to claim 1 , wherein the first object and the second object are three dimensional.

13. The method according to claim 1 , further comprising:

capturing an image of the portion of the second object using a visible light camera, wherein the thermal image is received from a thermal camera;

determining a first spatial transformation between the visible light camera and the thermal camera;

determining a second spatial transformation between the visible light camera and the portion of the second object,

determining, based on the first and second spatial transformations, a third spatial transformation between a coordinate system of the portion of the second object and a coordinate system of the thermal camera,

determining a position and orientation of the thermal camera in the coordinate system of the portion of the second object based on the third spatial transformation.

14. The method according to claim 1 , wherein the method is applied within a human machine interface in an Augmented Reality application.

15. The method according to claim 1 , wherein the method is used within an application using one of a video-see-through setup, an optical-see-through setup, and a projective augmented reality setup.

16. A non-transitory computer readable storage medium storing software code sections executable by a processor to:

receive a thermal image of a portion of a second object;

determine temperature measurements based on different locations in the thermal image;

determine a derivative of temperature with respect to position in the thermal image based on the temperature measurements, the temperature measurements corresponding to a sample line in the thermal image; and

determine a touch between the at least part of the first object and the at least part of the second object has occurred in response to a determination that the derivative of temperature with respect to position in the thermal image indicates:

a first section of the sample line is associated with a first rate of change in temperature with respect to position;

a second section of the sample line adjacent to the first section is associated with a second rate of change in temperature with respect to position greater than the first rate of change; and

a third section of the sample line adjacent to the second section is associated with a third rate of change in temperature with respect to position greater than the second rate of change.

17. A device for detecting a touch between at least part of a first object and at least part of a second object, wherein the at least part of the first object has a different temperature than the at least part of the second object, comprising:

a processing device adapted to receive a thermal image of a portion of the second object,

the processing device configured to determine temperature measurements based on different locations in the thermal image;

the processing device configured to determine a derivative of temperature with respect to position in the thermal image based on the temperature measurements, the temperature measurements corresponding to a sample line in the thermal image; and

the processing device configured to, determine a touch between the at least part of the first object and the at least part of the second object has occurred in response to a determination that the derivative of temperature with respect to position in the thermal image indicates:

a first section of the sample line is associated with a first rate of change in temperature with respect to position;

a second section of the sample line adjacent to the first section is associated with a second rate of change in temperature with respect to position greater than the first rate of change; and

a third section of the sample line adjacent to the second section is associated with a third rate of change in temperature with respect to position greater than the second rate of change.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 6, 2016
From: METAIO GMBH
To: APPLE INC.
Reel/Frame 040821/0462 →
Continuity (3)
Continuation 14223601 · Mar 24, 2014
Continuation In Part PCTEP2014053017 · Feb 17, 2014
Related Publication 20170075502A1 · Mar 16, 2017