IP Library Granted Patent US 10,481,740
Granted Patent B2
US 10,481,740 · App. 15/661,193 · Granted Nov 19, 2019

Projecting a structured light pattern onto a surface and detecting and responding to interactions with the same

Inventors: Florin Cutu (San Jose, CA); Chi Zhang (Fremont, CA)
Assignee: AMS SENSORS SINGAPORE PTE. LTD.
G06F3/0425G06F3/017G06K9/00335G06T7/11G06T7/136G06T7/155G06T7/246G06T7/248G06T7/521G06T2207/10048G06T2207/20036G06T2207/20092G06T2207/30241
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,481,740
App. No.
15/661,193
Granted
Nov 19, 2019
Kind
B2
Abstract

The present disclosure describes projecting a structured light pattern onto a surface and detecting and responding to interactions with the same. A method includes-acquiring an image based on light reflected from a vicinity of the projection surface, and identifying regions of the acquired image that correspond to a feature that is within a specified distance of the projection surface by identifying regions of the acquired image for which intensity data differs relative to other regions of the acquired image and which fit a specified homographic relationship with respect to corresponding regions of a reference image.

Claims (39)

1. A method comprising:

(a) projecting a structured light pattern onto a projection surface, the structured light pattern being composed of light of a particular wavelength;

(b) acquiring an image based on light of the particular wavelength reflected from a vicinity of the projection surface;

(c) identifying, by one or more processors, regions of the acquired image that correspond to a feature that is within a specified distance of the projection surface by identifying regions of the acquired image for which intensity data differs relative to other regions of the acquired image and which fit a specified homographic relationship with respect to corresponding regions of a reference image, and generating a binary image in which regions of the acquired image that are determined to correspond to the feature within the specified distance of the projection surface are assigned a first value, and regions of the acquired image that are determined not to correspond to the feature within the specified distance of the projection surface are assigned a second different value;

(d) use the binary image to determine, by the one or more processors, that the feature is a particular type of object;

(e) repeating (b), (c) and (d) one or more times;

(f) performing frame-to-frame tracking to determine a path of movement of the object; and

(g) determining, by the one or more processors, that the object's path of movement represents a specified type of gesture.

2. The method of claim 1 including determining the homographic relationship between a first set of coordinates in the reference image and a second set of coordinates in the acquired image.

3. The method of claim 1 including performing image morphology filtering with respect to the identified regions of the acquired image.

4. The method of claim 1 including performing shape descriptor filtering with respect to the identified regions of the acquired image.

5. The method of claim 1 wherein, in (c), regions of the acquired image are identified for which intensity data differs relative to other regions of the acquired image by at least a predetermined threshold value.

6. The method of claim 1 wherein the particular wavelength is in the infra-red or near infra-red portions of the spectrum.

7. The method of claim 1 further including projecting a visible pattern onto the projection surface at the same time as the structured light pattern is projected onto the projection surface.

8. The method of claim 1 further including:

in response to determining that the object's path of movement represents a specified type of gesture, causing another apparatus to perform a specified operation based on the type of gesture.

9. The method of claim 8 wherein causing another apparatus to perform a specified operation based on the type of gesture includes at least one of the following:

causing a display monitor to display an alphanumeric character;

causing a light to be turned on, off or dimmed;

causing a computing device to transition to a next slide of a presentation.

10. An apparatus comprising:

a first projector operable to project a structured light pattern onto a projection surface, the structured light pattern being composed of light of a particular wavelength;

an image sensor operable to acquire images based on light of the particular wavelength reflected from a vicinity of the projection surface; and

one or more processors operable collectively to:

(a) identify regions of an acquired image that correspond to a feature that is within a specified distance of the projection surface by identifying regions of the acquired image for which intensity data differs relative to other regions of the acquired image and which fit a specified homographic relationship with respect to corresponding regions of a reference image, and generate a binary image in which regions of the acquired image that are determined to correspond to the feature within the specified distance of the projection surface are assigned a first value, and regions of the acquired image that are determined not to correspond to the feature within the specified distance of the projection surface are assigned a second different value;

(b) use the binary image to determine that the feature is a particular type of object;

(c) repeat (a) and (b) one or more times for at least one other acquired image;

(d) perform frame-to-frame tracking to determine a path of movement of the object; and

(e) determine that the object's path of movement represents a specified type of gesture.

11. The apparatus of claim 10 wherein the one or more processors are operable to perform image morphology filtering with respect to the identified regions of the acquired image.

12. The apparatus of claim 10 wherein the one or more processors are operable to perform shape descriptor filtering with respect to the identified regions of the acquired image.

13. The apparatus of claim 10 wherein the one or more processors are operable, in (a), to identify regions of the acquired image for which intensity data differs relative to other regions of the acquired image by at least a predetermined threshold value.

14. The apparatus of claim 10 wherein the particular wavelength is in the infra-red or near infra-red portions of the spectrum.

15. The apparatus of claim 10 further including a second light projector operable to project a visible pattern onto the projection surface at the same time as the structured light pattern is projected onto the projection surface.

16. The apparatus of claim 10 further including a device, wherein the one or more processors are operable, in response to determining that the object's path of movement represents a specified type of gesture, to cause the device to perform a specified operation based on the type of gesture.

17. The apparatus of claim 10 wherein the one or more processors are operable, in response to determining that the object's path of movement represents a specified type of gesture, to perform at least one of the following:

cause a display monitor to display an alphanumeric character;

cause a light to be turned on, off or dimmed;

cause a computing device to transition to a next slide of a presentation.

Assignments (3)
CHANGE OF NAME Recorded Nov 3, 2025
From: AMS SENSORS SINGAPORE PTE. LTD.
To: AMS-OSRAM ASIA PACIFIC PTE. LTD.
Reel/Frame 073476/0659 →
CHANGE OF NAME Recorded Feb 8, 2019
From: HEPTAGON MICRO OPTICS PTE. LTD.
To: AMS SENSORS SINGAPORE PTE. LTD.
Reel/Frame 049222/0062 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 28, 2017
From: CUTU, FLORIN; ZHANG, CHI
To: HEPTAGON MICRO OPTICS PTE. LTD.
Reel/Frame 043126/0453 →
Continuity (2)
Provisional Application 62369255 · Aug 1, 2016
Related Publication 20180032210A1 · Feb 1, 2018