Granted Patent B2

US 12,230,029 · App. 16/904,544 · Granted Feb 18, 2025

Inventors: Imran A. Chaudhri (San Francisco, CA); Patrick Gates (San Francisco, CA); Monique Relova (South San Francisco, CA); Bethany Bongiorno (San Francisco, CA); Brian Huppi (San Francisco, CA); Shahzad Chaudhri (Arlington, VA)

Assignee: Humane, Inc.

G06V20/20G06F3/017H04N7/18

Abstract

Claims (53)

1. A body-worn apparatus comprising:

a camera;

a depth sensor;

a laser projection system;

one or more processors;

memory storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations comprising:

capturing, using the camera, a first set of digital images;

identifying a real-world object in the first set of digital images;

capturing, using the depth sensor, first depth data;

identifying, in the first set of digital images and the first depth data, a first gesture of a user wearing the apparatus,

wherein identifying the real-world object and the first gesture includes:

processing the first set of digital images through an object detection framework that uses a complex polygon to identify a hotspot region in the first set of images, wherein the hotspot region is smaller than the entire image and captures the first gesture and the real-world object while excluding all other objects in the first set of images;

sending, to a cloud computing platform, the hotspot region;

receiving, from the cloud computing platform, information related to the real-world object; and

projecting, with the laser projection system, at least some of the information on a surface.

2. The apparatus of claim 1 , wherein the information includes a text label for the real-world object.

3. The apparatus of claim 1 , wherein the information includes instructions for performing an action on the real-world object.

4. The apparatus of claim 1 , wherein the information includes a command for controlling the real-world object.

5. The apparatus of claim 1 , wherein the operations further comprise:

obtaining, using the depth sensor, a second gesture associated with the laser projection;

determining user input based on the second gesture; and

initiating one or more actions in accordance with the user input.

6. The apparatus of claim 5 , wherein the operations further comprise:

masking the laser projection to prevent projecting the data on a hand of the user making the second gesture.

7. The apparatus of claim 1 , the operations further comprising:

capturing, using the camera, reflections of the laser projection from the surface;

automatically adjusting an intensity of the laser projection to compensate for different indexes of refraction so that the laser projection has a uniform brightness.

8. The apparatus of claim 1 , further comprising:

a magnetic attachment mechanism configured to magnetically couple to a battery pack through a user's clothing, the magnetic attachment mechanism further configured to receive inductive charging from the battery back.

9. A method comprising:

capturing, using a camera, a set of digital images;

capturing, using a depth sensor of a body-worn apparatus, depth data;

identifying a real-world object in the set of digital images;

identifying, in the set of digital images and depth data, a first gesture, the first gesture made by a user wearing the apparatus;

wherein identifying the real-world object and the first gesture includes:

processing the set of digital images through an object detection framework that uses a complex polygon to identify a hotspot region in the set of images, wherein the hotspot region is less than the entire image and captures the first gesture and the real-world object while excluding all other objects in the set of images;

sending, to a cloud computing platform, the hotspot region;

receiving, from the cloud computing platform, information related to the real-world object; and

projecting, with the laser projection system, at least some of the information on a surface.

10. The method of claim 9 , further comprising:

obtaining, using the depth sensor, a second gesture by the user, the second gesture associated with the laser projection;

determining user input based on the second gesture; and

initiating one or more actions in accordance with the user input.

11. The method of claim 10 , wherein the one or more actions include controlling the real-world object.

12. The method of claim 11 , wherein the controllable real-world object is a television or computer screen, and the one or more actions include swiping through one or more images displayed by the television or computer screen.

13. The method of claim 11 , wherein the controllable real-world object is a thermostat and the one or more actions including changing a temperature setting of the thermostat.

14. The method of claim 10 , further comprising:

masking the laser projection to prevent projecting the data on a hand of the user making the second gesture.

15. The method of claim 9 , wherein the laser projection includes a size template for measuring the real-world object.

16. The method of claim 9 , further comprising:

receiving audio input from the user; and

associating the first gesture and audio input with a request or command for controlling the real-world object.

17. The method of claim 9 , where the surface is a palm of the user's hand.

Assignments (2)

Continuity (4)

References Cited (77)