IP Library Granted Patent US 12,405,657
Granted Patent B2
US 12,405,657 · App. 18/398,022 · Granted Sep 2, 2025

Hand-specific laser projected virtual interfaces and operations

Inventors: Kenneth Luke Kocienda (Mill Valley, CA); Imran A. Chaudhri (San Francisco, CA)
Assignee: Hewlett-Packard Development Company, L.P.
G06F3/011G06F3/017G06F3/0482G06T7/70G06V40/107G06T2207/30196G09G2354/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,405,657
App. No.
18/398,022
Granted
Sep 2, 2025
Kind
B2
Abstract

Systems, methods, devices and non-transitory, computer-readable storage mediums are disclosed for a wearable multimedia device and cloud computing platform with an application ecosystem for processing multimedia data captured by the wearable multimedia device. In an embodiment, a wearable multimedia device obtains first configuration data specifying first operations associated with a first hand of a user of the wearable multimedia device, and second configuration data specifying second operations associated with a second hand of the user. Further, the device captures sensor data from a camera and/or a depth sensor of the wearable multimedia device, and determines a presence of the first hand and/or the second hand based on the sensor data. Further, the device performs at least one of (i) the first operations responsive to determining the presence of the first hand, or (ii) the second operations responsive to determining the presence of the second hand.

Claims (83)

1. A method comprising:

capturing, by a wearable multimedia device, sensor data from at least one of a camera or a depth sensor of the wearable multimedia device;

determining, by the wearable multimedia device, based on the sensor data, a presence of at least one of a first hand or a second hand; and

at least one of:

responsive to determining the presence of the first hand, projecting, using a laser projector of the wearable multimedia device, a first virtual interface (VI) of a first application on a first surface of the first hand, or

responsive to determining the presence of the second hand, projecting, using the laser projector, a second VI of a second application on a second surface of the second hand,

wherein the first application is different from the second application.

2. The method of claim 1 , wherein the first surface is a palm of the first hand, and wherein

the second surface is a palm of the second hand.

3. The method of claim 1 , wherein the first application is selected from a first set of

applications, and wherein the second application is selected from a second set of applications different from the first set of applications.

4. The method of claim 1 , further comprising:

determining, based on configuration data, that the first application is associated with the first hand, and

determining, based on the configuration data, that the second application is associated with the second hand.

5. The method of claim 4 , wherein the configuration data is specified, at least in part, by a user.

6. The method of claim 1 , wherein determining the presence of at least one of the first hand or the second hand comprises:

determining that at least one of the first hand or the second hand is in a field of view of at least one of the camera or the depth sensor.

7. The method of claim 1 , wherein determining the presence of at least one of the first hand or the second hand comprises:

determining that at least one of the first hand or the second hand is in a projection area of the laser projector.

8. The method of claim 1 , further comprising:

determining, based on the sensor data, an orientation of the first hand, and at least one of:

performing one or more first operations responsive to determining than the first hand is oriented in a first orientation, or

performing one or more second operations responsive to determining than the first hand is oriented in a second orientation, wherein the first orientation is different form the second orientation.

9. The method of claim 8 , wherein the first orientation corresponds to a palm of the first hand facing the wearable multimedia device, and wherein the second orientation corresponds to a back of the first hand facing the wearable multimedia device.

10. The method of claim 8 , further comprising:

determining, based on the sensor data, an orientation of the second hand, and at least one of:

performing one or more third operations responsive to determining than the second hand is oriented in a third orientation, or

performing one or more fourth operations responsive to determining than the second hand is oriented in a fourth orientation, wherein the third orientation is different form the fourth orientation.

11. The method of claim 10 , wherein the third orientation corresponds to a palm of the second hand facing the wearable multimedia device, and wherein the fourth orientation corresponds to a back of the second hand facing the wearable multimedia device.

12. The method of claim 1 , further comprising:

determining, based on the sensor data, a number of fingers of the first hand that are extended outward,

determining one or more first operations based on the number of fingers of the first hand that are extended outward, and

performing the one or more first operations.

13. The method of claim 12 , further comprising:

determining, based on the sensor data, a number of fingers of the second hand that are extended outward,

determining one or more second operations based on the number of fingers of the second hand that are extended outward, and

performing the one or more second operations, wherein the one or more first operations are different from the one or more second operations.

14. The method of claim 1 , wherein the first application is one of:

a note taking application,

a calendar application,

a messaging application,

a content viewing application,

a map application,

a web browser applications,

a financial application,

a shopping application, or

a productivity application.

15. The method of claim 14 , wherein the second application is another one of:

the note taking application,

the calendar application,

the messaging application,

the map application,

the web browser applications,

the financial application,

the shopping application, or

the productivity application.

16. A wearable multimedia device, comprising:

at least one of a camera or a depth sensor;

a laser projector;

one or more processors; and

memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

capturing sensor data using at least one of the camera or the depth sensor;

determining, based on the sensor data, a presence of at least one of a first hand or a second hand; and

at least one of:

responsive to determining the presence of the first hand, projecting, using the laser projector, a first virtual interface (VI) of a first application on a first surface of the first hand, or

responsive to determining the presence of the second hand, projecting, using the laser projector, a second VI of a second application on a second surface of the second hand,

wherein the first application is different from the second application.

17. The wearable multimedia device of claim 16 , wherein

the first surface is a palm of the first hand, and wherein

the second surface is a palm of the second hand.

18. The wearable multimedia device of claim 16 , the operations further comprising:

determining, based on configuration data, that the first application is associated with the first hand, and

determining, based on the configuration data, that the second application is associated with the second hand.

19. One or more non-transitory computer-readable media storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

capturing, by a wearable multimedia device, sensor data from at least one of a camera or a depth sensor of the wearable multimedia device;

determining, by the wearable multimedia device, based on the sensor data, a presence of at least one of a first hand or a second hand; and

at least one of:

responsive to determining the presence of the first hand, projecting, using a laser projector of the wearable multimedia device, a first virtual interface (VI) of a first application on a first surface of the first hand, or

responsive to determining the presence of the second hand, projecting, using the laser projector, a second VI of a second application on a second surface of the second hand,

wherein the first application is different from the second application.

20. The one or more non-transitory computer-readable media claim 19 , the operations further comprising:

determining, based on configuration data, that the first application is associated with the first hand, and

determining, based on the configuration data, that the second application is associated with the second hand.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 8, 2025
From: HUMANE, INC.
To: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
Reel/Frame 071844/0747 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 9, 2025
From: KOCIENDA, KENNETH LUKE; CHAUDHRI, IMRAN A.
To: HUMANE, INC.
Reel/Frame 070793/0961 →
Continuity (2)
Continuation 17687533 · Mar 4, 2022
Related Publication 20240126363A1 · Apr 18, 2024
References Cited (22)
US 8519971B1 · Mackraz · 2013 [cited by applicant]
US 11868516B2 · Kocienda · 2024 [cited by examiner]
US 20100199232A1 · Mistry et al. · 2010 [cited by applicant]
US 20140055343A1 · Kim · 2014 [cited by examiner]
US 20150208019A1 · Stewart et al. · 2015 [cited by applicant]
US 20170123487A1 · Hazra et al. · 2017 [cited by applicant]
US 20170269808A1 · Chang · 2017 [cited by applicant]
US 20190121522A1 · Davis et al. · 2019 [cited by applicant]
US 20200111394A1 · Rakshit et al. · 2020 [cited by applicant]
US 20210117680A1 · Chaudhri et al. · 2021 [cited by applicant]
US 20230280822A1 · Kocienda et al. · 2023 [cited by applicant]
JP 2022012070 · 2022 [cited by applicant]
International Preliminary Report on Patentability in International Appln. No. PCT/US2023/014498, mailed on Sep. 19, 2024, 7 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln. No. PCT/US2023/014380, mailed on Apr. 4, 2023, 8 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln. No. PCT/US2023/014498, mailed on Mar. 29, 2023, 7 pages. [cited by applicant]
Kanel, “Sixth Sense Technology,” Thesis for the Bachelor Degree of Engineering in Information and Technology, Centria Univeristy of Applied Sciences, May 2014, 46 pages. [cited by applicant]
Mann et al., “Telepointer: Hands-Free Completely Self Contained Wearable Visual Augmented Reality without Headwear and without any Infrastructural Reliance”, IEEE Fourth International Symposium on Wearable Computers, At… [cited by applicant]
Mann, “Wearable Computing: A First Step Toward Personal Imaging,” IEEE Computer, Feb. 1997, 30(2):25-32. [cited by applicant]
Mann, “Wearable, tetherless computer-mediated reality,” American Association of Artificial Intelligence Technical Report, Feb. 1996, 62-69, 8 pages. [cited by applicant]
Metavision.com [online], “Sensularity with a Sixth Sense,” available on or before Apr. 7, 2015, via Internet Archive: Wayback Machine URL <http://web.archive.org/web/20170901072037/https://blog.metavision.com/professor-… [cited by applicant]
Mistry et al., “WUW—wear Ur world: a wearable gestural interface”, Proceedings of the 27th international conference Extended Abstracts on Human Factors in Computing Systems —CHI EA '09, Boston, MA, USA, Apr. 4-9, 2009, … [cited by applicant]
Shetty et al., “Sixth Sense Technology,” International Journal of Science and Research, Dec. 2014, 3(12):1068-1073. [cited by applicant]