Granted Patent B2

US 12,340,628 · App. 18/600,799 · Granted Jun 24, 2025

Inventors: Chang-Hua Wei (Taoyuan, TW); Yu-Ling Huang (Taoyuan, TW)

Assignee: HTC Corporation

G06V40/28G06F3/012G06F3/017G06F3/0346G06T7/70G06V10/764G06V40/11G06T2207/20221

Abstract

Claims (75)

1. A method for hand tracking, comprising:

obtaining, through a first camera of a head-mounted device, a first image of a hand;

determining, through the processor, a first pose of a first part of the hand based on the first image;

obtaining, through a second camera of a hand-held device, a second image of the hand;

determining, through the processor, a second pose of a second part of the hand based on the second image, wherein the first part and the second part complementarily form an entirety of the hand; and

determining, through the processor, a gesture of the hand based on the first pose and the second pose.

2. The method according to claim 1 , further comprising:

classifying, through the processor, fingers of the hand into the first part of a known pose or the second part of an unknown pose based on the first image.

3. The method according to claim 1 , further comprising:

determining whether each finger of the hand is extended or bent based on the gesture.

4. The method according to claim 1 , further comprising:

determining a percentage of a field of view of the second camera that is occupied by a finger of the second part of the hand based on the second image; and

determining the finger is extended or bent based on the percentage.

5. The method according to claim 1 , further comprising:

determining a presence of a fingernail of a finger of the second part is in a field of view of the second camera based on the second image; and

determining the finger is extended or bent based on the presence.

6. The method according to claim 1 , further comprising:

determining a percentage of a field of view of the second camera that is occupied by a finger of the second part of the hand based on the second image;

determining a presence of a fingernail of the finger is in a field of view of the second camera based on the second image; and

determining the finger is extended or bent based on the percentage and the presence.

7. The method according to claim 1 , further comprising:

determining an angle between a proximal boundary of a finger of the second part and a distal boundary of the finger; and

determining the finger is extended or bent based on the angle.

8. The method according to claim 1 , further comprising:

generating a simultaneous localization and mapping map of an environment around a user by a fusion of the first image and the second image; and

determining positions and orientations of the head-mounted device and the hand-held device based on the simultaneous localization and mapping map.

9. The method according to claim 1 , further comprising:

obtaining, through a capture sensor of the hand-held device, sensor data; and

determining whether the hand is in a grasping gesture based on the sensor data.

10. The method according to claim 9 , wherein the capture sensor is a distance sensor or a contact sensor and the sensor data is configured to indicate a distance between a finger of the hand and a body part of the hand-held device.

11. A host, comprising:

a storage circuit, storing a program code; and

a processor, coupled to the storage circuit and accessing the program code to execute:

obtaining, through a first camera of a head-mounted device, a first image of a hand;

determining a first pose of a first part of the hand based on the first image;

obtaining, through a second camera a hand-held device, a second image of the hand;

determining a second pose of a second part of the hand based on the second image,

wherein the first part and the second part complementarily form an entirety of the hand; and

determining a gesture of the hand based on the first pose and the second pose.

12. The host according to claim 11 , wherein the processor is further configured to access the program code to execute:

classifying, through the processor, fingers of the hand into the first part of a known pose or the second part of an unknown pose based on the first image.

13. The host according to claim 11 , wherein the processor is further configured to access the program code to execute:

determining whether each finger of the hand is extended or bent based on the gesture.

14. The host according to claim 11 , wherein the processor is further configured to access the program code to execute:

determining a percentage of a field of view of the second camera that is occupied by a finger of the second part of the hand based on the second image; and

determining the finger is extended or bent based on the percentage.

15. The host according to claim 11 , wherein the processor is further configured to access the program code to execute:

determining a presence of a fingernail of a finger of the second part is in a field of view of the second camera based on the second image; and

determining the finger is extended or bent based on the presence.

16. The host according to claim 11 , wherein the processor is further configured to access the program code to execute:

determining a percentage of a field of view of the second camera that is occupied by a finger of the second part of the hand based on the second image;

determining a presence of a fingernail of the finger is in a field of view of the second camera based on the second image; and

determining the finger is extended or bent based on the percentage and the presence.

17. The host according to claim 11 , wherein the processor is further configured to access the program code to execute:

determining an angle between a proximal boundary of a finger of the second part and a distal boundary of the finger; and

determining the finger is extended or bent based on the angle.

18. The host according to claim 11 , wherein the processor is further configured to access the program code to execute:

generating a simultaneous localization and mapping map of an environment around a user by a fusion of the first image and the second image; and

determining positions and orientations of the head-mounted device and the hand-held device based on the simultaneous localization and mapping map.

19. A hand tracking system, comprising:

a head-mounted device, adapted to be mounted on a head and comprising:

a first camera, configured to obtain a first image of a hand;

a hand-held device, adapted to be worn on the hand and comprising:

a body part, adapted to be hold by the hand; and

an extension part, extending from the body part and comprising:

a second camera, configured to obtain a second image of the hand;

a storage circuit, storing a program code; and

a processor, coupled to the storage circuit and accessing the program code to execute:

determining a first pose of a first part of the hand based on the first image;

determining a second pose of a second part of the hand based on the second image,

wherein the first part and the second part complementarily form an entirety of the hand; and

determining a gesture of the hand based on the first pose and the second pose.

20. The hand tracking system according to claim 19 , wherein the body part comprises a capture sensor and the processor is further configured to access the program code to execute:

obtaining, through the capture sensor of the hand-held device, sensor data; and

determining whether the hand is in a grasping gesture based on the sensor data.

Assignments (1)

Continuity (2)

References Cited (16)