Granted Patent B1

US 12,633,166 · App. 18/469,703 · Granted May 19, 2026

Inventors: Rui Zhao (Bellevue, WA); Charles Joseph Tucker Peach (Kirkland, WA); Baomin Wang (State College, PA); Evan Perillo (Woodinville, WA); Abhinav Kashyap (Redmond, WA); Manoj Aggarwal (Seattle, WA); Gerard Guy Medioni (Los Angeles, CA)

Assignee: AMAZON TECHNOLOGIES, INC.

G06V40/45A61B5/117G06T7/70G06V10/143G06V40/11H04N23/11H04N23/56G06T2207/10152G06T2207/30196

Abstract

Claims (142)

1 . A device comprising:

a first camera having a first field of view (FOV);

one or more light sources to illuminate at least a portion of the first FOV;

a spectroscopy sensor having a second FOV, wherein the second FOV overlaps at least a portion of the first FOV;

a memory, storing first computer-executable instructions; and

a hardware processor to execute the first computer-executable instructions to:

operate the one or more light sources to emit a first set of wavelengths;

acquire, using the first camera, first input image data at a first time;

determine, based on the first input image data, first position data indicative of a position of a hand of a user depicted within the first input image data;

determine that the first position data is indicative of the hand of the user being within a first threshold position;

determine, based on the first input image data, first pose data indicative of a pose of the hand of the user depicted within the first input image data;

determine that the first pose data is indicative of the hand of the user having a pose that is within a first threshold pose;

responsive to the determination that the first position data is indicative of the hand of the user being within the first threshold position and the determination that the first pose data is indicative of the hand of the user having a pose that is within the first threshold pose, determine that quality of the hand depicted within the first input image data is acceptable;

responsive to the determination that the quality of the hand depicted within the first input image data is acceptable for use:

operate the one or more light sources to emit a second set of wavelengths,

operate the spectroscopy sensor at a second time to determine a first set of intensity data indicative of intensity of light for a plurality of wavelengths, and

determine, based on the first set of intensity data, a first value indicative of liveness of the hand; and

responsive to the determination that the first value is indicative of liveness of the hand:

acquire, using the first camera, second input image data at a third time; and

perform, based on the second input image data, one or more of:

a continuity check, or

a validity check.

2 . The device of claim 1 , further comprising:

a distance sensor; and

the hardware processor to further execute the first computer-executable instructions to:

determine, using the distance sensor, first distance data; and

determine that the first distance data is indicative of presence of the hand within a threshold distance.

3 . The device of claim 1 , further comprising:

a communication interface; and

the hardware processor to further execute the first computer-executable instructions to:

determine that the first time and the second time are within a first time interval;

determine representation data based on one or more of the first input image data or the second input image data; and

send the representation data using the communication interface.

4 . The device of claim 1 , the hardware processor to further execute the first computer-executable instructions to:

determine, based on the second input image data, second position data indicative of a position of the hand of the user depicted within the second input image data;

determine that the second position data is indicative of the hand of the user being within a second threshold position;

determine, based on the second input image data, second pose data indicative of a pose of the hand of the user depicted within the second input image data;

determine that the second pose data is indicative of the hand of the user having a second pose that is within a second threshold pose;

determine that the third time and the second time are within a first time interval; and

determine representation data based on the second input image data.

5 . The device of claim 1 , wherein the first camera is responsive to visible light; and

the device further comprising:

a second camera having a third FOV, wherein the second camera is responsive to infrared light; and

the hardware processor to further execute the first computer-executable instructions to:

operate the one or more light sources to emit a third set of wavelengths that comprise infrared light;

acquire, using the second camera, third input image data at a fourth time;

determine, based on the third input image data, second position data indicative of a second position of the hand of the user depicted within the third input image data;

determine that the second position data is indicative of the hand of the user being within a second threshold position;

determine, based on the third input image data, second pose data indicative of a second pose of the hand of the user depicted within the third input image data;

determine that the second pose data is indicative of the hand of the user having a second pose that is within a second threshold pose;

determine that the fourth time and the second time are within a first time interval; and

determine representation data based on the third input image data.

6 . The device of claim 1 , the hardware processor to further execute the first computer-executable instructions to:

operate the one or more light sources to emit the second set of wavelengths at the second time as a sequence such that a first wavelength is emitted at a fourth time and a second wavelength is emitted at a fifth time; and

operate the spectroscopy sensor to determine first intensity data at the fourth time and second intensity data at the fifth time, wherein the first set of intensity data comprises the first intensity data and the second intensity data; and

wherein the spectroscopy sensor comprises a photodetector.

7 . The device of claim 1 , the hardware processor to further execute the first computer-executable instructions to:

operate the one or more light sources to emit the second set of wavelengths such that a first wavelength and a second wavelength are emitted at the second time; and

wherein the spectroscopy sensor comprises a first detector portion that is sensitive to the first wavelength and a second detector portion that is sensitive to the second wavelength.

8 . A method comprising:

operating one or more light sources to emit a first set of wavelengths;

acquiring, using a first camera having a first field of view (FOV), a first set of input images during a first time interval, wherein the first set of input images depict a hand of a user;

determining, based on the first set of input images, first characteristic data associated with the hand;

determining that the first characteristic data is within a specified threshold range;

based on the determining that the first characteristic data is within the specified threshold range, determining that a quality of the hand as depicted within the first set of input images is acceptable;

responsive to the determining that the quality of the hand as depicted within the first set of input images is acceptable:

operating the one or more light sources to emit a second set of wavelengths,

operating a first sensor during a second time interval to determine a first set of intensity data indicative of intensity of light for a plurality of wavelengths, and

determining, based on the first set of intensity data, a first value that is indicative of liveness of the hand; and

responsive to the determining that the first value is indicative of liveness of the hand:

acquire a second set of input images during a third time interval, and

check for validity of one or more of the first set of images or the second set of images.

9 . The method of claim 8 , wherein the first characteristic data is indicative of one or more of:

first position data indicative of a position of a hand within the first FOV, or

first pose data indicative of a pose of the hand within the first FOV.

10 . The method of claim 8 , further comprising:

determining a first subset of the first set of input images, wherein for each image of the first subset:

the each image is associated with a respective timestamp, and

the respective timestamp is within a fourth time interval of the second time interval; and

determining representation data based on the first subset.

11 . The method of claim 8 , wherein the first camera is responsive to visible light; and

the method further comprising:

operating the one or more light sources to emit a third set of wavelengths that comprise infrared light;

acquiring, using a second camera that is responsive to infrared light, a third set of input images during a second-fourth time interval;

determining, based on the third set of input images, second characteristic data;

determining a first subset of the third set of input images, wherein for each image of the first subset of the third set of input images:

the second characteristic data associated with the each image is within the specified threshold range,

the each image is associated with a respective timestamp, and

the respective timestamp is within a fifth time interval of the second time interval; and

determining representation data based on the first subset of the third set of input images.

12 . The method of claim 8 , wherein the second time interval is within the third time interval, and the method further comprising:

determining representation data based on the second set of input images.

13 . The method of claim 8 , further comprising:

determining, based on the second set of input images, second characteristic data;

determining that the second characteristic data associated with each image of the second set of images satisfies a specified threshold value; and

determining representation data based on the second set of input images.

14 . A device comprising:

a first camera having a first field of view (FOV);

one or more light sources to illuminate at least a portion of the first FOV;

a first sensor having a second FOV, wherein the second FOV overlaps at least a portion of the first FOV;

a memory, storing first computer-executable instructions; and

a hardware processor to execute the first computer-executable instructions to:

operate the one or more light sources to emit a first set of wavelengths;

acquire, using the first camera, a first set of input images during a first time interval, wherein the first set of input images depicts a hand of a user;

determine, based on the first set of input images, first characteristic data associated with the hand;

determine that the first characteristic data is within a specified threshold range;

based on the determination that the first characteristic data is within the specified threshold range, determine that a quality of the hand as depicted within the first set of input images is acceptable;

responsive to the determination that the quality of the hand as depicted within the first set of input images is acceptable:

operate the one or more light sources to emit a second set of wavelengths,

operate the first sensor during a second time interval to determine a first set of intensity data indicative of intensity of light for a plurality of wavelengths, and

determine, based on the first set of intensity data, a first value indicative of liveness of the hand.

15 . The device of claim 14 , wherein the first characteristic data associated with the hand is indicative of one or more of:

first position data indicative of a position of the hand within the first FOV, or

first pose data indicative of a pose of the hand within the first FOV.

16 . The device of claim 14 , further comprising:

a distance sensor; and

the hardware processor to further execute the first computer-executable instructions to:

determine, using the distance sensor, first distance data; and

determine the first distance data is indicative of presence of the hand within a threshold distance.

17 . The device of claim 14 , the hardware processor to further execute the first computer-executable instructions to:

determine a first subset of the first set of input images, wherein for each image in the first subset:

the each image is associated with a respective timestamp, and

the respective timestamp is within a third time interval of the second time interval; and

determine representation data based on the first subset.

18 . The device of claim 14 , wherein the first camera is responsive to visible light; and the device further comprising:

a second camera having a third FOV, wherein the second camera is responsive to infrared light;

the hardware processor to further execute the first computer-executable instructions to:

operate the one or more light sources to emit a third set of wavelengths that comprise infrared light;

acquire, using the second camera, a third set of input images during a third time interval;

determine, based on the third set of input images, second characteristic data;

determine a first subset of the third set of input images, wherein for each image of the first subset of the third set of input images:

the second characteristic data associated with the each image is within the specified threshold range,

the each image is associated with a respective timestamp, and

the respective timestamp is within a fourth time interval of the second time interval; and

determine representation data based on the first subset.

19 . The device of claim 14 , the hardware processor to execute the first computer-executable instructions to:

operate the one or more light sources to emit the second set of wavelengths as a sequence such that a first wavelength is emitted during a third time interval and a second wavelength is emitted during a fourth time interval; and

operate the first sensor to determine first intensity data during the third time interval and second intensity data during the fourth time interval, wherein the first set of intensity data comprises the first intensity data and the second intensity data; and

wherein the first sensor comprises a photodetector.

20 . The device of claim 14 , the hardware processor to further execute the first computer-executable instructions to:

operate the one or more light sources to emit the second set of wavelengths such that a first wavelength and a second wavelength are emitted at a same time; and

wherein the first sensor comprises a first detector portion that is sensitive to the first wavelength and a second detector portion that is sensitive to the second wavelength.

Assignments (1)

References Cited (16)