IP Library › Granted Patent US 12,072,440
Granted Patent B2
US 12,072,440 · App. 16/497,968 · Granted Aug 27, 2024

Identification system for subject or activity identification using range and velocity data

Inventors: Girish Acharya (Redwood City, CA); Douglas Bercow (Menlo Park, CA); John Brian Burns (Palo Alto, CA); Bradley J. Clymer (Austin, TX); Aaron J. Heller (Belmont, CA); Jeffrey Lubin (Princeton, NJ); Bhaskar Ramamurthy (Los Altos, CA); David Watters (Sunnyvale, CA); Aravind Sundaresan (San Jose, CA)
Assignee: SRI INTERNATIONAL
G01S7/415G01S7/412G01S7/417G01S13/584
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,072,440
App. No.
16/497,968
Granted
Aug 27, 2024
Kind
B2
Abstract

An identification system includes a radar sensor configured to generate a time-domain or frequency-domain signal representative of electromagnetic waves reflected from one or more objects within a three-dimensional space over a period of time and a computation engine executing on one or more processors. The computation engine is configured to process the time-domain or frequency-domain signal to generate range and velocity data indicating motion by a living subject within the three-dimensional space. The computation engine is further configured to identify, based at least on the range and velocity data indicating the motion by the living subject, the living subject and output an indication of an identity of the living subject.

Claims (58)

1. An identification system comprising:

a radar sensor configured to generate a time-domain or frequency-domain signal representative of electromagnetic waves reflected from one or more objects within a three-dimensional space over a period of time; and

a computation engine executing on one or more processors and configured to process the time-domain or frequency-domain signal to generate range and velocity data indicating motion of one or more of a head or an arm of a living subject within the three-dimensional space as two or more two-dimensional range-doppler images,

wherein the computation engine is configured to identify, based at least on the two or more two-dimensional range-doppler images indicating the motion of the one or more of the head or the arm of the living subject and on profile data representative of motions of one or more living subjects, biographical data for the living subject and output an indication of an identity of the living subject.

2. The system of claim 1 , wherein the computation engine is configured to process the range and velocity data using a machine learning system to match the motion by the living subject within the three-dimensional space to motions of one or more living subjects learned by the machine learning system.

3. The system of claim 2 , wherein the machine learning system is configured to apply a neural network (NN) to the range and velocity data to identify the living subject.

4. The system of claim 2 , wherein the machine learning system is configured to apply a convolution neural network (CNN) that learns a feature from the range and velocity data;

wherein the machine learning system is configured to apply a long short term memory (LSTM) network that learns temporal and dynamic aspects of the feature, and

wherein the machine learning system is configured to identify the living subject using a combination of results from both the CNN and the LSTM network.

5. The system of claim 1 , wherein the indication of the identity of the living subject comprises one or more of:

tracking information for the living subject that indicates a location of the living subject within the three-dimensional space;

activity information for the living subject that indicates one or more activities performed by the living subject; and

passive training information for training a machine learning system, the passive training information indicating identification data for the living subject, the identification data descriptive of one or more activities of the living subject.

6. The system of claim 1 , wherein the computation engine is configured to identify the biographical data based on the motion of both the head and arm of the living subject.

7. The system of claim 1 ,

wherein the motion by the living subject within the three-dimensional space corresponds to one or more activities performed by the living subject, and

wherein, to identify the living subject, the computation engine matches the one or more activities performed by the living subject to learned activities for one or more living subjects stored to profile data for the one or more living subjects.

8. The system of claim 1 , wherein the computation engine is configured to initiate an enrollment process for the living subject to at least cause the sensor to generate a training time-domain or frequency-domain signal representative of the three-dimensional space,

wherein the computation engine is configured to receive, as part of the enrollment process, the training time-domain or frequency-domain signal,

wherein the computation engine is configured to process the training time-domain or frequency-domain signal to generate training range and velocity data indicating motion by the living subject performing one or more enrollment activities, and

wherein the computation engine is configured to process, using a machine learning system, the training range and velocity data to generate a profile for the living subject for the one or more enrollment activities.

9. The system of claim 8 , further comprising:

a hardware user interface configured to output a challenge to the living subject to perform the one or more enrollment activities,

wherein, to initiate the enrollment process, the computation engine is configured to cause the hardware user interface to output a message to the living subject to perform the one or more enrollment activities.

10. The system of claim 8 , wherein, to initiate the enrollment process, the computation engine is configured to determine that the system does not store a profile for the living subject and that the living subject is to perform the one or more enrollment activities.

11. The system of claim 1 , further comprising:

a physiological sensor that receives physiological data indicating one or more physiological characteristics of the living subject,

wherein the computation engine is configured to receive the physiological data, and

wherein, to identify the living subject, the computation engine identifies the living subject based at least on the one or more physiological characteristics of the living subject and the range and velocity data indicating the motion by the living subject.

12. The system of claim 1 , wherein the one or more objects comprise at least one object in addition to the living subject and wherein the range and velocity data indicate the at least one object in addition to the living subject.

13. The system of claim 1 ,

wherein, to identify the biographical data, the computation engine is configured to process the two or more two-dimensional range-doppler images to match the motion of the one or more of the head or the arm of the living subject within the three-dimensional space to the motions of one of the one or more living subjects represented by the profile data.

14. A method comprising:

generating a time-domain or frequency-domain signal representative of electromagnetic waves reflected from one or more objects within a three-dimensional space over a period of time;

processing, by a computation engine executing on one or more processors, the time-domain or frequency-domain signal to generate range and velocity data indicating motion of one or more of a head or an arm of a living subject within the three-dimensional space as two or more two-dimensional range-doppler images;

identifying, by the computation engine, based at least on the two or more two-dimensional range-doppler images indicating the motion of the one or more of the head or the arm of the living subject and on profile data representative of motions of one or more living subjects, biographical data for the living subject; and

outputting, by the computation engine, an indication of an identity of the living subject.

15. The method of claim 14 , further comprising processing the range and velocity data using a machine learning system to match the motion by the living subject within the three-dimensional space to motions of one or more living subjects learned by the machine learning system.

16. The method of claim 15 , wherein processing the range and velocity data using the machine learning system comprises applying a neural network (NN) to the range and velocity data to identify the living subject.

17. The method of claim 15 , wherein processing the range and velocity data using the machine learning system comprises:

applying a convolution neural network (CNN) that learns a feature from the range and velocity data;

applying a long short term memory (LSTM) network that learns temporal and dynamic aspects of the feature; and

identifying the living subject using a combination of results from both the CNN and the LSTM network.

18. The method of claim 14 ,

wherein identifying the biographical data comprises processing the two or more two-dimensional range-doppler images to match the motion of the one or more of the head or the arm of the living subject within the three-dimensional space to the motions of one of the one or more living subjects represented by the profile data.

19. A computing system comprising:

a memory; and

one or more processors coupled to the memory, wherein the one or more processors are configured to:

receive a time-domain or frequency-domain signal, generated by a radar sensor, that is representative of electromagnetic waves reflected from one or more objects within a three-dimensional space over a period of time;

process the time-domain or frequency-domain signal to generate range and velocity data indicating motion of one or more of a head or an arm of a living subject within the three-dimensional space as two or more two-dimensional range-doppler images;

identify, based at least on the two or more two-dimensional range-doppler images indicating the motion of the one or more of the head or the arm of the living subject and on profile data representative of motions of one or more living subjects, biographical data for the living subject; and

output an indication of an identity of the living subject.

20. The computing system of claim 19 ,

wherein, to identify the biographical data, the one or more processors are configured to process the two or more two-dimensional range-doppler images to match the motion of the one or more of the head or the arm of the living subject within the three-dimensional space to the motions of one of the one or more living subjects represented by the profile data.

21. An identification system comprising:

a radar sensor configured to generate a time-domain or frequency-domain signal representative of electromagnetic waves reflected from one or more objects within a three-dimensional space over a period of time; and

a computation engine executing on one or more processors and configured to process the time-domain or frequency-domain signal to generate range and velocity data indicating motion of one or more of a head or an arm of a living subject within the three-dimensional space as two or more two-dimensional range-doppler images,

wherein the computation engine is configured to identify, based at least on the two or more two-dimensional range-doppler images indicating the motion of the one or more of the head or the arm of the living subject and on profile data representative of motions of one or more living subjects, an activity being performed by the living subject and output an indication of the activity being performed by the living subject.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 26, 2019
From: ACHARYA, GIRISH; BERCOW, DOUGLAS; BURNS, JOHN BRIAN; CLYMER, BRADLEY J.; HELLER, AARON J.; LUBIN, JEFFREY; RAMAMURTHY, BHASKAR; WATTERS, DAVID; SUNDARESAN, ARAVIND
To: SRI INTERNATIONAL
Reel/Frame 050501/0753 →
Continuity (2)
Provisional Application 62477910 · Mar 28, 2017
Related Publication 20200341114A1 · Oct 29, 2020
Cited By (1)
US 12,560,702