IP Library › Granted Patent US 11,386,285
Granted Patent B2
US 11,386,285 · App. 16/888,379 · Granted Jul 12, 2022

Systems and methods of person recognition in video streams

Inventors: Akshay R. Bapat (Mountain View, CA); George Alban Heitz (Mountain View, CA); Rizwan Ahmed Chaudhry (Palo Alto, CA); Navneet Dalal (Atherton, CA); James Edward Stewart (Mountain View, CA); Jennifer Bush (Mountain View, CA); Joe Delone Venters (Alameda, CA); Kara Gates (Mountain View, CA); Timothy Butler (Mountain View, CA); Yohannes Kifle (San Carlos, CA); Anton Bastov (Mountain View, CA)
Assignee: Google LLC
G06K9/6218G06V20/46G06V40/10G08B13/1966G08B13/19645G08B13/19663G08B13/19697H04N7/18G06F16/45G06V20/44G06V40/161G06V40/168G06V40/172H04N5/23219
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,386,285
App. No.
16/888,379
Granted
Jul 12, 2022
Kind
B2
Abstract

A method for recognizing persons in video streams includes obtaining a live video stream, detecting a first person in the live video stream, determining from analysis of the live video stream first information that identifies an attribute of the first person, determining based on at least some of the first information that the first person is not identifiable to the computing system, storing at least some of the first information, receiving a user classification of the first person as being a stranger, and deleting the stored first information.

Claims (68)

1. A method comprising:

at a computing system having one or more processors and memory:

obtaining a live video stream;

detecting a first person in the live video stream;

responsive to the detection of the first person in the live video stream, determining, from analysis of the live video stream, first information that identifies an attribute of the first person;

determining, based on at least some of the first information, that the first person is not identifiable to the computing system;

storing in a database at least some of the first information; and

in response to: (i) a determination that the first person is a stranger or (ii) not receiving a user classification of the first person within an amount of time,

deleting the at least some of the stored first information or a portion of the at least some of the stored first information.

2. The method of claim 1 , wherein determining the first information comprises:

selecting one or more images of the first person from the live video stream; and

characterizing a plurality of features of the first person based on the one or more images.

3. The method of claim 2 , further comprising:

identifying a pose of the first person in each of the one or more images; and

for each identified pose, in accordance with a determination that an image of the first person in the identified pose has not previously been stored, storing (i) the image of the first person in the identified pose, and (ii) corresponding pose information.

4. The method of claim 2 , wherein the plurality of features includes one or more features that can personally identify the first person.

5. The method of claim 2 , wherein the plurality of features includes one or more facial features of the first person.

6. The method of claim 1 , wherein:

detecting the first person comprises detecting an entity in the live video stream; and

the method further comprises:

receiving, from the user, a designation that the detected entity is not a person; and

responsive to the designation, storing the designation in association with the first information of the first person.

7. The method of claim 1 , further comprising, in accordance with the determination that the first person is not identifiable, generating a timely notification of detection of an unknown person.

8. The method of claim 1 , further comprising:

detecting a second person in the live video stream;

characterizing a plurality of features of the second person based on one or more images of the second person in the live video stream; and

determining whether the second person is a same person as the first person by comparing the plurality of characterized features of the second person with the stored first information regarding a plurality of characterized features of the first person.

9. A computing system comprising:

one or more processors; and

memory coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

obtaining a live video stream;

detecting a first person in the live video stream;

responsive to the detection of the first person in the live video stream, determining, from analysis of the live video stream, first information that identifies an attribute of the first person;

determining, based on at least some of the first information, that the first person is not identifiable to the computing system;

storing in a database at least some of the first information; and

in response to: (i) a determination that the first person is a stranger or (ii) not receiving a user classification of the first person within an amount of time,

deleting the at least some of the stored first information or a portion of the at least some of the stored first information.

10. The system of claim 9 , wherein detecting the first person comprises detecting the first person within a first predefined threshold time.

11. The system of claim 9 , wherein the one or more programs further include instructions for comparing the first information of the first person with stored first information of the first person previously classified by the user; and

wherein determining that the first person is not identifiable comprises determining that the first information of the first person does not match, within a predetermined threshold, the stored first information of the first person previously classified by the user.

12. The system of claim 9 , wherein the one or more programs further include instructions for:

determining that the first person is identifiable to the computing system; and

in accordance with the determination that the first person is identifiable, forgoing receiving the user classification of the first person.

13. The system of claim 12 , wherein the one or more programs further include instructions for forgoing generating a timely notification in accordance with the determination that the first person is identifiable.

14. The system of claim 12 , wherein the one or more programs further include instructions for generating a timely notification identifying the first person in accordance with the determination that the first person is identifiable.

15. The system of claim 14 , wherein the one or more programs further include instructions for:

determining a location of the first person;

determining whether the first person has previously been authorized by the user to access the location; and

wherein generating the timely notification identifying the first person comprises generating the timely notification identifying the first person and the location in accordance with a determination that the first person has not previously been authorized to access the location.

16. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing system, cause the system to:

obtain a live video stream;

detect a first person in the live video stream;

responsive to the detection of the first person in the live video stream, determine, from analysis of the live video stream, first information that identifies an attribute of the first person;

determine, based on at least some of the first information, that the first person is not identifiable to the computing system;

store in a database at least some of the first information; and

in response to: (i) a determination that the first person is a stranger or (ii) not receiving a user classification of the first person within an amount of time,

delete the at least some of the stored first information or a portion of the at least some of the stored first information.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the one or more programs further comprise instructions, which when executed by the computing system, cause the system to:

detect the first person by performing facial detection on a first frame in the live video stream; and

track the first person through time by analyzing a plurality of video frames in the live video stream, the plurality of video frames subsequent to the first frame.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the one or more programs further comprise instructions, which when executed by the computing system, cause the system to:

for each video frame of the first frame and the plurality of video frames, determine an identification quality factor for the video frame;

select one or more video frames of the first frame and the plurality of video frames based on the determined identification quality factors; and

wherein determining the first information comprises determining first information based on an analysis of the selected one or more video frames.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the one or more programs further comprise instructions, which when executed by the computing system, cause the system to:

determine an activity of the first person based on the tracking of the first person; and

send a notification with identification of the first person and the determined activity.

20. The non-transitory computer-readable storage medium of claim 16 , wherein the one or more programs further comprise instructions, which when executed by the computing system, cause the system to provide a notification to the first person in accordance with the determination that the first person is not identifiable.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 12, 2022
From: HEITZ, GEORGE ALBAN, III
To: GOOGLE LLC
Reel/Frame 058628/0756 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 8, 2020
From: BAPAT, AKSHAY R.; HEITZ, GEORGE ALBAN; CHAUDHRY, RIZWAN AHMED; DALAL, NAVNEET; STEWART, JAMES EDWARD; BUSH, JENNIFER; VENTERS, JOE DELONE; GATES, KARA; BUTLER, TIMOTHY
To: GOOGLE INC.
Reel/Frame 054571/0817 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 8, 2020
From: KIFLE, YOHANNES; BASTOV, ANTON
To: GOOGLE LLC
Reel/Frame 054571/0880 →
CHANGE OF NAME Recorded Dec 8, 2020
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 054635/0364 →
Continuity (4)
Continuation 16565365 · Sep 9, 2019
Continuation 15676848 · Aug 14, 2017
Provisional Application 62512697 · May 30, 2017
Related Publication 20200293818A1 · Sep 17, 2020
Cited By (3)
US 12,542,874 US 12,664,818 US 12,682,013