IP Library › Granted Patent US 11,783,010
Granted Patent B2
US 11,783,010 · App. 17/805,713 · Granted Oct 10, 2023

Systems and methods of person recognition in video streams

Inventors: Akshay R. Bapat (Mountain View, CA); George Alban Heitz, III (Mountain View, CA); Rizwan Ahmed Chaudhry (Palo Alto, CA); Navneet Dalal (Atherton, CA); James Edward Stewart (Mountain View, CA); Jennifer Bush (Mountain View, CA); Joe Delone Venters (Alameda, CA); Kara Gates (Mountain View, CA); Timothy Butler (Mountain View, CA); Yohannes Kifle (San Carlos, CA); Anton Bastov (Mountain View, CA)
Assignee: Google LLC
G06F18/23G06V20/46G06V40/10G08B13/1966G08B13/19645G08B13/19663G08B13/19697H04N7/18G06F16/45G06V20/44G06V40/161G06V40/168G06V40/172H04N23/611
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,783,010
App. No.
17/805,713
Granted
Oct 10, 2023
Kind
B2
Abstract

A method for recognizing persons in video streams includes obtaining a live video stream, detecting a first person in the live video stream, determining from analysis of the live video stream first information that identifies an attribute of the first person, determining based on at least some of the first information that the first person is not identifiable to the computing system, storing at least some of the first information, receiving a user classification of the first person as being a stranger, and deleting the stored first information.

Claims (41)

1. A method comprising:

obtaining information associated with a first person, the information associated with the first person including one or more images of the first person captured while in a field of view of a camera associated with a smart-home environment;

determining, based on the information associated with the first person, that the first person is not known to a computing system having a stored set of classified persons, the determination sufficient to classify the first person as an unknown person;

storing, in the stored set of classified persons, at least some of the information associated with the first person and the unknown person classification; and

in response to (i) a user designation to delete information associated with the first person or (ii) not receiving a user classification that alters the unknown person classification within a predetermined amount of time, deleting at least portions of the information associated with the first person, the at least portions of the information associated with the first person including at least one of audio data containing speech of the first person or one or more images of the first person.

2. The method as described in claim 1 , wherein information associated with the first person further includes at least one of:

one or more physical characteristics of the first person, including a facial feature signature;

one or more behavioral characteristics of the first person;

one or more clothing or accessory characteristics of the first person;

a time of day during which the one or more images were captured of the first person;

a day of a week during which the one or more images were captured of the first person;

one or more actions of the first person.

3. The method as described in claim 2 , wherein obtaining information associated with the first person comprises:

receiving audio/video data of the first person while in the field of view of the camera associated with the smart-home environment;

selecting one or more images of the first person from the audio/video data; and

characterizing a plurality of features of the first person based on the one or more images.

4. The method as described in claim 1 , further comprising maintaining at least other portions of the information associated with the first person in the stored set of classified persons.

5. The method as described in claim 1 , further comprising prompting, responsive to determining that the first person is not known to the computing system, the user to provide the user classification of the first person before the predetermined amount of time elapses.

6. The method as described in claim 1 , further comprising providing an option for the user to designate at least portions of the information associated with the first person for deletion.

7. The method as described in claim 6 , wherein the option for the user to designate at least portions of the information associated with the first person for deletion includes an image of a face of the first person.

8. The method as described in claim 1 , further comprising receiving, responsive to storing at least some of the information associated with the first person with the unknown person classification, input denying permission to maintain storage of the information associated with the first person.

9. The method as described in claim 8 , wherein deleting at least some portions of the information associated with the first person is in further response to the input denying permission to maintain storage.

10. The method as described in claim 1 , wherein the predetermined amount of time comprises 1 day, 1 week, 2 weeks, or 1 month.

11. The method as described in claim 1 , wherein deleting in response to the user designation to delete information associated with the first person deletes the information associated with the first person immediately.

12. The method as described in claim 1 , wherein deleting at least portions of the information associated with the first person is in further response to the user classification classifying the first person as a stranger.

13. The method as described in claim 12 , wherein deleting in response to the user classification classifying the first person as the stranger deletes the information associated with the first person after a second predetermined amount of time.

14. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing system, cause the computing system to:

obtain information associated with a first person, the information associated with the first person including one or more images of the first person captured while in a field of view of a camera associated with a smart-home environment;

determine, based on the information associated with the first person, that the first person is not known to a computing system having a stored set of classified persons, the determination sufficient to classify the first person as an unknown person;

store, in the stored set of classified persons, at least some of the information associated with the first person with the unknown person classification; and

in response to (i) a user designation to delete information associated with the first person or (ii) not receiving a user classification that alters the unknown person classification within a predetermined amount of time, delete at least portions of the information associated with the first person, the at least portions of the information associated with the first person including at least one of audio data containing speech of the first person or one or more images of the first person.

15. The non-transitory computer-readable storage medium of claim 14 , wherein the computing system, responsive to the user designation to deleted information associated with the first person, immediately deletes the at least portions of the information associated with the first person.

16. The non-transitory computer-readable storage medium of claim 14 , wherein the instructions, which when executed by the computing system, further cause the computing system to:

maintain at least other portions of the information associated with the first person in the stored set of classified persons.

17. The non-transitory computer-readable storage medium of claim 14 , wherein the instructions, which when executed by the computing system, further cause the computing system to:

prompt, responsive to the determination that the first person is not known to the computing system, the user to provide the user classification of the first person before the predetermined amount of time elapses.

18. The non-transitory computer-readable storage medium of claim 14 , wherein the instructions, which when executed by the computing system, further cause the computing system to:

provide an option for the user to designate at least portions of the information associated with the first person for deletion.

19. The non-transitory computer-readable storage medium of claim 14 , wherein the instructions, which when executed by the computing system, further cause the computing system to:

receive input denying permission to maintain storage of the information associated with the first person, and wherein the deletion of at least some portions of the information associated with the first person is in further response to the input denying permission to maintain storage.

20. The non-transitory computer-readable storage medium of claim 14 , wherein the computing system deletes at least portions of the information associated with the first person further in response to the user classification classifying the first person as a stranger.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 7, 2022
From: BAPAT, AKSHAY R.; HEITZ, GEORGE ALBAN; CHAUDHRY, RIZWAN AHMED; DALAL, NAVNEET; STEWART, JAMES EDWARD; BUSH, JENNIFER; VENTERS, JOE DELONE; GATES, KARA; BUTLER, TIMOTHY
To: GOOGLE INC.
Reel/Frame 060119/0941 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 7, 2022
From: KIFLE, YOHANNES; BASTOV, ANTON
To: GOOGLE LLC
Reel/Frame 060120/0040 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 7, 2022
From: HEITZ, GEORGE ALBAN, III
To: GOOGLE LLC
Reel/Frame 060120/0144 →
CHANGE OF NAME Recorded Jun 7, 2022
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 060301/0223 →
Continuity (5)
Continuation 16888379 · May 29, 2020
Continuation 16565365 · Sep 9, 2019
Continuation 15676848 · Aug 14, 2017
Provisional Application 62512697 · May 30, 2017
Related Publication 20220300753A1 · Sep 22, 2022
Cited By (3)
US 12,542,874 US 12,664,818 US 12,682,013