IP Library › Granted Patent US 10,657,382
Granted Patent B2
US 10,657,382 · App. 16/460,706 · Granted May 19, 2020

Methods and systems for person detection in a video feed

Inventors: Rizwan Ahmed Chaudhry (Palo Alto, CA); Navneet Dalal (Mountain View, CA); Jonathan Z. Ben-Meshulam (Mountain View, CA); George Alban Heitz, III (Mountain View, CA)
Assignee: GOOGLE LLC
G06K9/00765G06K9/00771G06K9/6219G06T7/246H04N7/18H04N7/181G06K9/00369G06K9/46G06K2009/00738G06T2207/10016G06T2207/20016G06T2207/30196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,657,382
App. No.
16/460,706
Granted
May 19, 2020
Kind
B2
Abstract

The various embodiments described herein include methods, devices, and systems for providing event alerts. In one aspect, a method includes obtaining a video feed. A frame of the video feed is analyzed at a first resolution to determine whether the frame includes a potential instance of a person. In accordance with the determination that the image includes the potential instance, a region is denoted around the potential instance. The region is analyzed at a second resolution, greater than the first resolution. In accordance with a determination that the region includes the instance of the person. a determination that the frame includes the person is made. An indication of the determination is stored for use in subsequent alert notification processing.

Claims (77)

1. A method, comprising:

at a computing system having one or more processors and memory:

obtaining a video feed; and

for a frame of the video feed, analyzing the frame at a first resolution to determine whether the frame includes a potential instance of a person;

in accordance with a determination that the frame includes the potential instance, denoting a region around the potential instance;

analyzing the region at a second resolution, greater than the first resolution; and

in accordance with a determination that the region includes the instance of the person:

determining that the frame includes the person; and

storing an indication of the determination for use in subsequent alert notification processing.

2. The method of claim 1 , wherein the video feed comprises a plurality of frames, the method further comprising, for each frame of the plurality of frames, assigning a confidence score to the frame.

3. The method of claim 2 , wherein the confidence score is based on the analysis of the frame at the first resolution.

4. The method of claim 2 , wherein the confidence score is based on the analysis of the region at the second resolution.

5. The method of claim 2 , wherein the video feed includes a motion event, and the method further comprises, in accordance with a determination that the confidence score for at least one frame of the plurality of frames exceeds a predetermined threshold, denoting the motion event as involving a person.

6. The method of claim 1 , wherein determining that the frame includes the potential instance of the person comprises:

detecting the potential instance of the person;

assigning a confidence score to the potential instance of the person; and

in accordance with a determination that the confidence score meets one or more predetermined criteria, determining that the frame includes the potential instance of the person.

7. The method of claim 1 , further comprising:

determining whether the region includes one or more persons other than the potential person; and

in accordance with a determination that the region includes the one or more other persons, denoting the frame as containing a person.

8. The method of claim 1 , wherein the video feed comprises a plurality of frames, the method further comprising:

determining that one or more frames of the plurality of images includes a person;

obtaining a second plurality of frames, the second plurality of frames preceding the motion event;

for each frame in the second plurality of frames, analyzing the frame to determine whether the image includes the person;

in accordance with a determination that one or more frames of the second plurality of frames do not include the person, denoting the motion event as involving the person; and

in accordance with a determination that one or more frames of the second plurality of frames include the person, forgoing denoting the motion event as involving the person.

9. The method of claim 1 , further comprising:

in accordance with a determination that the frame includes multiple potential instances of a person, denoting a region encompassing each potential instance; and

determining whether the region includes one or more instances of a person by analyzing the region at the second resolution, greater than the first resolution.

10. The method of claim 1 , further comprising:

for each frame in the plurality of frames, analyzing the frame to determine whether the frame includes a particular object, the analyzing including:

determining whether the frame includes a potential instance of the particular object by analyzing the frame at a first resolution;

in accordance with a determination that the frame includes a potential instance, denoting a region around the potential instance, wherein the area of the region is less than the area of the frame;

determining whether the region includes an instance of the particular object by analyzing the region at a second resolution, greater than the first resolution; and

in accordance with a determination that the region includes an instance of the particular object, determining that the frame includes the particular object.

11. The method of claim 1 , further comprising determining whether the motion event involves a person by analyzing one or more relationships between frames including persons of the plurality of frames.

12. A computing system comprising:

one or more processors; and

memory coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

obtaining a video feed; and

for a frame of the video feed, analyzing the frame at a first resolution to determine whether the frame includes a potential instance of a person:

in accordance with the determination that the frame includes the potential instance, denoting a region around the potential instance;

analyzing the region at a second resolution, greater than the first resolution; and

in accordance with a determination that the region includes the instance of the person:

determining that the frame includes the person; and

storing an indication of the determination for use in subsequent alert notification processing.

13. The computing system of claim 12 , wherein the video feed includes a plurality of frames, the one or more programs further include instructions for assigning a confidence score to each frame of the plurality of frames.

14. The computing system of claim 13 , wherein the video feed includes a motion event, and the one or more programs further include instructions for, in accordance with a determination that the confidence score for at least one frame of the plurality of frames exceeds a predetermined threshold, denoting the motion event as involving a person.

15. The computing system of claim 12 , wherein the instructions for determining that the frame includes the potential instance of the person comprises instructions for:

detecting the potential instance of the person;

assigning a confidence score to the potential instance of the person; and

in accordance with a determination that the confidence score meets one or more predetermined criteria, determining that the frame includes the potential instance of the person.

16. The computing system of claim 12 , wherein the video feed comprises a plurality of frames, the one or more programs further include instructions for:

determining that one or more frames of the plurality of images includes a person;

obtaining a second plurality of frames, the second plurality of frames preceding the motion event;

for each frame in the second plurality of frames, analyzing the frame to determine whether the frame includes the person;

in accordance with a determination that one or more frames of the second plurality of frames do not include the person, denoting the motion event as involving the person; and

in accordance with a determination that one or more frames of the second plurality of images include the person, forgoing denoting the motion event as involving the person.

17. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing system, cause the system to:

obtain a video feed; and

for a frame of the video feed, analyze the frame at a first resolution to determine whether the frame includes a potential instance of a person:

in accordance with the determination that the frame includes the potential instance, denote a region around the potential instance;

analyze the region at a second resolution, greater than the first resolution; and

in accordance with a determination that the region includes the instance of the person:

determine that the frame includes the person; and

store an indication of the determination for use in subsequent alert notification processing.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the one or more programs comprising instructions, which when executed by the computing system, cause the system to assign a confidence score to each frame of the plurality of frames.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the determination that the frame includes the potential instance of the person comprises:

detecting the potential instance of the person;

assigning a confidence score to the potential instance of the person; and

in accordance with a determination that the confidence score meets one or more predetermined criteria, determining that frame includes the potential instance of the person.

20. The non-transitory computer-readable storage medium of claim 17 , wherein the video feed includes a plurality of frames, and the one or more programs comprising instructions, which when executed by the computing system, cause the system to:

determine that one or more frames of the plurality of frames include a person;

obtain a second plurality of frames, the second plurality of frames preceding the motion event;

for each frame in the second plurality of frames, analyze the frame to determine whether the image includes the person;

in accordance with a determination that one or more frames of the second plurality of frames do not include the person, denote the motion event as involving the person; and

in accordance with a determination that one or more frames of the second plurality of images include the person, forgo denoting the motion event as involving the person.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 14, 2020
From: CHAUDHRY, RIZWAN AHMED; DALAL, NAVNEET; BEN-MESHULAM, JONATHAN Z.; HEITZ, GEORGE ALBAN, III
To: GOOGLE INC.
Reel/Frame 054638/0029 →
CHANGE OF NAME Recorded Dec 14, 2020
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 054749/0191 →
Continuity (2)
Continuation 15207459 · Jul 11, 2016
Related Publication 20190325228A1 · Oct 24, 2019
Cited By (6)
US 12,269,371 US 12,418,947 US 12,542,874 US 12,664,818 US 12,682,013 US 12,713,478