IP Library Granted Patent US 12,664,666
Granted Patent B2
US 12,664,666 · App. 18/421,795 · Granted Jun 23, 2026

Activity zone for camera video

Inventors: Sara Bolouki (Mountain View, CA); Yah Yah Cason (Las Vegas, NV); Jonghoon Jin (Cupertino, CA); Nils Ängquist (Los Altos, CA); Hendrik Dahlkamp (Redwood City, CA); Patrick L. Coffman (San Francisco, CA)
Assignee: Apple Inc.
G06T7/20G06N3/08G06T2207/10016G06T2207/20081G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,664,666
App. No.
18/421,795
Filed
Jan 24, 2024
Granted
Jun 23, 2026
Kind
B2
Art Unit
2665
USPC
382/107
Abstract

Methods and systems are provided for establishing an activity zone within video captured by a camera. A user device may establish an activity zone or an exclusion zone within video representing a physical view captured by the camera. An automatic monitoring system may analyze the video of the camera with respect to the activity zone or exclusion zone and trigger notifications that are associated with a zone so as to limit notifications transmitted to the user device to those notifications that are relevant to the user. With activity or exclusion zones video from a camera may be automatically monitored to reduce a volume of notifications transmitted to the user device.

Claims (82)

1 . A method, comprising:

receiving, by a resident device from a camera separate from the resident device, a communication indicating that motion was detected;

in response to receiving the communication indicating that motion was detected, requesting a portion of video captured by the camera corresponding to the motion that was detected, wherein the portion of video captures a physical view;

after requesting the portion of video captured by the camera corresponding to the motion, receiving, by a processor of a resident device, the portion of the video captured by the camera corresponding to the motion;

receiving, by the processor, information that identifies an exclusion zone within the portion of video captured by the camera corresponding to the motion, wherein the exclusion zone is a predetermined area of the physical view where motion is ignored by the processor;

detecting, by the processor, a portion of an object within the physical view;

determining, by the processor, that the motion corresponds to the object within the physical view;

determining, by the processor, whether the object within the physical view is an object of interest; and

in response to determining that the object within the physical view is the object of interest:

determining whether the object of interest is entirely within the exclusion zone; and

in response to determining that the object of interest is entirely within the exclusion zone, suppressing a notification, to a user device, about the detected motion within the physical view, wherein the user device is separate from the camera and the resident device.

2 . The method of claim 1 , further comprising:

after receiving the video captured by the camera corresponding to the motion, receiving, from the user device, a request to establish the exclusion zone; and

transmitting, to the user device and in response to the request, a frame of video of the physical view.

3 . The method of claim 1 , further comprising:

generating a log that stores an indication of the suppression of the notification, wherein the log is stored in memory.

4 . The method of claim 1 , wherein the portion of the object is a first portion of the object, and wherein the method further comprises:

receiving, by the processor, information that identifies an activity zone within the video stream that comprises the subset of the physical view; and

in response to determining that the object is the object of interest and that the object of interest is not entirely within the exclusion zone:

determining that a second portion of the object of interest is within the activity zone;

storing a subset of the video stream that corresponds to the object of interest; and

transmitting, to the user device, a second notification that identifies the subset of the video stream.

5 . The method of claim 4 , wherein the subset of the video stream includes two or more frames of the video stream that correspond to the motion of the object of interest.

6 . The method of claim 4 , further comprising:

receiving, from the user device, a request for the subset of the video stream; and

transmitting, in response to receiving the request, the subset of the video stream to the user device.

7 . The method of claim 4 , wherein the notification comprises an identification of the object of interest.

8 . A device, comprising:

a memory comprising computer-executable instructions; and

one or more processors in communication with the memory and configured to access the memory and execute the computer-executable instructions to perform operations including:

receiving, by a resident device from a camera separate from the resident device, a communication indicating that motion was detected;

in response to receiving the communication indicating that motion was detected, requesting a portion of video captured by the camera corresponding to the motion that was detected, wherein the portion of video captures a physical view;

after requesting the portion of video captured by the camera corresponding to the motion, receiving, by a processor of a resident device, the portion of the video captured by the camera corresponding to the motion;

receiving, by the processor, information that identifies an exclusion zone within the portion of video captured by the camera corresponding to the motion, wherein the exclusion zone is a predetermined area of the physical view where motion is ignored by the processor;

detecting, by the processor, a portion of an object within the physical view; determining, by the processor, that the motion corresponds to the object within the physical view;

determining, by the processor, whether the object within the physical view is an object of interest; and

in response to determining that the object within the physical view is the object of interest:

determining whether the object of interest is entirely within the exclusion zone; and

in response to determining that the object of interest is entirely within the exclusion zone suppressing a notification, to a user device, about the detected motion within the physical view, wherein the user device is separate from the camera and the resident device.

9 . The device of claim 8 , wherein the one or more processors configured to further perform operations including:

after receiving the video captured by the camera corresponding to the motion, receiving, from the user device, a request to establish the exclusion zone; and

transmitting, to the user device and in response to the request, a frame of video of the physical view.

10 . The device of claim 8 , wherein the one or more processors configured to further perform operations including:

generating a log that includes an indication of the suppression of the notification, wherein the log is stored in memory.

11 . The device of claim 8 , wherein the portion of the object is a first portion of the object, and wherein the one or more processors configured to further perform operations including:

receiving, by the one or more processors, information that identifies an activity zone within the video stream that comprises the subset of the physical view; and

in response to determining that the object is the object of interest and that the object of interest is not entirely within the exclusion zone:

determining that a second portion of the object is within the activity zone;

storing a subset of the video stream that corresponds to the object of interest; and

transmitting, to the user device, a second notification that identifies the subset of the video stream.

12 . The device of claim 11 , wherein the subset of the video stream includes two or more frames of the video stream that correspond to the motion of the object of interest.

13 . The device of claim 11 , wherein the one or more processors configured to further perform operations including:

receiving, from the user device, a request for the subset of the video stream; and

transmitting, to the user device and in response to receiving the request, the subset of the video stream to the user device.

14 . The device of claim 11 , wherein the notification includes an identification of the object of interest.

15 . A computer-readable storage medium comprising computer-executable instructions that, when executed by one or more processors, causes the one or more processors to perform operations including:

receiving, by a resident device from a camera separate from the resident device, a communication indicating that motion was detected;

in response to receiving the communication indicating that motion was detected, requesting a portion of video captured by the camera corresponding to the motion that was detected, wherein the portion of video captures a physical view;

after requesting the portion of video captured by the camera corresponding to the motion, receiving, by a processor of a resident device, the portion of the video captured by the camera corresponding to the motion;

receiving, by the processor, information that identifies an exclusion zone within the portion of video captured by the camera corresponding to the motion, wherein the exclusion zone is a predetermined area of the physical view where motion is ignored by the processor;

detecting, by the processor, a portion of an object within the physical view; determining, by the processor, that the motion corresponds to the object within the physical view;

determining, by the processor, whether the object within the physical view is an object of interest; and

in response to determining that the object within the physical view is the object of interest:

determining whether the object of interest is entirely within the exclusion zone; and

in response to determining that the object of interest is entirely within the exclusion zone suppressing a notification, to a user device, about the detected motion within the physical view, wherein the user device is separate from the camera and the resident device.

16 . The computer-readable storage medium of claim 15 , wherein the instructions, when executed, cause the one or more processors to perform operations including:

after receiving the video captured by the camera corresponding to the motion, receiving, from the user device, a request to establish the exclusion zone; and

transmitting, to the user device and in response to the request, a frame of video of the physical view.

17 . The computer-readable storage medium of claim 15 , wherein the instructions, when executed, cause the one or more processors to perform operations including:

generating a log that includes an indication of the suppression of the notification, wherein the log is stored in memory.

18 . The computer-readable storage medium of claim 15 , wherein the portion of the object is a first portion of the object, and wherein the instructions, when executed, cause the one or more processors to perform operations including:

receiving, by the one or more processors, information that identifies an activity zone within the video stream that comprises the subset of the physical view; and

in response to determining that the object is the object of interest and that the object of interest is not entirely within the exclusion zone:

determining that a second portion of the object of interest is within the activity zone;

storing a subset of the video stream that corresponds to the object of interest; and

transmitting, to the user device, a second notification that identifies the subset of the video stream.

19 . The computer-readable storage medium of claim 18 , wherein the instructions, when executed, cause the one or more processors to perform operations including:

receiving, from the user device, a request for the subset of the video stream; and

transmitting, in response to receiving the request, the subset of the video stream to the user device.

20 . The computer-readable storage medium of claim 18 , wherein the notification includes an identification of the object of interest.

21 . The method of claim 1 , further comprising:

in response to determining that the object within the physical view is not the object of interest, forgoing transmission of, to the user device, a notification about the detected motion within the physical view.

Continuity (3)
Continuation 17199166 · Mar 11, 2021
Provisional Application 63034262 · Jun 3, 2020
Related Publication 20240161308A1 · May 16, 2024
References Cited (118)
US 8063929B2 · Kurtz · 2011 [cited by examiner]
US 8154578B2 · Kurtz · 2012 [cited by examiner]
US 8189577B2 · Vaswani et al. · 2012 [cited by applicant]
US 8253770B2 · Kurtz · 2012 [cited by examiner]
US 8325033B2 · Feldstein · 2012 [cited by examiner]
US 8365010B2 · Olson et al. · 2013 [cited by applicant]
US 8711217B2 · Venetianer et al. · 2014 [cited by applicant]
US 8804997B2 · Gagvani et al. · 2014 [cited by applicant]
US 9197861B2 · Saptharishi et al. · 2015 [cited by applicant]
US 9392248B2 · Wheeler · 2016 [cited by examiner]
US 9779307B2 · Laska et al. · 2017 [cited by applicant]
US 9830503B1 · Martin et al. · 2017 [cited by applicant]
US 9883027B2 · Dave et al. · 2018 [cited by applicant]
US 10013153B1 · Freeman et al. · 2018 [cited by applicant]
US 10049281B2 · Verano et al. · 2018 [cited by applicant]
US 10217068B1 · Davis et al. · 2019 [cited by applicant]
US 10242266B2 · Jones · 2019 [cited by examiner]
US 10255683B1 · Gopal · 2019 [cited by examiner]
US 10425475B2 · Cahana et al. · 2019 [cited by applicant]
US 10586433B2 · Stewart · 2020 [cited by examiner]
US 10594987B1 · Nixon · 2020 [cited by examiner]
US 10762646B2 · Modestine · 2020 [cited by examiner]
US 10861168B2 · Yu · 2020 [cited by examiner]
US 10929798B1 · Kinney et al. · 2021 [cited by applicant]
US 10992492B2 · Beach · 2021 [cited by examiner]
US 11184589B2 · Scalisi · 2021 [cited by examiner]
US 11232688B1 · Lemberger · 2022 [cited by examiner]
US 11259076B2 · Matsuoka · 2022 [cited by examiner]
US 11295140B2 · Wang · 2022 [cited by examiner]
US 11328153B1 · Mumm et al. · 2022 [cited by applicant]
US 11356352B2 · Rauenbuehler et al. · 2022 [cited by applicant]
US 11514717B2 · Dahlkamp et al. · 2022 [cited by applicant]
US 11710348B2 · Dahlkamp et al. · 2023 [cited by applicant]
US 11908143B2 · Bolouki et al. · 2024 [cited by applicant]
US 11921831B2 · Khadloya et al. · 2024 [cited by applicant]
US 12014611B1 · Zareba · 2024 [cited by examiner]
US 12021724B2 · Rauenbuehler et al. · 2024 [cited by applicant]
US 12131583B2 · Dahlkamp et al. · 2024 [cited by applicant]
US 20030023690A1 · Lohtia · 2003 [cited by applicant]
US 20040136574A1 · Kozakaya et al. · 2004 [cited by applicant]
US 20060204058A1 · Kim et al. · 2006 [cited by applicant]
US 20070014244A1 · Srinivasan et al. · 2007 [cited by applicant]
US 20130086607A1 · Tom et al. · 2013 [cited by applicant]
US 20130182918A1 · Kovtun et al. · 2013 [cited by applicant]
US 20140050372A1 · Qi et al. · 2014 [cited by applicant]
US 20150074259A1 · Ansari et al. · 2015 [cited by applicant]
US 20150131648A1 · Shuman et al. · 2015 [cited by applicant]
US 20160005280A1 · Laska et al. · 2016 [cited by applicant]
US 20160043827A1 · Filson et al. · 2016 [cited by applicant]
US 20160092724A1 · Jeong · 2016 [cited by applicant]
US 20170257226A1 · Bi · 2017 [cited by applicant]
US 20170262472A1 · Goldenberg · 2017 [cited by applicant]
US 20170344703A1 · Ansari et al. · 2017 [cited by applicant]
US 20180151039A1 · Lemberger · 2018 [cited by examiner]
US 20180232592A1 · Stewart et al. · 2018 [cited by applicant]
US 20180268202A1 · Yu et al. · 2018 [cited by applicant]
US 20180276465A1 · Lee et al. · 2018 [cited by applicant]
US 20180285628A1 · Son et al. · 2018 [cited by applicant]
US 20180338120A1 · Lemberger · 2018 [cited by examiner]
US 20180350213A1 · Bart et al. · 2018 [cited by applicant]
US 20180373924A1 · Yoo et al. · 2018 [cited by applicant]
US 20190063140A1 · Trundle et al. · 2019 [cited by applicant]
US 20190087647A1 · Du · 2019 [cited by applicant]
US 20190139381A1 · Harpole · 2019 [cited by applicant]
US 20190155659A1 · Rodriguez Bravo et al. · 2019 [cited by applicant]
US 20190205620A1 · Yi et al. · 2019 [cited by applicant]
US 20190236342A1 · Madden et al. · 2019 [cited by applicant]
US 20200082157A1 · Susskind et al. · 2020 [cited by applicant]
US 20200388139A1 · Saha et al. · 2020 [cited by applicant]
US 20210006835A1 · Slattery et al. · 2021 [cited by applicant]
US 20210134100A1 · Larovere et al. · 2021 [cited by applicant]
US 20210383100A1 · Dahlkamp et al. · 2021 [cited by applicant]
US 20210383554A1 · Bolouki et al. · 2021 [cited by applicant]
CN 109788188B · 2020 [cited by examiner]
WO WO2008153822A2 · 2008 [cited by examiner]
WO WO2018175328A1 · 2018 [cited by examiner]
WO 2019222725A1 · 2019 [cited by applicant]
US 11,670,113 B2, 06/2023, Dahlkamp et al. (withdrawn) [cited by applicant]
Office Action issued in European Application No. EP21734694.9, dated Nov. 4, 2024 in 10 pages. [cited by applicant]
Sasidharan, et al., entitled IoT Based Cryptolock With Face Recognition Using AES-256 Bit Algorithm; Studies in Indian Place Names(UGC Care Journal), vol. 40, dated Nov. 4, 2024, pp. 1361-1366. [cited by applicant]
Non-Final Office Action issued in U.S. Appl. No. 17/469,630, dated Jan. 29, 2025 in 26 pages. [cited by applicant]
Non-Final Office Action issued in U.S. Appl. No. 17/469,630, dated Jul. 30, 2024 in 26 pages. [cited by applicant]
Supplemental Notice of Allowability in U.S. Appl. No. 17/949,880, dated Aug. 23, 2024 in 2 pages. [cited by applicant]
Notice of Allowance issued in U.S. Appl. No. 17/716,332, dated Feb. 22, 2024 in 8 pages. [cited by applicant]
Non-Final Office Action issued in U.S. Appl. No. 17/949,880, dated Feb. 5, 2024 in 41 pages. [cited by applicant]
Wang, et al., “Masked Face Recognition Dataset and Application,” Available online At: https://arxiv.org/pdf/2003.09093.pdf, Cornell University Library; dated Mar. 23, 2020 in 3 pages. [cited by applicant]
Ejaz, et al., “Masked Face Recognition Using Convolutional Neural Network,” 2019 International Conference on Sustainable Technologies for Industry 4.0 (STI), 978-1-7281-6099-3/19/$31.00 © 2019 IEEE, dated Dec. 25, 2019 … [cited by applicant]
Extended European Search Report issued in European Application No. EP21196868.0, dated Feb. 3, 2022 in 7 pages. [cited by applicant]
Office Action issued in European Application No. EP21196868.0, dated Dec. 6, 2023 in 5 pages. [cited by applicant]
First Examination Report issued in India Application No. IN2021140414, dated May 24, 2022 in 7 pages. [cited by applicant]
Non-Final Office Action issued in U.S. Appl. No. 17/469,630, dated Jan. 2, 2024 in 24 pages. [cited by applicant]
Final Office Action issued in U.S. Appl. No. 17/469,630, dated Apr. 11, 2024 in 27 pages. [cited by applicant]
Ejaz et al., “Masked Face Recognition Using Convolutional Neural Network”, 2019 International Conference on Sustainable Technologies for Industry 4.0 (STI), Dec. 25, 2019 in 6 pages. [cited by applicant]
First Examination Report issued in India Application No. IN202114041420, dated May 24, 2022 in 7 pages. [cited by applicant]
Wang et al., “Masked Face Recognition Dataset and Application”, Available online At: https://arxiv.org/pdf/2003.09093.pdf, Cornell University Library, Mar. 23, 2020, pp. 1-3. [cited by applicant]
Notice of Allowance issued in U.S. Appl. No. 17/949,880, dated Jun. 27, 2024 in 7 pages. [cited by applicant]
Article entitled, “Using Your Video Services”, Telus, Available Online at: https://www.telus.com/en/bc/support/article/using-your-video-services, dated Mar. 27, 2020, pp. 1-13 (as-of record in parent application). [cited by applicant]
Notice of Allowance issued in U.S. Appl. No. 17/032,619, dated Dec. 21, 2021 in 9 pages (as-of record in parent application). [cited by applicant]
Notice of Allowance issued in U.S. Appl. No. 17/032,619, dated Feb. 9, 2022 in 9 pages (as-of record in parent application). [cited by applicant]
Non-Final Office Action issued in U.S. Appl. No. 17/033,423, dated Mar. 22, 2022 in 62 pages (as-of record in parent application). [cited by applicant]
Notice of Allowance issued in U.S. Appl. No. 17/033,423, dated Aug. 24, 2022 in 7 pages (as-of record in parent application). [cited by applicant]
Final Office Action issued in U.S. Appl. No. 17/199,166, dated Jun. 20, 2023 in 18 pages (as-of record in parent application). [cited by applicant]
Non-Final Office Action issued in U.S. Appl. No. 17/199,166, dated Jan. 27, 2023 in 24 pages (as-of record in parent application). [cited by applicant]
Notice of Allowance issued in U.S. Appl. No. 17/199,166, dated Oct. 10, 2023 in 8 pages (as-of record in parent application). [cited by applicant]
Final Office Action issued in U.S. Appl. No. 17/716,332, dated Nov. 13, 2023 in 10 pages (as-of record in parent application). [cited by applicant]
Non-Final Office Action issued in U.S. Appl. No. 17/716,332, dated May 10, 2023 in 14 pages (as-of record in parent application). [cited by applicant]
Corrected Notice of Allowability issued in U.S. Appl. No. 17/949,876, dated Apr. 24, 2023 in 2 pages (as-of record in parent application). [cited by applicant]
Corrected Notice of Allowability issued in U.S. Appl. No. 17/949,876, dated Jun. 12, 2023 in 2 pages (as-of record in parent application). [cited by applicant]
Non-Final Office Action issued in U.S. Appl. No. 17/949,876, dated Jan. 5, 2023 in 23 pages (as-of record in parent application). [cited by applicant]
Notice of Allowance issued in U.S. Appl. No. 17/949,876, dated Jan. 25, 2023 in 7 pages (as-of record in parent application). [cited by applicant]
Final Office Action issued in U.S. Appl. No. 17/949,880, dated Jul. 24, 2023 in 50 pages (as-of record in parent application). [cited by applicant]
Non-Final Office Action issued in U.S. Appl. No. 17/949,880, dated Jan. 5, 2023 in 45 pages (as-of record in parent application). [cited by applicant]
Pandit et al., “Intelligent Security Lock”, International Conference on Trends in Electronics and Informatics ICEI 2017, IEEE, dated May 11-12, 2017, pp. 713-716 (as-of record in parent application). [cited by applicant]
Pawar et al., “Smart Home Security using IoT and Face Recognition”, 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA), dated Aug. 16-18, 2018, pp. 1-6 (as-of record in pare… [cited by applicant]
International Preliminary Report on Patentability issued in PCT Application No. PCT/US2021/034748, dated Dec. 15, 2022 in 7 pages (as-of record in parent application). [cited by applicant]
International Search Report and Written Opinion issued in PCT Application No. PCT/US2021/034748, dated Sep. 8, 2021 in 10 pages (as-of record in parent application). [cited by applicant]
International Preliminary Report on Patentability issued in PCT Application No. PCT/US2021/035005, dated Dec. 15, 2022 in 8 pages (as-of record in parent application). [cited by applicant]
International Search Report and Written Opinion issued in PCT Application No. PCT/US2021/035005, dated Sep. 21, 2021 in 11 pages (as-of record in parent application). [cited by applicant]