IP Library Granted Patent US 12,658,014
Granted Patent B2
US 12,658,014 · App. 18/655,829 · Granted Jun 16, 2026

Systems and methods for identifying user-customized relevant individuals in an ambient image at a doorbell device

Inventors: Soumitri Kolavennu (Blaine, MN); Nathaniel Kraft (Minnetonka, MN)
Assignee: Resideo USA LLC
G08B13/1968G06F16/51G06F16/535G06F16/56G06F16/5854G06F16/587G06N5/04G06N20/00G06Q50/265G06V40/172G06V40/50H04N23/61
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,658,014
App. No.
18/655,829
Granted
Jun 16, 2026
Kind
B2
Abstract

Systems and methods for identifying user-customized relevant individuals in an ambient image at a doorbell device are provided. Such systems and methods can include receiving user input that includes image information, using the image information to compile a custom image database containing a plurality of images that depict such relevant individuals, and storing the custom image database in local memory of the doorbell device. Then, such systems and methods can include capturing an ambient image with a camera of the doorbell device, determining whether any person depicted in the ambient image matches any of the relevant individuals by comparing the ambient image to the plurality of images at the doorbell device, and generating an alert when any person depicted in the ambient image matches any of the relevant individuals.

Claims (44)

1 . A method comprising:

capturing, via a camera associated with a device, an image, the image corresponding to a region outside a location, the image comprising content corresponding to a representation of a person;

analyzing, by the device, the image, and determining characteristics of the person;

extracting, by the device, from the image, the determined characteristics;

accessing, by the device, from memory of the device, a collection of stored images, the collection of stored images being images previously captured by the camera and stored in the memory of the device, each stored image comprising information related to a known threat previously determined via analysis of the previously captured images within the collection, the known threat based on a location range of a residence of a depicted individual in a respectively stored image to the location, wherein the collection of stored images has a maximum number that corresponds to processing power of the device;

comparing, by the device, the determined characteristics extracted from the captured image with characteristics associated with images in the collection of stored images to identify a match with the known threat;

determining, by the device, based on analysis of the captured image and the comparison based on the collection of stored images, an event to the location; and

generating, by the device, an alert at the location based on a type of the alert, the alert indicating detection of the known threat.

2 . The method of claim 1 , further comprising:

communicating, from a cloud server to the device, image information, the image information corresponding to a type of activity for the device to detect via the camera, wherein the communication causes the device to compile and store a plurality of images based on the image information.

3 . The method of claim 2 , further comprising:

storing, in storage associated with the device, the plurality of images, wherein the collection of stored images further comprises the plurality of images stored in storage.

4 . The method of claim 1 , wherein the compilation of the plurality of images is based on analysis of a threat criteria.

5 . The method of claim 1 , wherein the type of event corresponds to a determination that the person is a safe person.

6 . The method of claim 1 , wherein the type of event corresponds to a determination that the person is an un-safe person.

7 . The method of claim 1 , wherein the analysis of the image is based on a facial recognition analysis of the image.

8 . The method of claim 1 , wherein the analysis of the image is based on an artificial intelligence (AI) vector mapping of the image.

9 . The method of claim 1 , wherein the captured image is an ambient image.

10 . A device comprising:

a processor configured to:

capturing, via an associated camera, an image, the image corresponding to a region outside a location, the image comprising content corresponding to a representation of a person;

analyze, the image, and determine characteristics of the person;

extract, from the image, the determined characteristics;

access, from memory of the device, a collection of stored images, the collection of stored images being images previously captured by the camera and stored in the memory of the device, each stored image comprising information related to a known threat previously determined via analysis of the previously captured images within the collection, the known threat based on a location range of a residence of a depicted individual in a respectively stored image to the location, wherein the collection of stored images has a maximum number that corresponds to processing power of the device;

compare the determined characteristics extracted from the captured image with characteristics associated with images in the collection of stored images to identify a match with the known threat;

determine, based on analysis of the captured image and the comparison based on the collection of stored images, an event to the location; and

generate, by the device, an alert at the location based on a type of the alert, the alert indicating detection of the known threat.

11 . The device of claim 10 , wherein the processor is further configured to:

communicate, from a cloud server to the device, image information, the image information corresponding to a type of activity for the device to detect via the camera, wherein the communication causes the device to compile and store a plurality of images based on the image information.

12 . The device of claim 11 , wherein the processor is further configured to:

store, in storage associated with the device, the plurality of images, wherein the collection of stored images further comprises the plurality of images stored in storage.

13 . The device of claim 10 , wherein the compilation of the plurality of images is based on analysis of a threat criteria.

14 . The device of claim 10 , wherein the type of event corresponds to a determination that the person is one of a safe person or un-safe person.

15 . The device of claim 10 , wherein the analysis of the image is based on a facial recognition analysis of the image.

16 . The device of claim 10 , wherein the analysis of the image is based on an artificial intelligence (AI) vector mapping of the image.

17 . The device of claim 10 , wherein the captured image is an ambient image.

18 . A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions, that when executed by a device, perform a method comprising:

capturing, via a camera associated with the device, an image, the image corresponding to a region outside a location, the image comprising content corresponding to a representation of a person;

analyzing, by the device, the image, and determining characteristics of the person;

extracting, by the device, from the image, the determined characteristics;

accessing, by the device, from memory of the device, a collection of stored images, the collection of stored images being images previously captured by the camera and stored in the memory of the device, each stored image comprising information related to a known threat previously determined via analysis of the previously captured images within the collection, the known threat based on a location range of a residence of a depicted individual in a respectively stored image to the location, wherein the collection of stored images has a maximum number that corresponds to processing power of the device;

comparing, by the device, the determined characteristics extracted from the captured image with characteristics associated with images in the collection of stored images to identify a match with the known threat;

determining, by the device, based on analysis of the captured image and the comparison based on the collection of stored images, an event to the location; and

generating, by the device, an alert at the location based on a type of the alert, the alert indicating detection of the known threat.

Assignments (2)
CHANGE OF NAME Recorded Jul 4, 2025
From: ADEMCO INC.
To: RESIDEO LLC
Reel/Frame 071814/0718 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 6, 2024
From: KOLAVENNU, SOUMITRI; KRAFT, NATHANIEL
To: ADEMCO INC.
Reel/Frame 068508/0116 →
Continuity (2)
Continuation 16860157 · Apr 28, 2020
Related Publication 20240290186A1 · Aug 29, 2024
References Cited (82)
US 7634662B2 · Monroe · 2009 [cited by applicant]
US 10140718B2 · Chen et al. · 2018 [cited by applicant]
US 10475311B2 · Siminoff · 2019 [cited by applicant]
US 10489887B2 · El-Khamy et al. · 2019 [cited by applicant]
US 10726274B1 · Hasegawa et al. · 2020 [cited by applicant]
US 10861265B1 · Merkley et al. · 2020 [cited by applicant]
US 10997703B1 · Khalatian · 2021 [cited by examiner]
US 11004113B1 · Sorensen · 2021 [cited by examiner]
US 11304123B1 · Noonan · 2022 [cited by applicant]
US 11978328B2 · Kolavennu et al. · 2024 [cited by applicant]
US 20030018531A1 · Mahaffy et al. · 2003 [cited by applicant]
US 20040005086A1 · Wolff et al. · 2004 [cited by applicant]
US 20060212341A1 · Powers · 2006 [cited by applicant]
US 20090002157A1 · Donovan et al. · 2009 [cited by applicant]
US 20090176544A1 · Mertens · 2009 [cited by applicant]
US 20090181640A1 · Jones · 2009 [cited by applicant]
US 20090208052A1 · Kaplan · 2009 [cited by applicant]
US 20090222388A1 · Hua et al. · 2009 [cited by applicant]
US 20090319361A1 · Conrady · 2009 [cited by applicant]
US 20130127980A1 · Haddick · 2013 [cited by examiner]
US 20130202274A1 · Chan · 2013 [cited by examiner]
US 20140032538A1 · Arngren et al. · 2014 [cited by applicant]
US 20150363500A1 · Bhamidipati et al. · 2015 [cited by applicant]
US 20160065861A1 · Steinberg et al. · 2016 [cited by applicant]
US 20160132720A1 · Klare et al. · 2016 [cited by applicant]
US 20160196467A1 · Xia · 2016 [cited by applicant]
US 20170083790A1 · Risinger et al. · 2017 [cited by applicant]
US 20170085844A1 · Scalisi et al. · 2017 [cited by applicant]
US 20170092109A1 · Trundle et al. · 2017 [cited by applicant]
US 20180059660A1 · Heatzig et al. · 2018 [cited by applicant]
US 20180121571A1 · Tiwari et al. · 2018 [cited by applicant]
US 20180268674A1 · Siminoff · 2018 [cited by applicant]
US 20180285648A1 · Pan et al. · 2018 [cited by applicant]
US 20180307903A1 · Siminoff · 2018 [cited by applicant]
US 20190035242A1 · Vazirani · 2019 [cited by applicant]
US 20190130278A1 · Karras et al. · 2019 [cited by applicant]
US 20190130583A1 · Chen et al. · 2019 [cited by applicant]
US 20190188980A1 · Viswanathan et al. · 2019 [cited by applicant]
US 20190197848A1 · Bradley et al. · 2019 [cited by applicant]
US 20190304274A1 · Britton · 2019 [cited by examiner]
US 20190318283A1 · Kelly · 2019 [cited by examiner]
US 20190327448A1 · Fu · 2019 [cited by examiner]
US 20190373186A1 · Ortiz Egea et al. · 2019 [cited by applicant]
US 20190376808A1 · Shikanai · 2019 [cited by examiner]
US 20190377961A1 · Inai · 2019 [cited by examiner]
US 20200019921A1 · Buibas et al. · 2020 [cited by applicant]
US 20200020221A1 · Cutler et al. · 2020 [cited by applicant]
US 20200135182A1 · Kahlon et al. · 2020 [cited by applicant]
US 20200175303A1 · Bhat · 2020 [cited by examiner]
US 20200242336A1 · Boic · 2020 [cited by applicant]
US 20200301936A1 · Kahlon · 2020 [cited by applicant]
US 20200394804A1 · Barton et al. · 2020 [cited by applicant]
US 20210152880A1 · Marten et al. · 2021 [cited by applicant]
US 20210192186A1 · Kim · 2021 [cited by examiner]
US 20210209349A1 · Mehl et al. · 2021 [cited by applicant]
US 20230186626A1 · Latapie · 2023 [cited by examiner]
US 20230196898A1 · Snyder · 2023 [cited by examiner]
US 20240153275A1 · Jagadeesan · 2024 [cited by examiner]
US 20240265755A1 · Carter · 2024 [cited by examiner]
CN 108921001A · 2018 [cited by applicant]
CN 110414305A · 2019 [cited by applicant]
WO 2019202587A1 · 2019 [cited by applicant]
Chan et al., “A Fuzzy Qualitative Approach to Human Motion Recognition,” IEEE International Conference on Fuzzy Systems, pp. 1242-1249, Jun. 1, 2008. [cited by applicant]
Eng et al., “Dews: A Live Visual Surveillance System for Early Drowning Detection at Pool” IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, No. 2, pp. 197-208, Feb. 2008. [cited by applicant]
Europe IBM Intelligent Video Analytics V3.0, 5725-H94, IBM Intelligent Video Analytics V3.0, IBM Europe Sales Manual, Revised Apr. 23, 2019, https://ww-01.ibm.com/common/ssi/ShowDoc.wss? docURL=/common/ssi/rep_sm/4/877/… [cited by applicant]
Jalal et al., “A Depth Video-Based Human Detection and Activity Recognition Using Multi-Features and Embedded Hidden Markov Models for Health Care Monitoring Systems,” International Journal of Interactive Multimedia and… [cited by applicant]
Jia et al., “Super-Resolution with Deep Adaptive Image Resampling,” arXiv preprint: 1712.06463, pp. 1-10, Dec. 2017. [cited by applicant]
Lai et al., “Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 624-632, 2017. [cited by applicant]
Liu et al., “A Video Drowning Detection Device Based on Underwater Computer Vision” The Institution of Engineering and Technology Image Press, vol. 17, pp. 1905-1918, Feb. 5, 2023. [cited by applicant]
Nadeem et al., “Automatic Human Posture Estimation for Sport Activity Recognition with Robust Body Parts Detection and Entropy Markov Model,” Multimedia Tools and Applications, vol. 80, pp. 21465-21498, Jun. 2021. [cited by applicant]
“New EnhanceNet-PAT AI Turns Low-Resolution Images into High-Res”, https://edgy.app/new-ai-system-to-turn-low-resolution-images-to-high-resolution, 4 pages, Oct. 30, 2017. [cited by applicant]
Paul et al., “Human Detection in Surveillance Videos and its Applications—A Review,” Adv. Signal Process, vol. 1, pp. 1-6, Nov. 22, 2013. [cited by applicant]
Sajjadi et al., “EnhanceNet: Single Image-Super Resolution Through Automated Texture Synthesis”, Proceedings of the IEEE International Conference on Computer Vision, pp. 4491-4500, 2017. [cited by applicant]
“Taking Motion Analytics to a New Level With AI, AI Motion Analytics Software Solutions,” Artificial Intelligence, https://www.osplabs.com/ai-motion-analytics/ 4 pages, Jan. 9, 2020. [cited by applicant]
Thomas, “Deep Learning Based Super Resolution, Without Using a GAN”, Towards Data Science, https://towardsdatascience.com/deep-learning-based-super-resolution-without-using-gan, 50 pages, Feb. 24, 2019. [cited by applicant]
Wang et al., “Resolution-Aware Network for Image Super-Resolution”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 29, No. 5, pp. 1259-1269, May 2018. [cited by applicant]
Wei et al., “Unsupervised Recurrent Hyperspectral Imagery Super-Resolution Using Pixel-Aware Refinement”, IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-15, Dec. 11, 2020. [cited by applicant]
Yan et al., “Multi-Instance Deep Learning: Discover Discriminative Local Anatomies for Bodypart Recognition,” IEEE Transactions on Medical Imaging, vol. 35, No. 5, pp. 1332-1343, May 2016. [cited by applicant]
Yang et al., “Depth Map Super-Resolution Using Stereo-Vision-Assisted Model” Neurocomputing, vol. '39, pp. 1396-1406, Aug. 2014. [cited by applicant]
Yang et al., “LCSCNet: Linear Compressing-based Skip-Connecting Network for Image Super-Resolution” IEEE Transactions on Image Processing, vol. 29, vol. 39, pp. 1450-1464, 2019. [cited by applicant]
Zhao et al., “Stereo- and Neural Network-Based Pedestrian Detection,” IEEE Transactions on Intelligent Transportation Systems, vol. 1, No. 3, pp. 148-154, Sep. 2000. [cited by applicant]
International Search Report in International Application No. PCT/US2021/029380 dated Jul. 16, 2021. [cited by applicant]