IP Library Granted Patent US 11,019,272
Granted Patent B2
US 11,019,272 · App. 16/781,746 · Granted May 25, 2021

Automatic dynamic range control for audio/video recording and communication devices

Inventor: Changsoo Jeong (Rancho Palos Verdes, CA)
Assignee: Amazon Technologies, Inc.
H04N5/2351H04N5/2352H04N5/2353H04N5/2355H04N5/23219H04N5/243H04N7/186H04N5/144
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,019,272
App. No.
16/781,746
Granted
May 25, 2021
Kind
B2
Abstract

Automatic exposure control for audio/video (A/V) recording and communication devices in accordance with various embodiments of the present disclosure are provided. In one embodiment, an A/V recording and communication device is provided, comprising: a camera configured to capture image data of a foreground object within a field of view of the camera; a communication module; and a processing module comprising: a processor; and a camera application that configures the processor to: detect motion within the field of view that includes an active image region; capture image data of the field of view; detect the foreground object in the active image region and determine an external lighting level associated with the foreground object; determine at least one exposure control gain setting based on the determined external lighting level; and apply the at least one exposure control gain setting to configure the camera to capture image data focused on the foreground object.

Claims (46)

1. An audio/video recording and communication device (A/V device), comprising:

a camera having a field of view;

a processor operatively connected to the camera; and

memory storing a camera application comprising instructions that, when executed by the processor, cause the A/V device to:

detect motion within the field of view of the camera, wherein the field of view includes a multi-exposure image condition associated with image data in the field of view;

capture the image data representing images of the field of view using the camera;

detect a foreground object in an active image region of the field of view and determine an external lighting level associated with the foreground object in the active image region;

determine at least one dynamic resolution (DR) setting based on the external lighting level associated with the foreground object, wherein the at least one DR setting includes a wide dynamic range gain setting based on an average luminance level and a saturated pixel count; and

apply the at least one DR setting to the camera.

2. The A/V device of claim 1 , wherein the camera application comprises further instructions that, when executed by the processor, further cause the A/V device to track the foreground object within the field of view using the camera.

3. The A/V device of claim 1 , wherein the camera application comprises further instructions that, when executed by the processor, further cause the A/V device to determine the external lighting level associated with the foreground object using the image data captured using the camera.

4. The A/V device of claim 1 , wherein the average luminance level and the saturated pixel count correspond to a number of saturated pixels associated with a reference frame of the captured image data.

5. The A/V device of claim 1 , wherein the field of view including the multi-exposure image condition comprises:

a first portion of the field of view having at least one of a first exposure range, first brightness level, first lighting condition; and

a second portion of the field of view having at least one of a second exposure range different from the first exposure range, a second brightness level different from the first brightness level, a second lighting condition different from the first lighting condition.

6. The A/V device of claim 1 , wherein the field of view including the multi-exposure image condition comprises:

a first portion of the field of view includes an object in motion; and

a second portion of the field of view that does not include the object in motion.

7. An audio/video recording and communication device (A/V device), comprising:

a camera having a field of view;

a processor operatively connected to the camera; and

memory storing a camera application comprising instructions that, when executed by the processor, cause the A/V device to:

detect motion within the field of view of the camera;

capture test image data representing images of the field of view using the camera;

detect a multi-exposure image condition associated with the test image data captured using the camera; and

apply at least one dynamic resolution (DR) setting to the camera, wherein the at least one DR setting includes a wide dynamic range (WDR) gain setting based on an average luminance level and a saturated pixel count.

8. The A/V device of claim 7 , wherein the camera application further comprises instructions that, when executed by the processor, further cause the A/V device to select a test frame from the test image data and detect the multi-exposure image condition by determining a pixel distribution value associated with the test frame.

9. The A/V device of claim 8 , wherein the camera application further comprises instructions that, when executed by the processor, further cause the A/V device to apply the at least one DR setting to the camera to correct for the multi-exposure image condition if the pixel distribution value is less than a predetermined pixel distribution threshold.

10. The A/V device of claim 7 , wherein the applying the at least one DR setting to the camera configures the camera to capture image data having a reference frame with a single exposure range.

11. The A/V device of claim 10 , wherein the average luminance level and a saturated pixel count correspond to a number of saturated pixels associated with the reference frame.

12. The A/V device of claim 11 , wherein the camera application further comprises instructions that, when executed by the processor, further cause the A/V device to determine the WDR gain setting when the average luminance level is less than a predetermined luminance threshold and the saturated pixel count is more than a predetermined saturated pixels threshold.

13. The A/V device of claim 12 , wherein the camera application further comprises instructions that, when executed by the processor, further cause the A/V device to apply the WDR gain setting to configure the camera to capture WDR image data.

14. The A/V device of claim 13 , wherein the camera application further comprises instructions that, when executed by the processor, further cause the A/V device to apply a temporal noise removal filter if the WDR image data has a background noise level above a predetermined noise threshold.

15. The A/V device of claim 14 , wherein the camera application further comprises instructions that, when executed by the processor, further cause the A/V device to reduce the temporal noise filter gain strength if the WDR image data has a motion artifact level above a predetermined artifact threshold.

16. The A/V device of claim 14 , wherein the camera application further comprises instructions that, when executed by the processor, further cause the A/V device to increase the temporal noise filter gain strength if the WDR image data has a motion artifact level below a predetermined artifact threshold.

17. A method for an audio/video recording and communication device (A/V device) comprising a camera, the method comprising:

detecting motion within a field of view of the camera wherein the field of view includes a multi-exposure image condition associated with image data in the field of view that includes a first portion having a first external lighting level and a second portion having a second external lighting level;

capturing test image data representing images of the field of view using the camera;

detecting the multi-exposure image condition associated with the image data captured using the camera; and

applying at least one dynamic resolution (DR) setting to the camera, wherein the at least one DR setting includes a wide dynamic range (WDR) gain setting based on an average luminance level and a saturated pixel count.

18. The method of claim 17 , further comprising, after applying the at least one DR setting to the camera, capturing image data representing a first frame with a first exposure range associated with the first external lighting level and a second frame with a second exposure range associated with the second external lighting level, wherein the second exposure range does not overlap with the first exposure range.

19. The method of claim 17 , further comprising combining the first frame with the first exposure range and the second frame with the second exposure range to generate a single high dynamic range (HDR) frame.

20. The method of claim 17 , further comprising selecting a corresponding motion area in the first frame and the second frame to exclude when generating the single HDR frame.

21. The method of claim 17 , further comprising:

calculating an average luminance level and a saturated pixel count corresponding to a number of saturated pixels associated with a reference frame captured using a single exposure range; and

determining, as the at least one DR setting, the WDR gain setting based on the average luminance level being less than a predetermined luminance threshold and the saturated pixel count being greater than a predetermined saturated pixels threshold.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 4, 2020
From: A9.COM, INC.
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 052835/0254 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 5, 2020
From: JEONG, CHANGSOO
To: RING INC.
Reel/Frame 051722/0455 →
CHANGE OF NAME Recorded Feb 5, 2020
From: RING INC.
To: RING LLC
Reel/Frame 051815/0611 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 5, 2020
From: RING LLC
To: A9.COM, INC.
Reel/Frame 051815/0644 →
Continuity (3)
Continuation 15904214 · Feb 23, 2018
Provisional Application 62463685 · Feb 26, 2017
Related Publication 20200177782A1 · Jun 4, 2020