IP Library › Granted Patent US 10,510,153
Granted Patent B1
US 10,510,153 · App. 15/633,035 · Granted Dec 17, 2019

Camera-level image processing

Inventors: Alexander Bruce Gentles (Brampton, GB); Naushirwan Navroze Patuck (Cambridge, GB); Christopher Hong-Wen Tserng (Plano, TX); Vinod Kancharla Prasad (Monte Sereno, CA); Reto Koradi (San Jose, CA); Guhan Lakshminarayanan (Grapevine, TX); Kunal Patel (Seattle, WA)
Assignee: AMAZON TECHNOLOGIES, INC.
G06T7/44G06F3/0412G06T7/90G06T2207/20201H04N5/343
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,510,153
App. No.
15/633,035
Granted
Dec 17, 2019
Kind
B1
Abstract

Devices, systems and methods are disclosed for performing image processing at a camera-level. For example, a camera service may run on top of a camera hardware abstraction layer (HAL) and may be configured to perform image processing such as applying a blurring algorithm, applying a color filter and/or other video effects. An application may pass metadata to the camera service via an application programming interface (API) and the camera service may use the metadata to determine parameters for the image processing. The camera service may apply the blurring algorithm for a first period of time before transitioning to unblurred image data over a second period of time.

Claims (111)

1. A computer-implemented method, comprising:

receiving an indication indicating a portion of first image data;

sending, by an application to a camera service component, a request for image data, the request including first parameters that include pixel information corresponding to the portion of the first image data;

sending, by the camera service component to an image sensor component, an instruction to send first raw image data generated by the image sensor component;

receiving, by the camera service component from the image sensor component, the first raw image data;

determining, by the camera service component based on the pixel information, a portion of the raw image data to which to apply a blurring algorithm;

determining, by the camera service component, a first strength value indicating an amount of blur to apply using the blurring algorithm;

determining, by the camera service component, a first width value indicating a number of pixels associated with the blurring algorithm;

generating blurred image data by applying, by the camera service component, the blurring algorithm to the portion of the first raw image data using the first strength value and the first width value to blur the portion of the first raw image data, a portion of the blurred image data blurred relative to the portion of the first raw image data;

generating first processed image data by applying, by the camera service component, a color filter to at least a portion of the blurred image data, the color filter modifying color values of the blurred image data; and

sending, by the camera service component, the first processed image data.

2. The computer-implemented method of claim 1 , further comprising:

displaying the first image data on a touchscreen display;

receiving, via touch input on the touchscreen display, the indication indicating the portion of the first image data; and

determining the pixel information corresponding to the portion of the first image data.

3. The computer-implemented method of claim 1 , further comprising:

connecting to a remote device via a wireless link;

sending the first image data to the remote device via the wireless link;

receiving, from the remote device, the indication indicating the portion of the first image data; and

determining the pixel information corresponding to the portion of the first image data.

4. The computer-implemented method of claim 1 , further comprising:

sending the first image data to one or more servers;

receiving, from the one or more servers, an instruction to blur the portion of the first image data;

receiving, from the one or more servers, the indication indicating the portion of the first image data; and

determining the pixel information corresponding to the portion of the first image data.

5. A computer-implemented method, comprising:

receiving an indication indicating a portion of first image data;

sending, by an application, a request for image data, the request including first parameters that include pixel information corresponding to the portion of the first image data;

sending, to an image sensor component, an instruction to send first raw image data;

receiving, from the image sensor component, the first raw image data;

determining, based on the pixel information, a portion of the raw image data to which to apply a blurring algorithm;

generating first processed image data by applying, by a camera service component, the blurring algorithm to the portion of the first raw image data to blur the portion of the first raw image data, the camera service component being configured to forward requests for raw image data from the application to the image sensor component; and

sending, by the camera service component, the first processed image data.

6. The computer-implemented method of claim 5 , further comprising, by the camera service component:

determining a first strength value indicating the amount of blur to apply using the blurring algorithm;

determining a first width value indicating a number of pixels associated with the blurring algorithm;

generating blurred image data by applying the blurring algorithm to the portion of the first raw image data using the first strength value and the first width value; and

generating the first processed image data by applying a color filter to at least a portion of the blurred image data, the color filter modifying color values of the blurred image data.

7. The computer-implemented method of claim 5 , further comprising, by the camera service component:

determining, based on the pixel information, a second portion of the first raw image data that corresponds to the portion of the first image data;

determining the portion of the first raw image data that is not included in the second portion of the first raw image data; and

generating the first processed image data by applying the blurring algorithm to the portion of the first raw image data.

8. The computer-implemented method of claim 5 , further comprising:

determining, using computer vision processing, a region of interest represented in the first image data;

determining the portion of the first image data corresponding to the region of interest; and

determining the pixel information corresponding to the portion of the first image data.

9. The computer-implemented method of claim 5 , further comprising:

displaying the first image data on a touchscreen display;

receiving, via touch input on the touchscreen display, the indication indicating the portion of the first image data; and

determining the pixel information corresponding to the portion of the first image data.

10. The computer-implemented method of claim 5 , further comprising:

connecting to a remote device via a wireless link;

sending the first image data to the remote device via the wireless link; and

receiving, from the remote device, the indication indicating the portion of the first image data.

11. The computer-implemented method of claim 5 , further comprising:

sending the first image data to one or more servers;

receiving, from the one or more servers, an instruction to blur the portion of the first image data; and

receiving, from the one or more servers, the indication indicating the portion of the first image data.

12. The computer-implemented method of claim 5 , further comprising:

sending, by the first application to an application programming interface (API), the request;

generating, by the first application, a bounding box corresponding to at least a portion of a display;

sending, by the API to the camera service component, the request;

sending, by the camera service component to the API, the first processed image data;

sending, by the API, the first processed image data to the remote device; and

displaying, using the API, second image data corresponding to the first processed image data in the bounding box on the display.

13. A first device, comprising:

at least one processor; and

memory including instructions operable to be executed by the at least one processor to cause the first device to:

receive an indication indicating a portion of first image data;

send, by an application to a camera service component, a request for image data, the request including first parameters that include pixel information corresponding to the portion of the first image data;

send, by the camera service component to an image sensor component, an instruction to send first raw image data;

receive, by the camera service component from the image sensor component, the first raw image data;

determine, by the camera service component based on the pixel information, a portion of the raw image data to which to apply a blurring algorithm;

generate first processed image data by applying, by the camera service component and using first parameters, the blurring algorithm to the portion of the first raw image data to blur the portion of the first raw image data, wherein the first parameters include a first strength value indicating an amount of blur to apply using the blurring algorithm; and

send, by the camera service component, the first processed image data.

14. The first device of claim 13 , wherein the memory further comprises instructions that, when executed by the at least one processer, further cause the first device to:

determine the first strength value indicating the amount of blur to apply using the blurring algorithm;

determine a first width value indicating a number of pixels associated with the blurring algorithm;

generate blurred image data by applying the blurring algorithm to the portion of the first raw image data using the first strength value and the first width value; and

generate the first processed image data by applying a color filter to at least a portion of the blurred image data, the color filter modifying color values of the blurred image data.

15. The first device of claim 13 , wherein the memory further comprises instructions that, when executed by the at least one processer, further cause the first device to:

determine, based on the pixel information, a second portion of the first raw image data that corresponds to the portion of the first image data;

determine the portion of the first raw image data that is not included in the second portion of the first raw image data; and

generate the first processed image data by applying the blurring algorithm to the portion of the first raw image data.

16. The first device of claim 13 , wherein the memory further comprises instructions that, when executed by the at least one processer, further cause the first device to:

determine, using computer vision processing, a region of interest represented in the first image data;

determine the portion of the first image data corresponding to the region of interest; and

determine the pixel information corresponding to the portion of the first image data.

17. The first device of claim 13 , wherein the memory further comprises instructions that, when executed by the at least one processer, further cause the first device to:

display the first image data on a touchscreen display;

receive, via touch input on the touchscreen display, the indication indicating the portion of the first image data; and

determine the pixel information corresponding to the portion of the first image data.

18. The first device of claim 13 , wherein the memory further comprises instructions that, when executed by the at least one processer, further cause the first device to:

connect to a remote device via a wireless link;

send the first image data to the remote device via the wireless link; and

receive, from the remote device, the indication indicating the portion of the first image data.

19. The first device of claim 13 , wherein the memory further comprises instructions that, when executed by the at least one processer, further cause the first device to:

send the first image data to one or more servers;

receive, from the one or more servers, an instruction to blur the portion of the first image data; and

receive, from the one or more servers, the indication indicating the portion of the first image data.

20. The first device of claim 13 , wherein the memory further comprises instructions that, when executed by the at least one processer, further cause the first device to:

send, by the first application to an application programming interface (API), the request;

generate, by the first application, a bounding box corresponding to at least a portion of a display;

send, by the API to the camera service component, the request;

send, by the camera service component to the API, the first processed image data;

send, by the API, the first processed image data to the remote device; and

display, using the API, second image data corresponding to the first processed image data in the bounding box on the display.

21. The computer-implemented method of claim 5 , wherein generating the first processed image data further comprises:

determining a first strength value indicating an amount of blur to apply using the blurring algorithm;

determining one or more first parameters corresponding to the blurring algorithm; and

generating the first processed image data by applying the blurring algorithm to the first raw image data using the first strength value and the one or more first parameters.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2019
From: GENTLES, ALEXANDER BRUCE; PATUCK, NAUSHIRWAN NAVROZE; TSERNG, CHRISTOPHER HONG-WEN; PRASAD, VINOD KANCHARLA; KORADI, RETO; LAKSHMINARAYANAN, GUHAN; PATEL, KUNAL
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 050009/0001 →