IP Library Granted Patent US 10,270,958
Granted Patent B2
US 10,270,958 · App. 15/254,964 · Granted Apr 23, 2019

Systems and methods for adjusting focus based on focus target information

Inventors: William Guie Rivard (Menlo Park, CA); Brian J. Kindle (Sunnyvale, CA); Adam Barry Feder (Mountain View, CA)
Assignee: DUELIGHT LLC
H04N5/23212H04N5/2252H04N5/2253H04N5/2254H04N5/2257H04N5/2258H04N5/2356H04N5/23216H04N5/23219H04N5/23222H04N5/265H04N5/2256H04N5/23232
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,270,958
App. No.
15/254,964
Granted
Apr 23, 2019
Kind
B2
Abstract

A system, method, and computer program product are provided for adjusting focus based on focus target information. In use, an image is sampled as image data, using a camera module. Next, the image data is transmitted for processing, wherein the processing includes identifying one or more focus regions. Additionally, focus target information is determined corresponding to one or more focus regions. Further, a focus is adjusted, based on the focus target information, and the image is captured. Additional systems, methods, and computer program products are also presented.

Claims (55)

1. An apparatus, comprising:

a camera module, comprising:

an image sensor, configured to convert a first optical scene information to a first electrical representation, wherein the first electrical representation includes first focus pixel information;

a lens assembly, configured to adjust focus of the first optical scene information based on a focus control signal;

a controller, coupled to the image sensor and the lens assembly and configured to:

receive focus target information, wherein the focus target information includes at least one of: two dimensional (2D) coordinates, pixel image plane coordinates, integer pixel coordinates, normalized coordinates, or XY coordinates;

receive the first electrical representation from the image sensor;

continuously generate, by a focus control system within the controller, the focus control signal based on the focus target information and the first focus pixel information;

transmit the focus control signal to the lens assembly;

generate, by an array of analog-to-digital converter circuits, a digital representation of the first electrical representation; and

a camera interconnect, coupled to the controller, and configured to transmit the focus target information to the controller, and the digital representation from the controller,

wherein the lens assembly, the image sensor and the controller are packaged together in a rigid housing; and

an application processor, coupled to the camera interconnect, and configured to:

receive the digital representation;

identify one or more focus regions based on the digital representation;

continuously generate the focus target information based on the one or more focus regions; and

cause transmission of the focus target information to the controller through the camera interconnect,

wherein the application processor is packaged outside of the rigid housing.

2. The apparatus of claim 1 , wherein the one or more focus regions include one or more pixel points.

3. The apparatus of claim 1 , wherein the one or more focus regions include one or more pixel points with a radius and weight profile.

4. The apparatus of claim 3 , wherein the weight profile includes a start point and an end point, with a predefined function to be applied between the start point and the end point.

5. The apparatus of claim 1 , wherein the one or more focus regions include one or more pixel points with a weight mask.

6. The apparatus of claim 5 , wherein one or more weights associated with the weight mask are used to establish a focus estimate.

7. The apparatus of claim 1 , wherein first information from one or more focus pixels associated with the one or more focus regions is aggregated to generate a focus estimate for the one or more focus regions, the focus estimate based on the focus target information.

8. The apparatus of claim 1 , wherein the focus target information is saved as metadata.

9. The apparatus of claim 1 , wherein the one or more identified focus regions are selected by a user.

10. The apparatus of claim 1 , wherein the image sensor is further configured to convert a second optical scene information to a second electrical representation, wherein the second electrical representation includes second focus pixel information, the lens assembly is further configured to adjust focus of the second optical scene information, and the controller is further configured to:

receive second focus target information;

receive the second electrical representation from the image sensor; and

continuously generate, by the focus control system, a second focus control signal based on the second focus target information and the second focus pixel information.

11. The apparatus of claim 10 , wherein the application processor is configured to blend a first image based on the first optical scene information and the second optical scene information.

12. The apparatus of claim 10 , wherein the focus target information corresponds with a first focus setting, and the second focus target information corresponds with a second focus setting.

13. The apparatus of claim 12 , wherein the first focus setting is used to capture a first image, and the second focus setting is used to capture a second image, wherein the first image and the second image are blended to create a first blended image.

14. The apparatus of claim 1 , wherein the controller is further configured to:

receive updated focus target information; and

in response to receiving the updated focus target information, adjust the lens assembly, based on the updated focus target information.

15. The apparatus of claim 1 , wherein to continuously adjust focus, the controller is configured to:

generate a focus estimate based on the focus target information and second focus target information associated with a second optical scene;

generate a second focus signal based on the focus estimate; and

transmit the focus signal to the lens assembly to cause the lens assembly to adjust the focus.

16. The apparatus of claim 15 , wherein the focus estimate is generated according to a weighted sum of two or more products, wherein each product is calculated by multiplying a focus pixel difference signal by a corresponding weight value.

17. The apparatus of claim 1 , wherein the focus target information includes at least one coordinate and is represented as at least one of: a list, an array, or a bit map.

18. The apparatus of claim 1 , wherein the controller is further configured to translate at least one coordinate of the focus target information from at least one of: a three-dimensional (3D) mapping to a two-dimensional (2D) mapping, an image sensor row and column mapping to an image plane mapping, or an XY image coordinate to a pixel image coordinate.

19. The apparatus of claim 1 , wherein the one or more focus regions includes an object, and the application processor tracks the object such that when the object moves, the application processor updates the focus target information to maintain focus.

20. A method, comprising:

receiving, using a controller coupled to an image sensor and a lens assembly, focus target information, wherein the focus target information includes at least one of: two dimensional (2D) coordinates, pixel image plane coordinates, integer pixel coordinates, normalized coordinates, or XY coordinates;

receiving, using the controller, a first electrical representation, including first focus pixel information from the image sensor;

continuously generating, using the controller, by a focus control system within the controller, a focus control signal, configured to adjust focus of a first optical scene information of the lens assembly, based on the focus target information and the first focus pixel information;

transmitting, using the controller, the focus control signal to the lens assembly;

generating, using the controller, by an array of analog-to-digital converter circuits, a digital representation of the first electrical representation;

receiving, using an application processor, the digital representation;

identifying, using the application processor, one or more focus regions based on the digital representation;

continuously generating, using the application processor, the focus target information based on the one or more focus regions; and

transmitting, using the application processor, the focus target information to the controller through a camera interconnect;

wherein the lens assembly, the image sensor and the controller are packaged together in a rigid housing, and the application processor is packaged outside of the rigid housing.

Assignments (2)
SECURITY INTEREST Recorded Jul 30, 2021
From: DUELIGHT, LLC
To: ZILKA KOTAB, P.C.
Reel/Frame 057043/0301 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 10, 2017
From: RIVARD, WILLIAM GUIE; KINDLE, BRIAN J.; FEDER, ADAM BARRY
To: DUELIGHT LLC
Reel/Frame 042321/0915 →
Continuity (1)
Related Publication 20180063409A1 · Mar 1, 2018
Cited By (5)
US 12,401,911 US 12,401,912 US 12,418,727 US 12,445,736 US 12,666,159