IP Library Granted Patent US 8,941,722
Granted Patent B2
US 8,941,722 · App. 13/342,875 · Granted Jan 27, 2015

Automatic intelligent focus control of video

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,941,722
App. No.
13/342,875
Granted
Jan 27, 2015
Kind
B2
Abstract

The invention is directed to systems, methods and computer program products for providing focus control for an image-capturing device. An exemplary method includes capturing an image frame using an image-capturing device and recording video tracking data and gaze tracking data for one or more image frames following the captured image frame and/or for one or more images frames prior to the captured image frame. The exemplary method additionally includes calculating a focus distance and a depth of field based at least partially on the recorded video tracking data and recorded gaze tracking data. The exemplary method additionally includes displaying the captured image frame based on the calculated focus distance and the calculated depth of field.

Claims (76)

1. A method for providing focus control, the method comprising:

capturing an image frame using an image-capturing device;

recording video tracking data for at least one image frame prior to the captured image frame and following the captured image frame;

recording gaze tracking data for the at least one image frame prior to the captured image frame and following the captured image frame,

wherein the recording video tracking data step further comprises determining at least one moving object based at least partially on the recorded video tracking data,

wherein the recording gaze tracking data step further comprises determining movement of at least one focus area based at least partially on the recorded gaze data;

correlating the at least one moving object with the movement of the at least one focus area;

determining, based at least partially on the correlating step, at least one object the image-capturing device is focusing on;

determining whether the image-capturing device focuses on a single object during a time frame that comprises the captured image frame and the at least one image frame following the captured image frame;

in response to determining the image-capturing device focuses on a single object during the time frame, calculating a focus distance and a depth of field associated with the single object based at least partially on at least one of (i) the recorded video tracking data for the at least one image frame prior to the captured image frame, (ii) the recorded video tracking data for the at least one image frame following the captured image frame, (iii) the recorded gaze tracking data for the at least one image frame prior to the captured image frame, or (iv) the recorded gaze tracking data for the at least one image frame following the captured image frame; and

displaying the captured image frame based at least partially on the calculated focus distance and the calculated depth of field.

2. The method of claim 1 , wherein the recording video tracking data step further comprises determining a distance from the image-capturing device to the at least one moving object in the captured image frame.

3. The method of claim 2 , wherein the distance to the at least one moving object in the captured image frame is determined at a time after the capturing step.

4. The method of claim 1 , further comprising:

determining whether the image-capturing device shifts focus from a first object to a second object during a time frame that comprises the captured image frame and the at least one image frame following the captured image frame; and

in response to determining the image-capturing device shifts focus from the first object to the second object, initiating transition from a first focus setting and a first depth of field setting associated with the first object to a second focus setting and a second depth of field setting associated with the second object.

5. The method of claim 1 , wherein the image-capturing device is a lightfield image-capturing device.

6. The method of claim 1 , wherein the captured image frame and the at least one image frame following the captured image frame comprise a video.

7. The method of claim 1 , wherein the image-capturing device is a stereoscopic image capturing device.

8. The method of claim 1 , further comprising:

allowing a user to manually adjust the calculated focus distance and the calculated depth of field either using the image-capturing device or using a separate computing device.

9. The method of claim 1 , further comprising recording video tracking data and gaze tracking data for the image frame while simultaneously capturing the image frame.

10. The method of claim 1 , further comprising:

enabling the image-capturing device to predict an object's movement following the captured image frame based at least partially on the video tracking data recorded after the captured image frame, wherein the object is positioned in the captured image frame.

11. The method of claim 1 , further comprising:

determining whether a movement of the image-capturing device is an initiation of a change in focus of the image-capturing device or is a temporary movement of the image-capturing device based at least partially on the gaze tracking data recorded after the captured image frame; and

in response to determining the movement of the image-capturing device is an initiation of a change in focus of the image-capturing device, initiating a change in focus of the image-capturing device.

12. The method of claim 11 , further comprising:

in response to determining the movement of the image-capturing device is a temporary movement, retaining a current focus of the image-capturing device.

13. The method of claim 1 , wherein the image-capturing device determines a number of the at least one image frame that follows the captured image frame based at least partially on at least one of previously recorded video tracking data or previously recorded gaze tracking data.

14. A system for providing focus control, the system comprising:

an image-capturing component to capture an image frame;

a processor to initiate:

recording of video tracking data for at least one image frame prior to the captured image frame and following the captured image frame;

recording of gaze tracking data for the at least one image frame prior to the captured image frame and following the captured image frame,

wherein the recording video tracking data step further comprises determining at least one moving object based at least partially on the recorded video tracking data,

wherein the recording gaze tracking data step further comprises determining movement of at least one focus area based at least partially on the recorded gaze data;

correlate the at least one moving object with the movement of the at least one focus area;

determine, based at least partially on the correlating step, at least one object the image-capturing device is focusing on;

determine whether the image-capturing device focuses on a single object during a time frame that comprises the captured image frame and the at least one image frame following the captured image frame;

in response to determining the image-capturing device focuses on a single object during the time frame, calculate a focus distance and a depth of field associated with the single object based at least partially on at least one of (i) the recorded video tracking data for the at least one image frame prior to the captured image frame, (ii) the recorded video tracking data for the at least one image frame following the captured image frame, (iii) the recorded gaze tracking data for the at least one image frame prior to the captured image frame, or (iv) the recorded gaze tracking data for the at least one image frame following the captured image frame; and

displaying the captured image frame based at least partially on the calculated focus distance and the calculated depth of field.

15. A method for providing focus control, the method comprising:

capturing an image frame using an image-capturing device;

recording video tracking data for at least one image frame following the captured image frame;

recording gaze tracking data for the at least one image frame following the captured image frame;

calculating a focus distance and a depth of field based at least partially on the recorded video tracking data and the recorded gaze tracking data;

displaying the captured image frame based at least partially on the calculated focus distance and the calculated depth of field;

determining whether a movement of the image-capturing device is an initiation of a change in focus of the image-capturing device or is a temporary movement of the image-capturing device based at least partially on the gaze tracking data recorded after the captured image frame; and

in response to determining the movement of the image-capturing device is an initiation of a change in focus of the image-capturing device, initiating a change in focus of the image-capturing device.

16. The method of claim 15 , further comprising:

in response to determining the movement of the image-capturing device is a temporary movement, retaining a current focus of the image-capturing device.

17. A method for providing focus control, the method comprising:

capturing an image frame using an image-capturing device;

recording video tracking data for at least one image frame following the captured image frame;

recording gaze tracking data for the at least one image frame following the captured image frame;

calculating a focus distance and a depth of field based at least partially on the recorded video tracking data and the recorded gaze tracking data;

displaying the captured image frame based at least partially on the calculated focus distance and the calculated depth of field,

wherein the recording video tracking data step further comprises determining at least one moving object based at least partially on the recorded video tracking data,

wherein the recording gaze tracking data step further comprises determining movement of at least one focus area based at least partially on the recorded gaze data;

correlating the at least one moving object with the movement of the at least one focus area;

determining, based at least partially on the correlating step, at least one object the image-capturing device is focusing on;

determining whether the image-capturing device focuses on a single object during a time frame that comprises the captured image frame and the at least one image frame following the captured image frame; and

in response to determining the image-capturing device focuses on a single object during the time frame, calculating a focus distance and a depth of field associated with the single object.

18. A method for providing focus control, the method comprising:

capturing an image frame using an image-capturing device;

recording video tracking data for at least one image frame following the captured image frame;

recording gaze tracking data for the at least one image frame following the captured image frame;

calculating a focus distance and a depth of field based at least partially on the recorded video tracking data and the recorded gaze tracking data;

displaying the captured image frame based at least partially on the calculated focus distance and the calculated depth of field,

wherein the recording video tracking data step further comprises determining at least one moving object based at least partially on the recorded video tracking data,

wherein the recording gaze tracking data step further comprises determining movement of at least one focus area based at least partially on the recorded gaze data;

correlating the at least one moving object with the movement of the at least one focus area;

determining, based at least partially on the correlating step, at least one object the image-capturing device is focusing on;

determining whether the image-capturing device shifts focus from a first object to a second object during a time frame that comprises the captured image frame and the at least one image frame following the captured image frame; and

in response to determining the image-capturing device shifts focus from the first object to the second object, initiating transition from a first focus setting and a first depth of field setting associated with the first object to a second focus setting and a second depth of field setting associated with the second object.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 22, 2017
From: SONY MOBILE COMMUNICATIONS AB
To: SONY MOBILE COMMUNICATIONS INC.
Reel/Frame 043951/0529 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 21, 2017
From: SONY CORPORATION
To: SONY MOBILE COMMUNICATIONS INC.
Reel/Frame 043943/0631 →
CHANGE OF NAME Recorded Sep 20, 2017
From: SONY ERICSSON MOBILE COMMUNICATIONS AB
To: SONY MOBILE COMMUNICATIONS AB
Reel/Frame 043921/0551 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2014
From: SONY MOBILE COMMUNICATIONS AB
To: SONY CORPORATION; SONY MOBILE COMMUNICATIONS AB
Reel/Frame 034441/0469 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 2, 2012
From: ARONSSON, PAR-ANDERS; DE LEON, DAVID; JONSSON, HAKAN; KLINGHULT, GUNNAR; THORN, OLA
To: SONY ERICSSON MOBILE COMMUNICATIONS AB
Reel/Frame 027639/0615 →