IP Library Granted Patent US 10,708,491
Granted Patent B2
US 10,708,491 · App. 15/109,466 · Granted Jul 7, 2020

Adaptive camera control for reducing motion blur during real-time image capture

Inventors: Alexander Ilic (Zurich, CH); Manuel Werlberger (Zurich, CH)
Assignee: ML Netherlands C.V.
H04N5/23222H04N5/2352H04N5/23251H04N5/23267H04N5/232941H04N5/23254H04N5/23258
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,708,491
App. No.
15/109,466
Granted
Jul 7, 2020
Kind
B2
Abstract

A method of forming a visual representation of an object from a plurality of image frames acquired using a portable electronic device, the method comprising the steps of determining at least one parameter of motion of the portable electronic device; determining at least one capture condition for at least one first image frame of the plurality of image frames; computing, based on the at least one parameter of motion and the at least one capture condition, an indication of blur in the image frame; based on the indication of blur, conditionally taking a corrective action.

Claims (43)

1. A method of forming a visual representation of an object from a plurality of image frames acquired using a portable electronic device, the method being performed by at least one processor of the portable electronic device and comprising:

determining, using the at least one processor, at least one parameter of motion of the portable electronic device, wherein the at least one parameter of motion is a distance moved during an exposure time, the at least one parameter of motion is based on one image frame and a subsequent image frame of the plurality of image frames, and wherein the plurality of image frames are captured continuously;

determining, using the at least one processor, at least one capture condition for at least one first image frame of the plurality of image frames;

computing, using the at least one processor and based on the at least one parameter of motion and the at least one capture condition, an indication of blur in the at least one first image frame; and

based on the indication of blur, conditionally taking a corrective action, using the at least one processor, wherein the corrective action comprises:

generating user feedback for a user of the portable electronic device, the user feedback comprising a message instructing the user to slow the motion of the portable electronic device; and

changing a capture condition for at least one second image frame after the at least one first image frame.

2. The method of claim 1 , wherein:

changing the capture condition comprises changing an exposure time for the at least one second image frame.

3. The method of claim 1 , wherein:

changing the capture condition comprises changing a gain for the at least one second image frame.

4. A method of forming a visual representation of an object from a plurality of image frames acquired using a portable electronic device, the method being performed by at least one processor of the portable electronic device and comprising:

determining, using the at least one processor, at least one parameter of motion of the portable electronic device, wherein the at least one parameter of motion is a distance moved during an exposure time, the at least one parameter of motion is based on one image frame and a subsequent image frame of the plurality of image frames, and wherein the plurality of image frames are captured continuously;

determining, using the at least one processor, at least one capture condition for at least one first image frame of the plurality of image frames;

computing, using the at least one processor and based on the at least one parameter of motion and the at least one capture condition, an indication of blur in the at least one first image frame; and

based on the indication of blur, conditionally taking a corrective action, using the at least one processor, wherein the corrective action comprises:

generating user feedback for a user of the portable electronic device, wherein the user feedback comprises a message instructing the user to slow the motion of the portable electronic device, and wherein the user feedback comprises a message instructing the user to turn on a flash of the portable electronic device.

5. The method of claim 1 , wherein conditionally taking the corrective action comprises taking the corrective action based on a comparison of the indication of blur to a threshold.

6. A method of forming a visual representation of an object from a plurality of image frames acquired using a portable electronic device, the method being performed by at least one processor of the portable electronic device and comprising:

determining, using the at least one processor, at least one parameter of motion of the portable electronic device, wherein the at least one parameter of motion is a distance moved during an exposure time, the at least one parameter of motion is based on one image frame and a subsequent image frame of the plurality of image frames, and wherein the plurality of image frames are captured continuously;

determining, using the at least one processor, at least one capture condition for at least one first image frame of the plurality of image frames;

computing, using the at least one processor and based on the at least one parameter of motion and the at least one capture condition, an indication of blur in the at least one first image frame; and

based on the indication of blur, conditionally taking a corrective action, using the at least one processor, wherein the corrective action comprises:

generating user feedback for a user of the portable electronic device, wherein the user feedback comprises a message instructing the user to slow the motion of the portable electronic device; and

computing an exposure time and a gain based on the at least one determined parameter of motion.

7. The method of claim 6 , wherein conditionally taking the corrective action comprises:

selecting a change in a capture condition and/or to provide user feedback based on the at least one current capture condition.

8. The method of claim 6 , wherein:

conditionally taking the corrective action comprises selecting a change in the capture condition when the computed gain is above a threshold.

9. The method of claim 6 , wherein:

conditionally taking the corrective action comprises selecting to provide user feedback when the computed gain is above a threshold.

10. The method of claim 9 , wherein:

detecting user compliance with the corrective action and removing the user feedback.

11. The method of claim 1 , wherein the at least one parameter of motion is based on at least one inertial sensor of the portable electronic device.

12. At least one non-transitory, tangible computer readable storage medium having computer-executable instructions, that when executed by a processor, perform a method of forming a representation of an object from a plurality of image frames captured with a camera of a portable electronic device, the method comprising:

determining at least one parameter of motion of the portable electronic device, wherein the at least one parameter of motion is based on one image frame and a subsequent image frame of the plurality of image frames, and wherein the plurality of image frames are captured continuously;

determining at least one capture condition for the camera for at least one first image frame of the plurality of image frames;

computing, based on the at least one parameter of motion and the at least one capture condition, an indication of blur in the at least one first image frame;

based on the indication of blur, conditionally taking a corrective action, wherein the corrective action comprises adjusting, iteratively, one or more camera parameters within thresholds set for the camera parameters and generating user feedback for a user of the portable electronic device, the user feedback comprising a message instructing the user to turn on a flash of the portable electronic device.

13. The at least one non-transitory, tangible computer readable storage medium of claim 12 , wherein the corrective action comprises changing a capture condition for at least one second image frame after the at least one first image frame.

14. The at least one non-transitory, tangible computer readable storage medium of claim 12 , wherein adjusting, iteratively, one or more camera parameters comprises iteratively changing an exposure time and a gain.

15. The at least one non-transitory, tangible computer readable storage medium of claim 12 , wherein computing, based on the at least one parameter of motion and the at least one capture condition, an indication of blur in the at least one first image frame comprises using the at least one parameter of motion and a focus distance to determine a relative motion of an object within the at least one first image frame.

16. The at least one non-transitory, tangible computer readable storage medium of claim 14 , wherein iteratively changing the exposure time and the gain comprises preferentially changing the exposure time.

Assignments (6)
SECURITY INTEREST Recorded Oct 24, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073255/0581 →
MERGER Recorded Oct 3, 2022
From: ML NETHERLANDS C.V.
To: MR INTERNATIONAL HOLDING LLC
Reel/Frame 061290/0365 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 3, 2022
From: MR INTERNATIONAL HOLDING LLC
To: MAGIC LEAP, INC.
Reel/Frame 061290/0606 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 14, 2017
From: DACUDA AG
To: ML NETHERLANDS C.V.
Reel/Frame 041254/0163 →
CORRECTIVE ASSIGNMENT TO CORRECT THE 2ND INVENTOR NAME PREVIOUSLY RECORDED AT REEL: 040713 FRAME: 0699. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 23, 2016
From: ILIC, ALEXANDER; WERLBERGER, MANUEL
To: DACUDA AG
Reel/Frame 041188/0486 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2016
From: ILIC, ALEXANDER; WERBERGER, MANUEL
To: DACUDA AG
Reel/Frame 040713/0699 →
Continuity (2)
Provisional Application 61924698 · Jan 7, 2014
Related Publication 20160330374A1 · Nov 10, 2016
Cited By (1)
US 12,309,333