IP Library › Granted Patent US 11,991,445
Granted Patent B2
US 11,991,445 · App. 17/413,813 · Granted May 21, 2024

Camera mobile device stabilization using a high frame rate auxiliary imaging element

Inventor: Hao Qian (Hangzhou, CN)
Assignee: HANGZHOU TARO POSITIONING TECHNOLOGY CO., LTD.
H04N23/6811G03B17/561H04N23/56H04N23/6815H04N23/685H04N23/689H04N23/695
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,991,445
App. No.
17/413,813
Granted
May 21, 2024
Kind
B2
Abstract

A method to stabilize image capture for a camera device ( 110 ). The method includes capturing, at an auxiliary frame rate and using an auxiliary imaging element ( 112 ) rigidly coupled with the camera device ( 110 ), auxiliary images of a light source ( 143 ) attached to an object ( 142 ), where the auxiliary frame rate exceeds a camera frame rate of the image capture ( 1211 ); detecting each of a plurality of locations of the light source ( 143 ) in a corresponding one of the auxiliary images ( 1212 ); analyzing the plurality of locations to generate a measure of uncontrolled movement ( 1213 ); and adjusting, based at least on the measure of uncontrolled movement, the camera device ( 110 ) during the image capture of the object by the camera device ( 110 ) at the camera frame rate ( 1214 ).

Claims (97)

1. A method to stabilize image capture for a camera device, comprising:

capturing, at an auxiliary frame rate and using an auxiliary imaging element rigidly coupled with the camera device, a plurality of auxiliary images of a light source attached to an object, wherein the auxiliary frame rate exceeds a camera frame rate of the image capture;

detecting each of a plurality of locations of the light source in a corresponding one of the plurality of auxiliary images;

analyzing, by a hardware processor, the plurality of locations to generate a measure of uncontrolled movement; and

adjusting, based at least on the measure of uncontrolled movement, the camera device during the image capture of the object by the camera device at the camera frame rate.

2. The method of claim 1 ,

wherein the measure of uncontrolled movement represents a deviation from a predicted trajectory of the object across the plurality of auxiliary images.

3. The method of claim 2 , wherein analyzing the plurality of locations comprises:

determining the predicted trajectory by at least applying a curve fitting algorithm to the plurality of locations.

4. The method of claim 3 , further comprising:

analyzing the plurality of locations to generate a measure of controlled movement,

wherein the measure of controlled movement represents the predicted trajectory of the object across the plurality of auxiliary images.

5. The method of claim 4 , further comprising:

mitigating, based on the measure of controlled movement, a rolling shutter effect of the image capture.

6. The method of claim 4 , further comprising:

generating, based on the measure of uncontrolled movement and the measure of controlled movement, a control signal,

wherein the control signal adjusts, during the image capture, a field-of-view of the camera device to perform at least one selected from a group consisting of image stabilization based on the uncontrolled movement and object tracking based on the controlled movement.

7. The method of claim 6 , further comprising:

sending, during the image capture, the control signal to a camera device holder holding the camera device, wherein the control signal comprises at least one selected from a group consisting of a rotating control signal, a tilting control signal, and an additional rotating control signal, wherein adjusting the field-of-view of the camera device comprises at least one selected from a group consisting of:

activating, using the rotating control signal, a rotating motor of the camera device holder;

activating, using the tilting control signal, a tilting motor of the camera device holder; and

activating, using the additional rotating control signal, an additional rotating motor of the camera device holder.

8. A stabilized camera device holder for stabilizing image capture, comprising:

a computer processor and memory storing instructions, when executed, causing the computer processor to:

receive, from an auxiliary imaging element rigidly coupled with a camera device, a plurality of auxiliary images of a light source attached to an object, wherein the plurality of auxiliary images are captured by the auxiliary imaging element at an auxiliary frame rate that exceeds a camera frame rate of the image capture;

detect each of a plurality of locations of the light source in a corresponding one of the plurality of auxiliary images; and

analyze the plurality of locations to generate a measure of uncontrolled movement; and

at least one motor configured to:

adjust, based at least on the measure of uncontrolled movement, the camera device during the image capture of the object by the camera device at the camera frame rate.

9. The stabilized camera device holder of claim 8 , the instructions, when executed, further causing the computer processor to:

wherein the measure of uncontrolled movement represents a deviation from a predicted trajectory of the object across the plurality of auxiliary images.

10. The stabilized camera device holder of claim 9 , wherein analyzing the plurality of locations comprises:

determining the predicted trajectory by at least applying a curve fitting algorithm to the plurality of locations.

11. The stabilized camera device holder of claim 10 , the instructions, when executed, further causing the computer processor to:

analyze the plurality of locations to generate a measure of controlled movement,

wherein the measure of controlled movement represents the predicted trajectory of the object across the plurality of auxiliary images.

12. The stabilized camera device holder of claim 11 , the instructions, when executed, further causing the computer processor to:

mitigate, based on the measure of controlled movement, a rolling shutter effect of the image capture.

13. The stabilized camera device holder of claim 11 , the instructions, when executed, further causing the computer processor to:

generate, based on the measure of uncontrolled movement and the measure of controlled movement, a control signal,

wherein the control signal causes the at least one motor to adjust, during the image capture, a field-of-view of the camera device to perform at least one selected from a group consisting of image stabilization based on the uncontrolled movement and object tracking based on the controlled movement.

14. The stabilized camera device holder of claim 13 ,

wherein the control signal comprises at least one selected from a group consisting of a rotating control signal, a tilting control signal, and an additional rotating control signal,

wherein adjusting the field-of-view of the camera device comprises at least one selected from a group consisting of:

activating, using the rotating control signal, a rotating motor of the camera device holder;

activating, using the tilting control signal, a tilting motor of the camera device holder; and

activating, using the additional rotating control signal, an additional rotating motor of the camera device holder.

15. The stabilized camera device holder of claim 8 ,

wherein the auxiliary imaging element is integrated with at least one selected from a group consisting of the stabilized camera device holder and the camera device as a single device.

16. The stabilized camera device holder of claim 8 ,

wherein the auxiliary frame rate is between 6 times to 10 times of the camera frame rate.

17. A stabilization controller device for stabilizing image capture, comprising:

a computer processor; and

memory storing instructions, when executed, causing the computer processor to:

receive, from an auxiliary imaging element rigidly coupled with a camera device, a plurality of auxiliary images of a light source attached to an object, wherein the plurality of auxiliary images are captured by the auxiliary imaging element at an auxiliary frame rate that exceeds a camera frame rate of the image capture;

detect each of a plurality of locations of the light source in a corresponding one of the plurality of auxiliary images;

analyze the plurality of locations to generate a measure of uncontrolled movement; and

adjust, based at least on the measure of uncontrolled movement, the camera device during the image capture of the object by the camera device at the camera frame rate.

18. The stabilization controller device of claim 17 ,

wherein the measure of uncontrolled movement represents a deviation from a predicted trajectory of the object across the plurality of auxiliary images.

19. The stabilization controller device of claim 18 , wherein analyzing the plurality of locations comprises:

determining the predicted trajectory by at least applying a curve fitting algorithm to the plurality of locations.

20. The stabilization controller device of claim 17 , the instructions, when executed, further causing the computer processor to:

analyze the plurality of locations to generate a measure of controlled movement,

wherein the measure of controlled movement represents the predicted trajectory of the object across the plurality of auxiliary images.

21. The stabilization controller device of claim 20 , the instructions, when executed, further causing the computer processor to:

mitigate, based on the measure of controlled movement, a rolling shutter effect of the image capture.

22. The stabilization controller device of claim 20 , the instructions, when executed, further causing the computer processor to:

generate, based on the measure of uncontrolled movement and the measure of controlled movement, a control signal,

wherein the control signal causes a camera device holder of the camera device to adjust, during the image capture, a field-of-view of the camera device to perform at least one selected from a group consisting of image stabilization based on the uncontrolled movement and object tracking based on the controlled movement.

23. The stabilization controller device of claim 22 ,

wherein the control signal comprises at least one selected from a group consisting of a rotating control signal, a tilting control signal, and an additional rotating control signal,

wherein adjusting the field-of-view of the camera device comprises at least one selected from a group consisting of:

activating, using the rotating control signal, a rotating motor of the camera device holder;

activating, using the tilting control signal, a tilting motor of the camera device holder; and

activating, using the additional rotating control signal, an additional rotating motor of the camera device holder.

24. The stabilization controller device of claim 17 ,

wherein the auxiliary imaging element is integrated with at least one selected from a group consisting of the stabilization controller device and the camera device as a single device.

25. The stabilization controller device of claim 17 ,

wherein the auxiliary frame rate is between 6 times to 10 times of the camera frame rate.

26. A non-transitory computer readable medium storing instructions for stabilizing image capture for a camera device, the instructions, when executed by a computer processor, comprising functionality for:

capturing, at an auxiliary frame rate and using an auxiliary imaging element rigidly coupled with the camera device, a plurality of auxiliary images of a light source attached to an object, wherein the auxiliary frame rate exceeds a camera frame rate of the image capture;

detecting each of a plurality of locations of the light source in a corresponding one of the plurality of auxiliary images;

analyzing the plurality of locations to generate a measure of uncontrolled movement; and

adjusting, based at least on the measure of uncontrolled movement, the camera device during the image capture of the object by the camera device at the camera frame rate.

27. The non-transitory computer readable medium of claim 26 ,

wherein the measure of uncontrolled movement represents a deviation from a predicted trajectory of the object across the plurality of auxiliary images.

28. The non-transitory computer readable medium of claim 27 , wherein analyzing the plurality of locations comprises:

determining the predicted trajectory by at least applying a curve fitting algorithm to the plurality of locations.

29. The non-transitory computer readable medium of claim 26 , the instructions, when executed by the computer processor, further comprising functionality for:

analyzing the plurality of locations to generate a measure of controlled movement,

wherein the measure of controlled movement represents the predicted trajectory of the object across the plurality of auxiliary images.

30. The non-transitory computer readable medium of claim 29 , the instructions, when executed by the computer processor, further comprising functionality for:

mitigating, based on the measure of controlled movement, a rolling shutter effect of the image capture.

31. The non-transitory computer readable medium of claim 29 , the instructions, when executed by the computer processor, further comprising functionality for:

generating, based on the measure of uncontrolled movement and the measure of controlled movement, a control signal,

wherein the control signal adjusts, during the image capture, a field-of-view of the camera device to perform at least one selected from a group consisting of image stabilization based on the uncontrolled movement and object tracking based on the controlled movement.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 17, 2024
From: QIAN, HAO
To: HANGZHOU TARO POSITIONING TECHNOLOGY CO., LTD.
Reel/Frame 067138/0435 →
Continuity (1)
Related Publication 20220078347A1 · Mar 10, 2022