IP Library Granted Patent US 10,412,272
Granted Patent B2
US 10,412,272 · App. 16/146,334 · Granted Sep 10, 2019

Methods for camera movement compensation

Inventors: Ryan Fink (Vancouver, WA); Gary Peck (Portland, OR); Bryan Ransil (San Francisco, CA)
Assignee: Atheer, Inc.
H04N5/144H04N5/23258H04N5/23261H04N5/23277H04N5/33
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,412,272
App. No.
16/146,334
Granted
Sep 10, 2019
Kind
B2
Abstract

A method, system, apparatus, and/or device for adjusting or removing frames in a set of frames. The method, system, apparatus, and/or device may include: associating a first frame of a set of frames with motion data that is captured approximately contemporaneously with the first frame; when a sampling rate of the motion data is greater than a frame rate of the set of frames, aggregating a first sample of the motion data captured at the first frame and a second sample of the motion data captured between the first frame and a second frame of the set of frames to obtain a movement value; when the movement value does not exceed a first threshold value, accepting the first frame from the set of frames; and when the movement value exceeds the first threshold value, rejecting the first frame from the set of frames.

Claims (41)

1. A method, comprising:

associating, by a processing device, a first frame of a set of frames with motion data that is captured approximately contemporaneously with the first frame, wherein the motion data is indicative of an amount of movement of an image capturing device when the image capturing device captured the first frame;

when a sampling rate of the motion data is greater than a frame rate of the set of frames, aggregating, by the processing device, a first sample of the motion data captured at the first frame and a second sample of the motion data captured between the first frame and a second frame of the set of frames to obtain a movement value;

when the movement value does not exceed a first threshold value, accepting, by the processing device, the first frame from the set of frames; and

when the movement value exceeds the first threshold value, rejecting, by the processing device, the first frame from the set of frames.

2. The method of claim 1 , wherein the first threshold value is indicative of a threshold difference between a first image shown in the first frame and a second image shown in the second frame.

3. The method of claim 1 , wherein the set of frames is a sequential series of frames from a video stream.

4. The method of claim 1 , further comprising:

receiving, from the image capturing device, the set of frames captured at the frame rate; and

receiving, from a sensor, the motion data captured at the sampling rate.

5. The method of claim 1 , further comprising replacing the first frame with a third frame that is previous the first frame in the set of frames when the movement value exceeds the first threshold value.

6. The method of claim 1 , further comprising adjusting the first frame when the movement value does not exceed the first threshold value and exceeds a second threshold value, wherein the first frame is adjusted using at least one of the motion data associated with the first frame or at least a portion of a third frame that is previous to the first frame in the set of frames.

7. A method, comprising:

associating, by a processing device, a first frame of a set of frames with motion data, wherein the motion data is indicative of an amount of movement of an image capturing device when the image capturing device captured the first frame;

when a sampling rate of the motion data is greater than a frame rate of the set of frames, aggregating, by the processing device, a first sample of the motion data captured at the first frame and a second sample of the motion data captured between the first frame and a second frame of the set of frames to obtain a movement value;

when the movement value does not exceed a first threshold value, accepting, by the processing device, the first frame; and

when the movement value exceeds the first threshold value,

replacing, by the processing device, the first frame with the second frame.

8. The method of claim 7 , further comprising:

identifying a third frame subsequent to the first frame with a second amount of movement of the image capturing device that exceeds the first threshold value; and

replacing the third frame with the second frame.

9. The method of claim 7 , further comprising:

receiving, from the image capturing device, the set of frames from a video stream; and

receiving, from a sensor, the motion data.

10. The method of claim 9 , wherein the sensor is physically attached to a housing of the image capturing device to detect the movement value of the image capturing device.

11. The method of claim 7 , further comprising removing the first frame from the set of frames when the first amount of movement of the image capturing device exceeds a second threshold value.

12. A system, comprising:

an image capturing device configured to capture a set of frames;

a sensor configured to capture motion data corresponding to movement of the image capturing device; and

a processing device coupled to the image capturing device and the sensor, wherein the processing device is configured to:

associate a first frame of the set of frames with the motion data, wherein the motion data is indicative of an amount of movement of the image capturing device when the image capturing device captured the first frame;

when a sampling rate of the motion data is greater than a frame rate of the set of frames, aggregating, by the processing device, a first sample of the motion data captured at the first frame and a second sample of the motion data captured between the first frame and a second frame of the set of frames to obtain a movement value;

when the movement value does not exceed a first threshold value, accept the first frame in the set of frames; and

when the movement value exceeds the first threshold value, reject the first frame from the set of frames.

13. The system of claim 12 , wherein the motion data is captured approximately contemporaneously with the first frame.

14. The system of claim 12 , wherein:

the motion data comprises at least one of angular rotation data of the image capturing device when the image capturing device captured the first frame or acceleration data of the image capturing device when the image capturing device captured the first frame; and

the motion data exceeds the first threshold value when the angular rotation data exceeds a second threshold value or the acceleration data exceeds a third threshold value.

15. The system of claim 12 , wherein the processing device is further configured to adjust the first frame when the motion data does not exceed the first threshold value and exceeds a second threshold value.

16. The system of claim 12 , wherein the processing device is further configured to replace the first frame with the second frame that is previous to the first frame when the motion data exceeds the first threshold value, wherein the second frame was previously accepted by the processing device.

17. The system of claim 12 , wherein the processing device is further to adjust the first frame when the motion data exceeds a second threshold value, wherein adjusting the first frame includes at least one of utilizing the motion data associated with the first frame or incorporating at least a portion of the second frame that is previous to the first frame into the first frame.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 7, 2022
From: ATHEER, INC.
To: WEST TEXAS TECHNOLOGY PARTNERS, LLC
Reel/Frame 058962/0067 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 12, 2022
From: FINK, RYAN; PECK, GARY; RANSIL, BRYAN
To: ATHEER, INC.
Reel/Frame 058637/0455 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME PREVIOUSLY RECORDED ON REEL 047153 FRAME 0168. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jan 12, 2022
From: FINK, RYAN
To: ATHEER, INC.
Reel/Frame 058707/0171 →
SECURITY INTEREST Recorded Nov 20, 2018
From: ATHEER, INC.
To: COTA CAPITAL MASTER FUND, L.P., AS COLLATERAL AGENT
Reel/Frame 048154/0759 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 12, 2018
From: FINK, RYAN
To: ATHEER LABS, INC.
Reel/Frame 047153/0168 →
Continuity (3)
Continuation 14806926 · Jul 23, 2015
Provisional Application 62037239 · Aug 14, 2014
Related Publication 20190037112A1 · Jan 31, 2019