IP Library Granted Patent US 10,674,086
Granted Patent B2
US 10,674,086 · App. 15/924,428 · Granted Jun 2, 2020

Image processing method and device and non-transitory computer-readable medium

Inventors: Yiwei Wang (Beijing, CN); Lingyan Liang (Beijing, CN)
Assignee: Ricoh Company, Ltd.
H04N5/23254G06T7/248H04N5/23248H04N5/23258H04N5/23261H04N5/23264H04N5/23267H04N5/23296G06T2207/30241
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,674,086
App. No.
15/924,428
Granted
Jun 2, 2020
Kind
B2
Abstract

Image processing method and device are disclosed. The image processing method is inclusive of a step of obtaining a target image; a step of performing motion estimation on the target image so as to attain motion paths; a step of determining, on the basis of the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image; and a step of taking, if there exists a fluctuation meeting the predetermined requirement in the target image, the target image as a reference image. The reference image is utilized to carry out inter frame motion estimation with respect to the respective images acquired.

Claims (46)

1. An image processing method comprising:

obtaining a target image;

performing motion estimation on the target image so as to attain motion paths;

determining, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image; and

taking, if there exists a fluctuation meeting the predetermined requirement in the target image, the target image as a reference image, the reference image being used to carry out inter frame motion estimation with respect to respective images acquired, wherein

the performing motion estimation on the target image so as to attain motion paths includes conducting inter frame motion estimation and real motion estimation in regard to the target image so as to attain an inter frame motion trajectory and a real motion trajectory, respectively, and

the determining, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image includes judging whether there is a fluctuation meeting the predetermined requirement in the inter frame motion trajectory and the real motion trajectory, respectively, wherein

the image processing method further comprising:

after the determining, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image, gaining, if there is a fluctuation meeting the predetermined requirement in the inter frame motion trajectory and the real motion trajectory, respectively, a first duration of the fluctuation in the inter frame motion trajectory and a second duration of the fluctuation in the real motion trajectory,

wherein, the taking the target image as a reference image includes

letting, if the first duration and the second duration are both greater than a predetermine threshold, the target image be a reference image.

2. The image processing method according to claim 1 , wherein, the determining, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image includes

judging whether there is a fluctuation meeting the predetermined requirement in the real motion trajectory.

3. The image processing method according to claim 2 , further comprising:

after the determining, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image, gaining, if there is a fluctuation meeting the predetermined requirement in the real motion trajectory, a duration of the fluctuation in the real motion trajectory,

wherein, the taking the target image as a reference image includes

letting, if the duration is greater than a predetermined threshold, the target image be a reference image.

4. The image processing method according to claim 1 , further comprising:

conducting vertical correction in regard to the reference image so as to procure a corrected reference image which is utilized to carry out inter frame motion estimation with respect to the respective image acquired;

performing a smoothing process on the motion paths; and

carrying out motion compensation based on the motion paths after smoothing.

5. The image processing method according to claim 4 , wherein, the conducting vertical correction in regard to the reference image includes

calculating an average level of motion within each sampling period in the real motion trajectory;

obtaining a three dimensional reference image corresponding to the reference image; and

rotating, based on the average levels of motion, the three dimensional reference image, so as to cause a two dimensional unfolded view corresponding to the rotated three dimensional reference image to be perpendicular to a line of sight of a relevant user.

6. The image processing method according to claim 4 , wherein, the performing a smoothing process on the motion paths includes

conducting, by means of median filtering, the smoothing process in regard to the motion paths.

7. The image processing method according to claim 4 , wherein, the carrying out motion compensation based on the motion paths after smoothing includes

projecting the target image onto a spherical coordinate system so as to attain a spherical image;

carrying out coordinate conversion with respect to pixel points in the spherical image so as to gain a spherical image after image stabilization; and

transforming the spherical image after image stabilization into a two dimensional image.

8. An image processing device comprising:

a memory; and

a processor coupled to the memory, wherein

the processor

obtains a target image,

performs motion estimation on the target image so as to attain motion paths,

determines, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image,

takes, if there exists a fluctuation meeting the predetermined requirement in the target image, the target image as a reference image, the reference image being utilized to carry out inter frame motion estimation with respect to respective images acquired, wherein

when performing motion estimation on the target image so as to attain motion paths, the processor conducts inter frame motion estimation and real motion estimation in regard to the target image so as to attain an inter frame motion trajectory and a real motion trajectory, respectively, and

when determining, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image, the processor judges whether there is a fluctuation meeting the predetermined requirement in the inter frame motion trajectory and the real motion trajectory, respectively, and wherein

the processor,

after the processor's determining, based on the motion paths, whether there exists a fluctuation meeting a predetermined requirement in the target image, gains, if there is a fluctuation meeting the predetermined requirement in the inter frame motion trajectory and the real motion trajectory, respectively, a first duration of the fluctuation in the inter frame motion trajectory and a second duration of the fluctuation in the real motion trajectory,

wherein, the processor's taking the target image as a reference image includes

letting, if the first duration and the second duration are both greater than a predetermine threshold, the target image be a reference image.

9. A non-transitory computer-readable medium having computer-executable instructions for execution by a processing system, wherein, the computer-executable instructions, when executed, cause the processing system to carry out the image processing method according to claim 1 .

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2018
From: WANG, YIWEI; LIANG, LINGYAN
To: RICOH COMPANY, LTD.
Reel/Frame 045264/0646 →
Priority Claims (1)
CN 2017 1 0428920 · Jun 8, 2017 · national
Continuity (1)
Related Publication 20180359418A1 · Dec 13, 2018