IP Library Granted Patent US 10,489,885
Granted Patent B2
US 10,489,885 · App. 15/921,646 · Granted Nov 26, 2019

System and method for stitching images

Inventors: Dajun Ding (Beijing, CN); Lili Zhao (Beijing, CN)
Assignee: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
G06T3/4038G06K9/3233G06K9/6202G06K9/6212
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,489,885
App. No.
15/921,646
Granted
Nov 26, 2019
Kind
B2
Abstract

An image stitching method includes: obtaining a first and a second images; selecting a first and a second search regions from the first and second images, respectively, the first and second search regions at least partially overlapping with each other along a first direction; dividing each of the first and second search regions into a plurality of equally-sized pixel blocks; moving the first and second images toward each other along the first direction using the pixel blocks as a step size, until each of the first and second search regions has been traversed; after each movement, determining a pixel difference value representing a difference between all overlapping pixel blocks of the first search region and the second search region; determining an optimal block-matching position for the first and second images based on the pixel difference value; and stitching the first and second images according to the optimal block-matching position.

Claims (129)

1. An image stitching method, comprising:

obtaining a first source image and a second source image;

selecting a first search region from the first source image and a second search region from the second source image, the first search region and second search region at least partially overlapping with each other along a first direction;

dividing each of the first and second search regions into a plurality of pixel blocks, the pixel blocks in the first and second search regions having equal sizes;

moving the first source image and the second source image toward each other along the first direction using the pixel blocks as a step size, until each of the first and second search regions has been traversed;

after each movement, determining a pixel difference value representing a difference between all overlapping pixel blocks of the first search region and the second search region;

determining an optimal block-matching position for the first and second source images based on the pixel difference value; and

stitching the first and second source images according to the block-matching optimal position.

2. The method according to claim 1 , further comprising:

moving the first source image and the second source image toward each other along a row direction of the pixel blocks;

after each movement, determining the pixel difference value by averaging block pixel difference values of all overlapping pixel blocks, each of the block pixel difference value being a difference between average pixel values of two overlapping pixel blocks.

3. The method according to claim 2 , wherein when the first and second search regions have equal sizes, the following formula is used to compute the average value of the block pixel difference values:

Vb

=

i

=

1

M

j

=

1

N

(

Lij

-

Rij

)

×

(

Lij

-

Rij

)

n

×

M

;

wherein Vb is the average value of the block pixel difference values, M is the number of rows of pixel blocks in the search regions, N is the number of columns of pixel blocks in the search regions, Lij is the average value of all pixel values in the pixel block at row i and column j of the first source image, Rij is the average value of all pixel values in the pixel block at row i and column j of the second source image, and n is the movement as measured by the number of pixel block columns.

4. The method according to claim 2 , wherein the selection of the optimal block-matching position for the first source image and the second source image based on the pixel difference value comprises:

when the average value of the block pixel difference values is at its minimum, determining the relative position of the first source image and second source image as the block-matching optimal position.

5. The method according to claim 2 , wherein the stitching of the first source image and the second source image according to the optimal block-matching position comprises:

moving the first source image and the second source image toward each other along the row direction with the pixel as the step size in the block-matching optimal position, and computing the difference value of the pixel values of all overlapping pixels after each movement;

when the difference value of the pixel values is at its minimum, determining the relative position of the first source image and second source image as the pixel-matching optimal position; and

stitching the first source image and the second source image according to the pixel-matching optimal position.

6. The method according to claim 5 , wherein the following formula is used to compute the difference value of the pixel values:

Vp=Σ x=1 H Σ y=1 L ( Lxy−Rxy )×( Lxy−Rxy );

wherein Vp is the difference value of the pixel values; H is the number of rows of all overlapping pixels in the block-matching optimal position, L is the number of columns of all overlapping pixels in the block-matching optimal position, Lxy is the pixel value of the pixel at row x and column y of the first source image, and Rxy is the pixel value of the pixel at row x and column y of the second source image.

7. The method according to claim 1 , wherein after the obtainment of the first source image and the second source image to be stitched, the method further comprises:

converting the first source image and second source image into images in the HSV format, the pixel difference value being the hue difference value.

8. The method according to claim 5 , wherein after the obtainment of the first source image and the second source image to be stitched, the method further comprises:

computing the hue difference value and the gradient difference value of all pixels in the first source image and the second source image; and

computing the sum value of the hue difference value and gradient difference value of all pixels, and selecting the pixels corresponding to the minimum sum value as the seam position.

9. The method according to claim 8 , wherein the stitching of the first source image and the second source image according to the optimal block-matching position comprises:

stitching the first source image and the second source image in the pixel-matching optimal position according to the seam position.

10. The method according to claim 9 , wherein the stitching of the first source image and the second source image in the pixel-matching optimal position according to the seam position comprises:

in the regions between the seam position and the pixel-matching optimal position, selecting pixels corresponding to the first source image and the second source image, respectively, for the stitching; and

along the seam position, using linear blending to stitch pixels corresponding to the first source image and the second source image, in order to complete the stitching of the first source image and the second source image.

11. The method according to claim 8 , wherein before the stitching of the first source image and the second source image according to the block-matching optimal position, the method further comprises:

converting the format of the first source image and the second source image to YCrCb;

after the format conversion, computing the average of luminance values in the first source image and the second source image at the seam position with samples;

using the luminance average value to compute a plurality of exposure factors in the pixel-matching optimal position between the first source image and the second source image; and

using the plurality of exposure factors for exposure compensation for the first source image and the second source image, in order to balance the change in luminance in the pixel-matching optimal position between the first source image and the second source image.

12. The method according to claim 8 , wherein before the computation of the hue difference value and the gradient difference value of all overlapping pixels in the first source image and the second source image in the pixel-matching optimal position, the method further comprises:

performing a downsampling operation on the first source image and the second source image.

13. The method according to claim 1 , wherein after the obtainment of the first source image and the second source image to be stitched, the method further comprises:

performing a cylindrical projection on the first source image and the second source image in order to obtain the projected images of the first source image and the second source image.

14. The method according to claim 13 , wherein the performance of a cylindrical projection on the first source image and the second source image comprises:

performing a cylindrical projection on the first source image and the second source image based on the pre-obtained shooting angles for the first source image and the second source image.

15. An image stitching apparatus, comprising:

a memory storing instructions; and

a processor configured to execute the instructions to:

obtain a first source image and a second source image;

select a first search region from the first source image and a second search region from the second source image, the first search region and second search region at least partially overlapping with each other along a first direction;

divide each of the first and second search regions into a plurality of pixel blocks, the pixel blocks in the first and second search regions having equal sizes;

move the first source image and the second source image toward each other along the first direction using the pixel blocks as a step size, until each of the first and second search regions has been traversed;

after each movement, determine a pixel difference value representing a difference between all overlapping pixel blocks of the first search region and the second search region;

determine an optimal block-matching position for the first and second source images based on the pixel difference value; and

stitch the first and second source images according to the block-matching optimal position.

16. The apparatus according to claim 15 , wherein the processor is further configured to execute the instructions to:

move the first source image and the second source image toward each other along a row direction of the pixel blocks;

after each movement, determine the pixel difference value by averaging block pixel difference values of all overlapping pixel blocks, each of the block pixel difference value being a difference between average pixel values of two overlapping pixel blocks.

17. The apparatus according to claim 16 , wherein when the search regions in the first source image and the second source image are the same in size, the processor is further configured to execute the instructions to use the following formula to compute the average value of the block pixel difference values:

Vb

=

i

=

1

M

j

=

1

N

(

Lij

-

Rij

)

×

(

Lij

-

Rij

)

n

×

M

;

wherein Vb is the average value of the block pixel difference values, M is the number of rows of pixel blocks in the search regions, N is the number of columns of pixel blocks in the search regions, Lij is the average value of all pixel values in the pixel block at row i and column j of the first source image, Rij is the average value of all pixel values in the pixel block at row i and column j of the second source image, and n is the movement as measured by the number of pixel block columns.

18. The apparatus according to claim 16 , wherein when the average value of the block pixel difference values is at its minimum, the processor is further configured to execute the instructions to take the relative position of the first source image and second source image as the block-matching optimal position.

19. The apparatus according to claim 16 , wherein the processor is further configured to execute the instructions to:

move the first source image and the second source image toward each other along the row direction with the pixel as the step size in the block-matching optimal position, and compute the difference value of the pixel values of all overlapping pixels after each movement;

take the relative position of the first source image and second source image as the pixel-matching optimal position when the difference value of the pixel values is at its minimum; and

stitch the first source image and the second source image according to the pixel-matching optimal position.

20. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform an image stitching method comprising:

obtaining a first source image and a second source image;

selecting a first search region from the first source image and a second search region from the second source image, the first search region and second search region at least partially overlapping with each other along a first direction;

dividing each of the first and second search regions into a plurality of pixel blocks, the pixel blocks in the first and second search regions having equal sizes;

moving the first source image and the second source image toward each other along the first direction using the pixel blocks as a step size, until each of the first and second search regions has been traversed;

after each movement, determining a pixel difference value representing a difference between all overlapping pixel blocks of the first search region and the second search region;

determining an optimal block-matching position for the first and second source images based on the pixel difference value; and

stitching the first and second source images according to the block-matching optimal position.

Assignments (4)
RELEASE OF SECURITY INTEREST Recorded Nov 5, 2024
From: EAST WEST BANK
To: KAMI VISION INCORPORATED
Reel/Frame 070792/0551 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Mar 28, 2022
From: KAMI VISION INCORPORATED
To: EAST WEST BANK
Reel/Frame 059512/0101 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2022
From: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
To: KAMI VISION INC.
Reel/Frame 059275/0652 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 15, 2018
From: DING, DAJUN; ZHAO, LILI
To: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
Reel/Frame 045229/0148 →
Priority Claims (1)
CN 2017 1 0149873 · Mar 14, 2017 · national
Continuity (1)
Related Publication 20180268521A1 · Sep 20, 2018