Method and device for fusing panoramic video images

1. A method for fusion of a panoramic video, comprising:

acquiring a panoramic video image, wherein the panoramic video image is formed by stitching at least two cuboid three-dimensional images;

extracting a moving target in a target image from a cuboid three-dimensional image corresponding to a stitching area in the at least two cuboid three-dimensional images when it is detected that the moving target is displayed abnormally at a stitching area in the panoramic video image, by comparing the panoramic video image with the at least two cuboid three-dimensional images;

performing image fusion processing on the panoramic video image and the moving tar-get at the stitching area to form a panoramic video image incorporated with the moving target;

wherein, performing image fusion processing on the panoramic video image and the moving target comprises:

inserting the moving target into a to-be-fused area of the panoramic video image;

determining color values of pixels in the to-be-fused area of the panoramic video image in which the moving target is inserted by using a Poisson fusion algorithm;

performing image reconstruction on the to-be-fused area according to the determined color values of the pixels;

wherein, determining color values of pixels in the to-be-fused area of the panoramic video image in which the moving target is inserted by using a Poisson fusion algorithm comprises:

determining a divergence of a color value of the panoramic video image in which the moving target is inserted; and

generating a sparse matrix according to a boundary constraint condition of the to-be-fused area of the panoramic video image;

constructing a following Poisson equation:

Ax=B   (1)

wherein, A is the sparse matrix, and B is the divergence of a color value of the panoramic video image inserted with the moving target, and x is the color value of a pixel to be solved;

solving the Poisson equation to determine the color values of the pixels in the to-be-fused area.

2. The method of claim 1 , wherein, extracting a moving target from the at least two cuboid three-dimensional images comprises:

determining a target image from a cuboid three-dimensional image corresponding to a stitching area;

determining pixels corresponding to a moving foreground image in the target image;

identifying a contour of the moving target according to the determined pixels;

acquiring mask information of an image area corresponding to the contour of the moving target;

extracting the moving target according to the mask information.

3. The method of claim 2 , wherein, determining a target image from the cuboid three-dimensional image corresponding to the stitching area comprises:

determining a cuboid three-dimensional image corresponding to the stitching area in which the moving target is located;

taking a ground image of the determined cuboid three-dimensional image as a target image, wherein the ground image is an image only of ground on which a moving target is walking.

4. The method of claim 2 , wherein, determining pixels corresponding to the moving foreground image in the target image comprises:

performing Gaussian background modeling detection on each pixel in the selected target image by using a Gaussian background modeling algorithm to determine the pixels corresponding to the moving foreground image.

5. An apparatus for fusion of a panoramic video, comprising:

an acquiring unit, configured for acquiring a panoramic video image, wherein the panoramic video image is formed by at least two cuboid three-dimensional images;

an extracting unit, configured for extracting a moving target from the at least two cuboid three-dimensional images;

a fusing unit, configured for performing image fusion processing on the panoramic video image and the moving target at the stitching area to form a panoramic video image incorporated with the moving target;

wherein the extracting unit is configured for:

extracting a moving target in a target image from a cuboid three-dimensional image corresponding to a stitching area when it is detected that the moving target is displayed abnormally at a stitching area in the panoramic video image, by comparing the panoramic video image with the at least two cuboid three-dimensional images;

wherein the fusing unit is configured for:

inserting the moving target into a to-be-fused area of the panoramic video image;

determining color values of pixels in the to-be-fused area of the panoramic video image in which the moving target is inserted by using a Poisson fusion algorithm;

performing image reconstruction on the to-be-fused area according to the determined color values of the pixels;

wherein, the fusing unit, when determining color values of pixels in the to-be-fused area of the panoramic video image in which the moving target is inserted by using a Poisson fusion algorithm, is configured for:

determining a divergence of a color value of the panoramic video image in which the moving target is inserted; and

generating a sparse matrix according to a boundary constraint condition of the to-be-fused area of the panoramic video image;

constructing a following Poisson equation:

Ax=B   (1)

wherein, A is the sparse matrix, and B is the divergence of a color value of the panoramic video image inserted with the moving target, and x is the color value of a pixel to be solved;

solving the Poisson equation to determine the color values of the pixels in the to-be-fused area.

6. The apparatus of claim 5 , wherein the extracting unit is configured for:

determining a target image from a cuboid three-dimensional image corresponding to a stitching area;

determining pixels corresponding to a moving foreground image in the target image;

identifying a contour of the moving target according to the determined pixels;

acquiring mask information of an image area corresponding to the contour of the moving target;

extracting the moving target according to the mask information.

7. The apparatus of claim 6 , wherein, the extracting unit, when determining a target image from the cuboid three-dimensional image corresponding to the stitching area, is configured for:

determining a cuboid three-dimensional image corresponding to the stitching area in which the moving target is located;

taking a ground image of the determined cuboid three-dimensional image as a target image, wherein the ground image is an image only of ground on which a moving target is walking.

8. The apparatus of claim 6 , wherein, the extracting unit, when determining pixels corresponding to the moving foreground image in the target image, is configured for:

performing Gaussian background modeling detection on each pixel in the selected target image by using a Gaussian background modeling algorithm to determine the pixels corresponding to the moving foreground image.

9. An electronic device, comprising a processor and a memory, wherein the memory is configured for storing executable code, and the processor is configured for performing the method of claim 1 by reading the executable code stored in the memory.

10. A non-transitory storage medium, which is configured for storing executable code, wherein the executable code is configured to perform the method for fusion of a panoramic video according to claim 1 when executed.

11. A system for fusion of a panoramic video, comprising at least two panoramic cameras and an image processor, wherein:

the at least two panoramic cameras are configured for respectively capturing a two-dimensional image of each local space in a preset scene;

the image processor is configured for determining, for each of the local spaces, a cuboid three-dimensional image of the local space according to the two-dimensional image of the local space and a cuboid three-dimensional model corresponding to the local space;

a panoramic video image is formed by stitching at least two cuboid three-dimensional images;

extracting a moving target in a target image from a cuboid three-dimensional image corresponding to a stitching area in the at least two cuboid three-dimensional images when it is detected that the moving target is displayed abnormally at a stitching area in the panoramic video image, by comparing the panoramic video image with the at least two cuboid three-dimensional images;

performing image fusion processing on the panoramic video image and the moving target at the stitching area to form a panoramic video image incorporated with the moving target;

wherein, performing image fusion processing on the panoramic video image and the moving target comprises:

inserting the moving target into a to-be-fused area of the panoramic video image;

determining color values of pixels in the to-be-fused area of the panoramic video image in which the moving target is inserted by using a Poisson fusion algorithm;

performing image reconstruction on the to-be-fused area according to the determined color values of the pixels;

wherein, determining color values of pixels in the to-be-fused area of the panoramic video image in which the moving target is inserted by using a Poisson fusion algorithm comprises:

determining a divergence of a color value of the panoramic video image in which the moving target is inserted; and

generating a sparse matrix according to a boundary constraint condition of the to-be-fused area of the panoramic video image;

constructing a following Poisson equation:

Ax=B   (1)

wherein, A is the sparse matrix, and B is the divergence of a color value of the panoramic video image inserted with the moving target, and x is the color value of a pixel to be solved;

solving the Poisson equation to determine the color values of the pixels in the to-be-fused area.