Image reconstruction method and device
Embodiments of the present application provides an image reconstruction method and device. The method comprises: when a long-focus image set acquisition condition is met, driving the pan-tilt platform of a binocular long-focus and short-focus pan-tilt camera to rotate throughout a field-of-view range, obtaining one long-focus image every preset horizontal and/or vertical angle, and constituting a first long-focus image set with all long-focus images acquired throughout the field-of-view range; receiving a zoom request, and performing interpolating on the first short-focus image currently acquired to obtain a second short-focus image that meets the zoom request; downsampling each long-focus image in the first long-focus image set to obtain a second long-focus image set; dividing the second short-focus image into blocks, searching, for each of the divided blocks, all long-focus images in the second long-focus image set for a matching block; fusing each of the divided blocks in the second short-focus image with the matching block for this divided block to obtain a reconstructed short-focus image. The embodiments of the present application not only retain the advantage of a large field-of-view of the short-focus image, but also improves the definition.
1. An image reconstruction method, comprising:
when a preset long-focus image set acquisition condition is met, driving a pan-tilt platform of a binocular long-focus and short-focus pan-tilt camera to rotate throughout a field-of-view range, obtaining one long-focus image every preset horizontal and/or vertical angle, and constituting a first long-focus image set with all long-focus images acquired throughout the field-of-view range, wherein adjacent long-focus images overlap partially;
receiving a zoom request, and performing interpolating on a first short-focus image currently acquired by the binocular long-focus and short-focus pan-tilt camera to obtain a second short-focus image that meets the zoom request;
downsampling each long-focus image in the first long-focus image set to obtain a second long-focus image set, wherein a target in each long-focus image in the second long-focus image set and the same target in the second short-focus image are matched in size;
dividing the second short-focus image into blocks, and searching, for each of the divided blocks, all long-focus images in the second long-focus image set for a matching block;
fusing each of the divided blocks in the second short-focus image with the matching block for this divided block to obtain a reconstructed short-focus image;
wherein the binocular long-focus and short-focus pan-tilt camera is composed of a short-focus camera and a long-focus camera, relative positions of the short-focus camera and the long-focus camera are fixed, and the short-focus camera and the long-focus camera rotate simultaneously with the pan-tilt platform.
2. The method of claim 1 , wherein, fusing each of the divided blocks in the second short-focus image with the matching block for this divided block comprises:
extracting a high-frequency component of the matching block for each of the divided blocks, and fusing each of the divided blocks with the high-frequency component of the matching block for this divided block.
3. The method of claim 2 , wherein, fusing each of the divided blocks in the second short-focus image with the high-frequency component of the matching block for this divided block comprises:
assigning a fusion weight for each matching block according to a principle that the larger a similarity between the matching block and its corresponding divided block is, the larger the fusion weight of the matching block is;
multiplying the high-frequency component of each matching block by the corresponding fusion weight to obtain a result and then superimposing the result with the corresponding divided block, to obtain the reconstructed short-focus image.
4. The method of claim 3 , wherein, assigning the fusion weight for each matching block according to the principle that the larger a similarity between the matching block and its corresponding divided block is, the larger the fusion weight of the matching block is comprises:
calculating
W
k
=
(
E
k
E
min
)
-
α
wherein W K is a fusion weight of a matching block for a kth divided block of the second short-focus image,
E
k
=
1
S
k
,
S K is a similarity between the kth divided block of the second short-focus image and the matching block,
E
min
=
1
S
max
,
S max is a maximum similarity among similarities between all divided blocks of the second short-focus image and the matching blocks for the divided blocks, and α is a preset value.
5. The method of claim 1 , wherein, searching, for each of the divided blocks, all long-focus images in the second long-focus image set for the matching block comprises:
searching, for each of the divided blocks of the second short-focus image, all the long-focus images in the second long-focus image set for a matching block that matches the divided block in position according to an angle of the pan-tilt platform when the first short-focus image is acquired, an angle of the pan-tilt platform when each long-focus image in the first long-focus image set is acquired and a field-of-view difference between a short-focus lens and a long-focus lens; or
searching, for each of the divided blocks, all the long-focus images in the second long-focus image set for a matching block having a highest image similarity with the divided block by using a preset template matching algorithm.
6. The method of claim 2 , wherein, extracting the high-frequency component of the matching block for each of the divided blocks in the second short-focus image comprises:
downsampling the long-focus image in which the matching block is located in the second long-focus image set such that a resolution of the downsampled image is the same as that of the first short-focus image, then performing interpolating on the downsampled image such that a resolution of the interpolated image is equal to that of the second short-focus image, and subtracting a corresponding block of the matching block in the interpolated image from the matching block to obtain the high-frequency component of the matching block.
7. The method of claim 1 , wherein, the preset long-focus image set acquisition condition is met in a case comprising:
when the binocular long-focus and short-focus pan-tilt camera is started, or, when a difference between a photometric value of a currently acquired short-focus image and a photometric value of a long-focus image in a last acquired long-focus image set is greater than a preset first threshold, or when an image similarity between divided blocks at same positions in the currently acquired short-focus image and in a long-focus image in the last acquired long-focus image set is less than a preset second threshold.
8. The method of claim 1 , wherein, after the preset long-focus image set acquisition condition is met and before the pan-tilt platform of the binocular long-focus and short-focus pan-tilt camera is driven to rotate throughout the field-of-view range, the method further comprises:
determining whether there is no alarm target in a currently acquired short-focus image and there is currently no image preview requirement,
when it is determined that there is no alarm target in the currently acquired short-focus image and there is currently no image preview requirement, driving the pan-tilt platform of the binocular long-focus and short-focus pan-tilt camera to rotate throughout the field-of-view range, obtaining one long-focus image every preset horizontal and/or vertical angle, and constituting the first long-focus image set with all long-focus images acquired throughout the field-of-view range; and
when it is determined that there is an alarm target in the currently acquired short-focus image or there is currently an image preview requirement, waiting for a preset time, and then returning to the action of determining whether there is no alarm target in the currently acquired short-focus image and there is currently no image preview requirement.
9. An image reconstruction device, which comprises a memory and a processor that can access the memory having stored therein instructions which, when executed by the processor, cause the processor to implement steps of the method of claim 1 .
10. A binocular long-focus and short-focus pan-tilt camera, which comprises a short-focus camera, a long-focus camera and a pan-tilt platform, wherein relative positions of the short-focus camera and the long-focus camera are fixed, and the short-focus camera and the long-focus camera rotate simultaneously with the pan-tilt platform; and
each of the short-focus camera and the long-focus camera has an interface to communicate with the image reconstruction device of claim 9 , and the short-focus camera and the long-focus camera send an original short-focus image and an original long-focus image respectively acquired by the short-focus camera and the long-focus camera to the image reconstruction device through the respective communication interfaces.
11. A non-transitory computer readable storage medium having stored therein a computer program which, when executed by a processor, causes the processor to perform steps of the method of claim 1 .