Depth map generation method, and device and storage medium
Provided are a depth map generation method, a device and a storage medium, which belong to the technical field of image processing. The method includes: generating, according to a first spherical image acquired by a first fisheye lens and a second spherical image acquired by a second fisheye lens, a first disparity map of a spatial region where a terminal device is located; generating a second disparity map of the spatial region according to depth information of the spatial region acquired by an active depth sensor; and generating a target depth map of the spatial region according to the first disparity map and the second disparity map.
1. A depth map generation method, applied to a terminal device, wherein the terminal device comprises a first fisheye lens, a second fisheye lens and an active depth sensor, and the method comprises:
generating, according to a first spherical image acquired by the first fisheye lens and a second spherical image acquired by the second fisheye lens, a first disparity map of a spatial region where the terminal device is located;
generating a second disparity map of the spatial region according to depth information of the spatial region acquired by the active depth sensor; and
generating a target depth map of the spatial region according to the first disparity map and the second disparity map;
wherein the generating, according to the first spherical image acquired by the first fisheye lens and the second spherical image acquired by the second fisheye lens, the first disparity map of the spatial region where the terminal device is located comprises:
fusing the first spherical image and the second spherical image to obtain a target planar image having a preset field of view;
converting the first spherical image into a first planar image, and converting the second spherical image into a second planar image;
determining, according to the first planar image and the second planar image, feature point matching pairs corresponding to a plurality of spatial points in the spatial region where the terminal device is located to obtain a plurality of feature point matching pairs; and
generating, according to the plurality of feature point matching pairs and the target planar image, the first disparity map of the spatial region where the terminal device is located.
2. The depth map generation method according to claim 1 , wherein the fusing the first spherical image and the second spherical image to obtain the target planar image having a preset field of view comprises:
calibrating the first spherical image and the second spherical image; and
fusing the first calibrated spherical image and the second calibrated spherical image to obtain the target planar image having a preset field of view.
3. The depth map generation method according to claim 2 , wherein the fusing the first calibrated spherical image and the second calibrated spherical image to obtain the target planar image having a preset field of view comprises:
converting the first calibrated spherical image into a first three-dimensional image, the first three-dimensional image comprising a forward mapping image, a leftward mapping image, a rightward mapping image, an upward mapping image and a downward mapping image of the first calibrated spherical image;
converting the second calibrated spherical image into a second three-dimensional image, the second three-dimensional image comprising a forward mapping image, a leftward mapping image, a rightward mapping image, an upward mapping image and a downward mapping image of the second calibrated spherical image; and
fusing the first three-dimensional image and the second three-dimensional image to obtain the target planar image having a preset field of view.
4. The depth map generation method according to claim 3 , wherein the fusing the first three-dimensional image and the second three-dimensional image to obtain the target planar image having a preset field of view comprises:
splicing the forward mapping image, the leftward mapping image and the rightward mapping image of the first three-dimensional image to obtain a first image having a preset field of view;
splicing the forward mapping image, the leftward mapping image and the rightward mapping image of the second three-dimensional image to obtain a second image having a preset field of view;
fusing the first image and the second image to obtain a first fused image, and fusing the upward mapping image of the first three-dimensional image and the upward mapping image of the second three-dimensional image to obtain a second fused image;
fusing the downward mapping image of the first three-dimensional image and the downward mapping image of the second three-dimensional image to obtain a third fused image; and
splicing the first fused image, the second fused image and the third fused image to obtain the target planar image having a preset field of view.
5. The depth map generation method according to claim 2 , wherein the calibrating the first spherical image and the second spherical image comprises:
converting the first spherical image into a third three-dimensional image, and converting the second spherical image into a fourth three-dimensional image;
determining, according to the third three-dimensional image and the fourth three-dimensional image, feature point matching pairs corresponding to a plurality of spatial points in the spatial region where the terminal device is located to obtain the plurality of feature point matching pairs;
removing abnormal feature point matching pairs from the plurality of feature point matching pairs to obtain a plurality of target feature point matching pairs; and
calibrating the first spherical image and the second spherical image according to the plurality of target feature point matching pairs.
6. The depth map generation method according to claim 1 , wherein the converting the first spherical image into the first planar image comprises:
converting the first spherical image into a three-dimensional image, the three-dimensional image comprising a forward mapping image, a leftward mapping image, a rightward mapping image, an upward mapping image and a downward mapping image of the first spherical image;
splicing the forward mapping image, the leftward mapping image and the rightward mapping image of the three-dimensional image to obtain a first image; and
splicing the upward mapping image and the downward mapping image of the three-dimensional image and the first image to obtain the first planar image.
7. The depth map generation method according to claim 1 , wherein after the fusing the first spherical image and the second spherical image to obtain a target planar image having a preset field of view, the method further comprises:
acquiring a historical planar image, the historical planar image being determined according to a first spherical image and a second spherical image acquired at a previous moment; and
generating, according to the target planar image and the historical planar image, the first disparity map of the spatial region where the terminal device is located.
8. A terminal device, comprising a first fisheye lens, a second fisheye lens, an active depth sensor, a processor, a memory, a computer program stored on the memory and capable of being executed by the processor, and a data bus configured to implement communication between the processor and the memory, wherein when the computer program is executed by the processor, cause the processor to implement steps of the depth map generation method as claimed in claim 1 .
9. A Non-transitory storage medium, used for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs are capable of being executed by one or more processors, so as to implement steps of the depth map generation method as claimed in claim 1 .
10. A terminal device, comprising a first fisheye lens, a second fisheye lens, an active depth sensor, a processor, a memory, a computer program stored on the memory and capable of being executed by the processor, and a data bus configured to implement communication between the processor and the memory, wherein when the computer program is executed by the processor, cause the processor to implement steps of the depth map generation method as claimed in claim 2 .
11. A terminal device, comprising a first fisheye lens, a second fisheye lens, an active depth sensor, a processor, a memory, a computer program stored on the memory and capable of being executed by the processor, and a data bus configured to implement communication between the processor and the memory, wherein when the computer program is executed by the processor, cause the processor to implement steps of the depth map generation method as claimed in claim 3 .
12. A terminal device, comprising a first fisheye lens, a second fisheye lens, an active depth sensor, a processor, a memory, a computer program stored on the memory and capable of being executed by the processor, and a data bus configured to implement communication between the processor and the memory, wherein when the computer program is executed by the processor, cause the processor to implement steps of the depth map generation method as claimed in claim 4 .
13. A terminal device, comprising a first fisheye lens, a second fisheye lens, an active depth sensor, a processor, a memory, a computer program stored on the memory and capable of being executed by the processor, and a data bus configured to implement communication between the processor and the memory, wherein when the computer program is executed by the processor, cause the processor to implement steps of the depth map generation method as claimed in claim 5 .
14. A terminal device, comprising a first fisheye lens, a second fisheye lens, an active depth sensor, a processor, a memory, a computer program stored on the memory and capable of being executed by the processor, and a data bus configured to implement communication between the processor and the memory, wherein when the computer program is executed by the processor, cause the processor to implement steps of the depth map generation method as claimed in claim 6 .
15. A terminal device, comprising a first fisheye lens, a second fisheye lens, an active depth sensor, a processor, a memory, a computer program stored on the memory and capable of being executed by the processor, and a data bus configured to implement communication between the processor and the memory, wherein when the computer program is executed by the processor, cause the processor to implement steps of the depth map generation method as claimed in claim 7 .
16. A Non-transitory storage medium, used for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs are capable of being executed by one or more processors, so as to implement steps of the depth map generation method as claimed in claim 2 .
17. A Non-transitory storage medium, used for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs are capable of being executed by one or more processors, so as to implement steps of the depth map generation method as claimed in claim 3 .