Method and apparatus for reconstructing scene, terminal device, and storage medium
A scene reconstruction method, apparatus, terminal device, and storage medium. The method includes: acquiring a first image set matching a to-be-reconstructed scene ( 11 ); extracting a characteristic region of an image in the first image set by using a characteristic extraction algorithm ( 12 ); performing recognition on the characteristic region to obtain a scene characteristic region in the image ( 13 ); and performing three-dimensional reconstruction on the to-be-reconstructed scene according to the scene characteristic region in the image, and rendering and generating the to-be-reconstructed scene ( 14 ). Some useless and unstable characteristic regions are filtered off, and the three-dimensional reconstruction on the to-be-reconstructed scene is performed only according to the scene characteristic region associated with the to-be-reconstructed scene, thereby improving the efficiency and accuracy of reconstruction.
1. A method for reconstructing a scene, comprising:
acquiring a first image set matching a to-be-reconstructed scene;
extracting a characteristic region of an image in the first image set by using a characteristic extraction algorithm;
performing recognition on the characteristic region to obtain a scene characteristic region in the image; and
performing three-dimensional reconstruction on the to-be-reconstructed scene according to the scene characteristic region in the image, and rendering and generating the to-be-reconstructed scene,
wherein the acquiring a first image set matching a to-be-reconstructed scene comprises:
searching for a second image set related to the to-be-reconstructed scene on the Internet or a user generated content (UGC) image library by using an image recognition technology; and
deleting images not meeting a preset requirement from the second image set, and using remaining images as the first image set matching the to-be-reconstructed scene.
2. The method according to claim 1 , wherein the characteristic region comprises: at least one of a human characteristic region, an object characteristic region, and a scenery characteristic region, and before the performing three-dimensional reconstruction on the to-be-reconstructed scene according to the scene characteristic region in the image, the method further comprises:
deleting the human characteristic region, the object characteristic region, and the scenery characteristic region from the image.
3. The method according to claim 2 , wherein after the performing three-dimensional reconstruction on the to-be-reconstructed scene according to the scene characteristic region in the image, and before the rendering and generating the to-be-reconstructed scene, the method further comprises:
selecting images comprising more than a preset number of scene characteristic regions from the images as key frames;
determining a neighboring relationship between the key frames according to a spatial relationship between scenes displayed in the key frames and times when the images are captured; and
performing path planning for the to-be-reconstructed scene according to determined neighboring key frames.
4. The method according to claim 1 , wherein after the performing three-dimensional reconstruction on the to-be-reconstructed scene according to the scene characteristic region in the image, and before the rendering and generating the to-be-reconstructed scene, the method further comprises:
selecting images comprising more than a preset number of scene characteristic regions from the images as key frames;
determining a neighboring relationship between the key frames according to a spatial relationship between scenes displayed in the key frames and times when the images are captured; and
performing path planning for the to-be-reconstructed scene according to determined neighboring key frames.
5. The method according to claim 4 , wherein the performing path planning for the to-be-reconstructed scene according to determined neighboring key frames comprises:
if there is a temporal difference between the neighboring key frames, inserting, between the neighboring key frames, a transitional image within the temporal difference; and
if there is a spatial difference between the neighboring key frames, selecting an image spatially associated with the key frames as a transitional image.
6. The method according to claim 1 , wherein the rendering and generating the to-be-reconstructed scene comprises:
rendering and generating reconstructed scenes at different times according to the times when the images in the first image set are captured.
7. The method according to claim 1 , wherein after the performing three-dimensional reconstruction on the to-be-reconstructed scene according to the scene characteristic region in the image, and before the rendering and generating the to-be-reconstructed scene, the method further comprises:
selecting images comprising more than a preset number of scene characteristic regions from the images as key frames;
determining a neighboring relationship between the key frames according to a spatial relationship between scenes displayed in the key frames and times when the images are captured; and
performing path planning for the to-be-reconstructed scene according to determined neighboring key frames.
8. A terminal device for reconstructing a scene, comprising:
one or more processors; and
a memory;
wherein one or more modules are stored in the memory, and when executed by the one or more processors, the one or more modules cause the one or more processors to perform operations comprising:
acquiring a first image set matching a to-be-reconstructed scene;
extracting a characteristic region of an image in the first image set by using a characteristic extraction algorithm;
performing recognition on the characteristic region to obtain a scene characteristic region in the image; and
performing three-dimensional reconstruction on the to-be-reconstructed scene according to the scene characteristic region in the image, and rendering and generating the to-be-reconstructed scene,
wherein the acquiring a first image set matching a to-be-reconstructed scene comprises:
searching for a second image set related to the to-be-reconstructed scene on the Internet or a user generated content (UGC) image library by using an image recognition technology; and
deleting images not meeting a preset requirement from the second image set, and using remaining images as the first image set matching the to-be-reconstructed scene.
9. The terminal device according to claim 8 , wherein the characteristic region comprises: at least one of a human characteristic region, an object characteristic region, and a scenery characteristic region, and before the performing three-dimensional reconstruction on the to-be-reconstructed scene according to the scene characteristic region in the image, the operations further comprise:
deleting the human characteristic region, the object characteristic region, and the scenery characteristic region from the image.
10. The terminal device according to claim 8 , wherein after the performing three-dimensional reconstruction on the to-be-reconstructed scene according to the scene characteristic region in the image, and before the rendering and generating the to-be-reconstructed scene, the operations further comprise:
selecting images comprising more than a preset number of scene characteristic regions from the images as key frames;
determining a neighboring relationship between the key frames according to a spatial relationship between scenes displayed in the key frames and times when the images are captured; and
performing path planning for the to-be-reconstructed scene according to determined neighboring key frames.
11. The terminal device according to claim 10 , wherein the performing path planning for the to-be-reconstructed scene according to determined neighboring key frames comprises:
if there is a temporal difference between the neighboring key frames, inserting, between the neighboring key frames, a transitional image within the temporal difference; and
if there is a spatial difference between the neighboring key frames, selecting an image spatially associated with the key frames as a transitional image.
12. The terminal device according to claim 8 , wherein the rendering and generating the to-be-reconstructed scene comprises:
rendering and generating reconstructed scenes at different times according to the times when the images in the first image set are captured.
13. A non-transitory computer storage medium storing one or more modules, the one or more modules when executed by a device for executing a method of reconstructing a scene, causing the device to perform operations comprising: acquiring a first image set matching a to-be-reconstructed scene; extracting a characteristic region of an image in the first image set by using a characteristic extraction algorithm; performing recognition on the characteristic region to obtain a scene characteristic region in the image; and performing three-dimensional reconstruction on the to-be-reconstructed scene according to the scene characteristic region in the image, and rendering and generating the to-be-reconstructed scene, wherein the acquiring a first image set matching a to-be-reconstructed scene comprises: searching for a second image set related to the to-be-reconstructed scene on the Internet or a user generated content (UGC) image library by using an image recognition technology; and deleting images not meeting a preset requirement from the second image set, and using remaining images as the first image set matching the to-be-reconstructed scene.