Method and apparatus for constructing map

1. A method for constructing a map, comprising:

performing semantic segmentation on each of panoramic image frames from a panoramic image sequence of a target area, and deleting a preset segmentation target from each of the panoramic image frames to obtain preprocessed panoramic image frames;

determining pose information of each panoramic image frame from the preprocessed panoramic image frames, and determining a perspective image sequence from each panoramic image frame;

determining a feature track corresponding to the panoramic image sequence based on perspective image sequences corresponding to adjacent panoramic image frames; and

constructing a visual map of the target area based on the feature track and the pose information of each panoramic image frame.

2. The method of claim 1 , wherein the feature track is obtained by performing following feature point matching on the panoramic image sequence:

determining a first panoramic image frame in the panoramic image sequence as a target frame, and determining a preset number of panoramic image frames following the target frame in the panoramic image sequence as a matching frame sequence, the preset number being greater than 1;

reading a perspective image sequence corresponding to the target frame and perspective image sequences corresponding to respective matching frames in the matching frame sequence into a preset sliding window respectively, and obtaining a partial feature track based on a result of performing inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames;

clearing the sliding window in response to completion of inter-frame feature point matching between the target frame and the respective matching frames, and deleting the target frame from the panoramic image sequence to obtain an updated panoramic image sequence; and

performing the feature point matching on the updated panoramic image sequence until a last panoramic image frame in the panoramic image sequence has been used as a matching frame to complete the inter-frame feature point matching with the target frame to obtain the feature track.

3. The method of claim 2 , wherein the obtaining a partial feature track based on a result of inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames comprises:

taking the result of performing inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames as the partial feature track.

4. The method of claim 2 , wherein the obtaining a partial feature track based on a result of inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames comprises:

using the result of performing inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames as a first matching result;

performing inter-frame feature point matching between each perspective image in the perspective image sequence corresponding to the target frame and a perspective image sequence corresponding to a matching frame closest to the target frame to obtain a second matching result; and

obtaining the partial feature track based on the first matching result and the second matching result.

5. The method of claim 1 , wherein the method further comprises performing following operations on the visual map:

performing a triangulation on feature points in the visual map to determine three-dimensional coordinates of the feature points;

performing a global beam adjustment optimization based on a visual constraint and a relative pose constraint on the feature points and the pose information to obtain updated feature points and pose information;

performing a triangulation based on Random Sample Consensus (RANSAC) on the updated feature points, and deleting a mismatching point in the feature points to obtain second time updated feature points;

performing the global beam adjustment optimization based on the visual constraint and the relative pose constraint on the updated pose information and the second time updated feature points to obtain second time updated pose information and third time updated feature points; and

determining an updated visual map based on the second time updated pose information and the third time updated feature points.

6. An electronic device, comprising:

one or more processors; and

a memory storing one or more programs thereon,

wherein the one or more programs when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:

performing semantic segmentation on each of panoramic image frames from a panoramic image sequence of a target area, and deleting a preset segmentation target from each of the panoramic image frames to obtain preprocessed panoramic image frames;

determining pose information of each panoramic image frame from the preprocessed panoramic image frames, and determining a perspective image sequence from each panoramic image frame;

determining a feature track corresponding to the panoramic image sequence based on perspective image sequences corresponding to adjacent panoramic image frames; and

constructing a visual map of the target area based on the feature track and the pose information of each panoramic image frame.

7. The electronic device of claim 6 , wherein the feature track is obtained by performing following feature point matching on the panoramic image sequence:

determining a first panoramic image frame in the panoramic image sequence as a target frame, and determining a preset number of panoramic image frames following the target frame in the panoramic image sequence as a matching frame sequence, the preset number being greater than 1;

reading a perspective image sequence corresponding to the target frame and perspective image sequences corresponding to respective matching frames in the matching frame sequence into a preset sliding window respectively, and obtaining a partial feature track based on a result of performing inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames;

clearing the sliding window in response to completion of inter-frame feature point matching between the target frame and the respective matching frames, and deleting the target frame from the panoramic image sequence to obtain an updated panoramic image sequence; and

performing the feature point matching on the updated panoramic image sequence until a last panoramic image frame in the panoramic image sequence has been used as a matching frame to complete the inter-frame feature point matching with the target frame to obtain the feature track.

8. The electronic device of claim 7 , wherein the obtaining a partial feature track based on a result of inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames comprises:

taking the result of performing inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames as the partial feature track.

9. The electronic device of claim 7 , wherein the obtaining a partial feature track based on a result of inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames comprises:

using the result of performing inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames as a first matching result;

performing inter-frame feature point matching between each perspective image in the perspective image sequence corresponding to the target frame and a perspective image sequence corresponding to a matching frame closest to the target frame to obtain a second matching result; and

obtaining the partial feature track based on the first matching result and the second matching result.

10. The electronic device of claim 6 , wherein the operations further comprise performing following operations on the visual map:

performing a triangulation on feature points in the visual map to determine three-dimensional coordinates of the feature points;

performing a global beam adjustment optimization based on a visual constraint and a relative pose constraint on the feature points and the pose information to obtain updated feature points and pose information;

performing a triangulation based on Random Sample Consensus (RANSAC) on the updated feature points, and deleting a mismatching point in the feature points to obtain second time updated feature points;

performing the global beam adjustment optimization based on the visual constraint and the relative pose constraint on the updated pose information and the second time updated feature points to obtain second time updated pose information and third time updated feature points; and

determining an updated visual map based on the second time updated pose information and the third time updated feature points.

11. A non-transitory computer readable medium storing a computer program thereon, wherein the program, when executed by a processor, cause the processor to perform operations, the operations comprising:

performing semantic segmentation on each of panoramic image frames from a panoramic image sequence of a target area, and deleting a preset segmentation target from each of the panoramic image frames to obtain preprocessed panoramic image frames;

determining pose information of each panoramic image frame from the preprocessed panoramic image frames, and determining a perspective image sequence from each panoramic image frame;

determining a feature track corresponding to the panoramic image sequence based on perspective image sequences corresponding to adjacent panoramic image frames; and

constructing a visual map of the target area based on the feature track and the pose information of each panoramic image frame.

12. The non-transitory computer readable medium of claim 11 , wherein the feature track is obtained by performing following feature point matching on the panoramic image sequence:

determining a first panoramic image frame in the panoramic image sequence as a target frame, and determining a preset number of panoramic image frames following the target frame in the panoramic image sequence as a matching frame sequence, the preset number being greater than 1;

reading a perspective image sequence corresponding to the target frame and perspective image sequences corresponding to respective matching frames in the matching frame sequence into a preset sliding window respectively, and obtaining a partial feature track based on a result of performing inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames;

clearing the sliding window in response to completion of inter-frame feature point matching between the target frame and the respective matching frames, and deleting the target frame from the panoramic image sequence to obtain an updated panoramic image sequence; and

performing the feature point matching on the updated panoramic image sequence until a last panoramic image frame in the panoramic image sequence has been used as a matching frame to complete the inter-frame feature point matching with the target frame to obtain the feature track.

13. The non-transitory computer readable medium of claim 12 , wherein the obtaining a partial feature track based on a result of inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames comprises:

taking the result of performing inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames as the partial feature track.

14. The non-transitory computer readable medium of claim 11 , wherein the obtaining a partial feature track based on a result of inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames comprises:

using the result of performing inter-frame feature point matching in the sliding window between the target frame and the perspective image sequences corresponding to the respective matching frames as a first matching result;

performing inter-frame feature point matching between each perspective image in the perspective image sequence corresponding to the target frame and a perspective image sequence corresponding to a matching frame closest to the target frame to obtain a second matching result; and

obtaining the partial feature track based on the first matching result and the second matching result.

15. The non-transitory computer readable medium of claim 11 , wherein the operations further comprise performing following operations on the visual map:

performing a triangulation on feature points in the visual map to determine three-dimensional coordinates of the feature points;

performing a global beam adjustment optimization based on a visual constraint and a relative pose constraint on the feature points and the pose information to obtain updated feature points and pose information;

performing a triangulation based on Random Sample Consensus (RANSAC) on the updated feature points, and deleting a mismatching point in the feature points to obtain second time updated feature points;

performing the global beam adjustment optimization based on the visual constraint and the relative pose constraint on the updated pose information and the second time updated feature points to obtain second time updated pose information and third time updated feature points; and

determining an updated visual map based on the second time updated pose information and the third time updated feature points.