Map optimizing method, related electronic device and non-transitory computer readable storage medium

1. A map optimizing method, applicable to an electronic device configured to store a plurality of distance values and a map, comprising:

identifying, from the map, a first map point with a first estimated coordinate and a second map point with a second estimated coordinate generated based on a first marker and a second marker, respectively;

generating, according to the first estimated coordinate and the second estimated coordinate, a plurality of virtual cameras controlled by a virtual pose and with optical axes intersected at a first intersection coordinate and a second intersection coordinate separated from each other by a first distance value of the plurality of distance values, wherein the plurality of virtual cameras provide a plurality of virtual frames indicating that the first marker and the second marker are observed at the first intersection coordinate and the second intersection coordinate, respectively; and

performing a global bundle adjustment to optimize the map, comprising adjusting the first estimated coordinate and the second estimated coordinate to reduce a sum of re-projection errors calculated according to a plurality of real keyframes and the plurality of virtual frames, wherein the electronic device is configured to perform positioning according to the optimized map.

2. The map optimizing method of claim 1 , wherein the plurality of real keyframes are captured by the electronic device at a plurality of camera poses, respectively, and the plurality of camera poses are stored in the electronic device, wherein performing the global bundle adjustment to optimize the map further comprises:

adjusting the plurality of camera poses and the virtual pose to reduce the sum of re-projection errors.

3. The map optimizing method of claim 1 , wherein performing the global bundle adjustment to optimize the map further comprises calculating the sum of re-projection errors, wherein calculating the sum of re-projection errors comprises:

projecting the first estimated coordinate onto real keyframes and virtual frames with first pixel points related to the first map point, so as to generate a plurality of first projection points;

calculating first re-projection errors between each of the first pixel points and a corresponding one of the plurality of first projection points;

projecting the second estimated coordinate onto real keyframes and virtual frames with second pixel points related to the second map point, so as to generate a plurality of second projection points;

calculating second re-projection errors between each of the second pixel points and a corresponding one of the plurality of second projection points; and

adding the first re-projection errors, the second re-projection errors and re-projection errors of other map points of the map, so as to generate the sum of re-projection errors.

4. The map optimizing method of claim 1 , wherein a number of the plurality of virtual cameras is twelve, wherein optical axes of half of the plurality of virtual cameras intersect at the first intersection coordinate, and optical axes of another half of the plurality of virtual cameras intersect at the second intersection coordinate.

5. The map optimizing method of claim 1 , wherein when the plurality of virtual cameras are generated, the first intersection coordinate, the second intersection coordinate, the first estimated coordinate and the second estimated coordinate are aligned with each other in an imaginary straight line.

6. The map optimizing method of claim 5 , wherein when the plurality of virtual cameras are generated, a distance from the first estimated coordinate to the first intersection coordinate equals a distance from the second estimated coordinate to the second intersection coordinate.

7. The map optimizing method of claim 1 , wherein the first distance value represents a real-world distance between the first marker and the second marker.

8. The map optimizing method of claim 1 , wherein each of the plurality of distance values represents a real-word distance between two markers of a plurality of markers comprising the first marker and the second marker,

wherein the electronic device is configured to generate a plurality of sets of virtual cameras, and the plurality of virtual cameras is a set of the plurality of sets of virtual cameras,

wherein each set of the plurality of sets of virtual cameras is separated into two groups kept separated from each other by a corresponding one of the plurality of distance values while performing the global bundle adjustment, so as to introduce the plurality of distance values into the global bundle adjustment.

9. An electronic device, comprising:

a storage circuit, configured to store a plurality of distance values and a map; and

a computing circuit, configured to:

identify, from the map, a first map point with a first estimated coordinate and a second map point with a second estimated coordinate generated based on a first marker and a second marker, respectively;

generate, according to the first estimated coordinate and the second estimated coordinate, a plurality of virtual cameras controlled by a virtual pose and with optical axes intersected at a first intersection coordinate and a second intersection coordinate separated from each other by a first distance value of the plurality of distance values, wherein the plurality of virtual cameras provide a plurality of virtual frames indicating that the first marker and the second marker are observed at the first intersection coordinate and the second intersection coordinate, respectively; and

perform a global bundle adjustment to optimize the map, comprising adjusting the first estimated coordinate and the second estimated coordinate to reduce a sum of re-projection errors calculated according to a plurality of real keyframes and the plurality of virtual frames, wherein the electronic device is configured to perform positioning according to the optimized map.

10. The electronic device of claim 9 , wherein the plurality of real keyframes are captured by the electronic device at a plurality of camera poses, respectively, and the storage circuit is configured to store the plurality of camera poses, wherein when performing the global bundle adjustment to optimize the map, the computing circuit is further configured to:

adjust the plurality of camera poses and the virtual pose to reduce the sum of re-projection errors.

11. The electronic device of claim 9 , wherein when performing the global bundle adjustment to optimize the map, the computing circuit is configured to calculate the sum of re-projection errors by performing the following operations:

projecting the first estimated coordinate onto real keyframes and virtual frames with first pixel points related to the first map point, so as to generate a plurality of first projection points;

calculating first re-projection errors between each of the first pixel points and a corresponding one of the plurality of first projection points;

projecting the second estimated coordinate onto real keyframes and virtual frames with second pixel points related to the second map point, so as to generate a plurality of second projection points;

calculating second re-projection errors between each of the second pixel points and a corresponding one of the plurality of second projection points; and

adding the first re-projection errors, the second re-projection errors and re-projection errors of other map points of the map, so as to generate the sum of re-projection errors.

12. The electronic device of claim 9 , wherein a number of the plurality of virtual cameras is twelve, wherein optical axes of half of the plurality of virtual cameras intersect at the first intersection coordinate, and optical axes of another half of the plurality of virtual cameras intersect at the second intersection coordinate.

13. The electronic device of claim 9 , wherein when the plurality of virtual cameras are generated, the first intersection coordinate, the second intersection coordinate, the first estimated coordinate and the second estimated coordinate are aligned with each other in an imaginary straight line.

14. The electronic device of claim 13 , wherein when the plurality of virtual cameras are generated, a distance from the first estimated coordinate to the first intersection coordinate equals a distance from the second estimated coordinate to the second intersection coordinate.

15. The electronic device of claim 9 , wherein the first distance value represents a real-world distance between the first marker and the second marker.

16. The electronic device of claim 9 , wherein each of the plurality of distance values represents a real-word distance between two markers of a plurality of markers comprising the first marker and the second marker,

wherein the computing circuit is configured to generate a plurality of sets of virtual cameras, and the plurality of virtual cameras is a set of the plurality of sets of virtual cameras,

wherein each set of the plurality of sets of virtual cameras is separated into two groups kept separated from each other by a corresponding one of the plurality of distance values while performing the global bundle adjustment, so as to introduce the plurality of distance values into the global bundle adjustment.

17. A non-transitory computer readable storage medium, storing a plurality of computer readable instructions for controlling an electronic device comprising a computing circuit and a storage circuit configured to store a plurality of distance values and a map, the plurality of computer readable instructions, when being executed by the computing circuit, causing the computing circuit to perform:

identifying, from the map, a first map point with a first estimated coordinate and a second map point with a second estimated coordinate generated based on a first marker and a second marker, respectively;

generating, according to the first estimated coordinate and the second estimated coordinate, a plurality of virtual cameras controlled by a virtual pose and with optical axes intersected at a first intersection coordinate and a second intersection coordinate separated from each other by a first distance value of the plurality of distance values, wherein the plurality of virtual cameras provide a plurality of virtual frames indicating that the first marker and the second marker are observed at the first intersection coordinate and the second intersection coordinate, respectively; and

performing a global bundle adjustment to optimize the map, comprising adjusting the first estimated coordinate and the second estimated coordinate to reduce a sum of re-projection errors calculated according to a plurality of real keyframes and the plurality of virtual frames, wherein the electronic device is configured to perform positioning according to the optimized map.

18. The non-transitory computer readable storage medium of claim 17 , wherein the plurality of real keyframes are captured by the electronic device at a plurality of camera poses, respectively, and the plurality of camera poses are stored in the electronic device, wherein performing the global bundle adjustment to optimize the map comprises:

adjusting the plurality of camera poses and the virtual pose to reduce the sum of re-projection errors.

19. The non-transitory computer readable storage medium of claim 17 , wherein performing the global bundle adjustment to optimize the map comprises calculating the sum of re-projection errors, wherein calculating the sum of re-projection errors comprises:

projecting the first estimated coordinate onto real keyframes and virtual frames with first pixel points related to the first map point, so as to generate a plurality of first projection points;

calculating first re-projection errors between each of the first pixel points and a corresponding one of the plurality of first projection points;

projecting the second estimated coordinate onto real keyframes and virtual frames with second pixel points related to the second map point, so as to generate a plurality of second projection points;

calculating second re-projection errors between each of the second pixel points and a corresponding one of the plurality of second projection points; and

adding the first re-projection errors, the second re-projection errors and re-projection errors of other map points of the map, so as to generate the sum of re-projection errors.

20. The non-transitory computer readable storage medium of claim 17 , wherein a number of the plurality of virtual cameras is twelve, wherein optical axes of half of the plurality of virtual cameras intersect at the first intersection coordinate, and optical axes of another half of the plurality of virtual cameras intersect at the second intersection coordinate.