Distributed device mapping
The present invention relates to the efficient use of both local and remote computational resources and communication bandwidth to provide distributed environment mapping using a plurality of mobile sensor-equipped devices. According to a first aspect, there is provided a method of determining a global position of one or more landmarks on a global map, the method comprising the steps of determining one or more differences between sequential sensor data captured by one or more moving devices; determining one or more relative localisation landmark positions with respect to the one or more moving devices; determining relative device poses based one or more differences between sequential sensor data relative to the one or more relative localisation landmark positions; and determining a correlation between each device pose and the one or more relative localisation landmarks positions.
1. A method comprising:
determining, by a computing system, one or more estimated landmark positions for one or more localization landmarks relative to one or more moving devices based on sequential sensor data captured by the one or more moving devices;
determining, by the computing system, one or more estimated device positions for the one or more moving devices based on the one or more estimated landmark positions relative to the one or more moving devices;
determining, by the computing system, one or more refined device positions for the one or more moving devices in reference to a map by determining a variable that minimizes error between one or more global positions on the map and the one or more estimated device positions; and
updating, by the computing system, the map with one or more refined global landmark positions for the one or more localization landmarks based on the variable and a correlation between the one or more refined device positions and the one or more estimated relative landmark positions.
2. The method of claim 1 , wherein the determining the one or more refined device positions comprises minimizing a constraint based on estimated relative poses between sequential sensor poses associated with the one or more moving devices.
3. The method of claim 1 , further comprising determining one or more differences between sequential sensor poses captured by the one or more moving devices based on one or more of: a time stamp for each sequential sensor pose or a relative pose between the sequential sensor poses.
4. The method of claim 1 , wherein the one or more moving devices comprise one or more of: an image camera, a video camera, a laser based sensor, a depth camera, a stereo camera, a high dynamic range camera, a light detection and ranging sensor, and a radio detection and ranging sensor.
5. The method of claim 1 , further comprising receiving determined differences between sequential sensor poses captured by the one or more moving devices which is determined in real-time by the one or more moving devices.
6. The method of claim 1 , wherein the map comprises one or more portions linked by localization nodes associated with the one or more localization landmarks.
7. The method of claim 1 , wherein the determining the one or more estimated landmark positions is based on triangulation of the sequential sensor data captured by the one or more moving devices.
8. The method of claim 1 , wherein the determining the one or more estimated landmark positions comprises receiving one or more estimated relative landmark positions which are determined in real-time by the one or more moving devices.
9. The method of claim 1 , further comprising determining absolute device poses based on one or more differences between sensor poses, wherein the determining the absolute device poses based on the one or more differences between the sensor poses comprises estimating a relative pose between captures by the one or more moving devices.
10. The method of claim 1 , further comprising determining absolute device poses, wherein the determining the absolute device poses is performed on a centralized server or distributed across a cluster of computers within a network of computers.
11. The method of claim 1 , further comprising determining one or more absolute global positions for the one or more localization landmarks by independently computing each absolute global position.
12. The method of claim 1 , further comprising determining absolute device positions by minimizing a difference between the one or more estimated device positions and the one or more refined device positions.
13. The method of claim 1 , further comprising determining an absolute global position for the one or more localization landmarks by using one or more of Global Navigation Satellite System (GNSS) data or Inertial Measurement Unit (IMU) data.
14. The method of claim 1 wherein the sequential sensor data captured by the one or more moving devices is distributed across a plurality of devices within a geographical area and the one or more refined global landmark positions for the one or more localization landmarks is determined independently by the plurality of devices.
15. The method of claim 1 , further comprising:
re-determining the one or more refined device positions from known landmarks; and
re-determining the one or more refined global landmark positions of the one or more localization landmarks.
16. The method of claim 1 , wherein the updating the map further comprising updating the map using the sequential sensor data, wherein the updating the map comprises one or more of:
creating a new constraint or pose data in the map;
updating or amending an existing node in the map;
deleting a constraint or pose data in the map;
optimizing the map; or
performing global loop-closures to link positions of different localization constraint or pose data.
17. The method of claim 1 , wherein the determining the one or more estimated landmark positions is further based on detected image features in the sequential sensor data.
18. The method of claim 1 , further comprising providing a portion of the map to a device, wherein at least one of the one or more refined device positions or at least one of the one or more localization landmarks can be determined by the device based on the portion of the map.
19. The method of claim 1 , wherein the one or more global positions on the map are allocated to the one or more estimated device positions for the one or more moving devices.
20. A non-transitory computer-readable medium comprising computer-executable instructions which, when executed, performs a method comprising:
determining one or more estimated landmark positions for one or more localization landmarks relative to one or more moving devices based on sequential sensor data captured by the one or more moving devices;
determining one or more estimated device positions for the one or more moving devices based on the one or more estimated landmark positions relative to the one or more moving devices;
determining one or more refined device positions for the one or more moving devices in reference to a map by determining a variable that minimizes error between one or more global positions on the map and the one or more estimated device positions; and
updating the map with one or more refined global landmark positions for the one or more localization landmarks based on the variable and a correlation between the one or more refined device positions and the one or more estimated relative landmark positions.
21. The non-transitory computer-readable medium of claim 20 , wherein the determining the one or more refined device positions comprises minimizing a constraint based on estimated relative poses between sequential sensor poses associated with the one or more moving devices.
22. A system comprising a processor and memory storing instructions that, when executed by the processor, cause the system to perform a method comprising:
determining one or more estimated landmark positions for one or more localization landmarks relative to one or more moving devices based on sequential sensor data captured by the one or more moving devices;
determining one or more estimated device positions for the one or more moving devices based on the one or more estimated landmark positions relative to the one or more moving devices;
determining one or more refined device positions for the one or more moving devices in reference to a map by optimizing determining a variable that minimizes error between one or more global positions on the map and the one or more estimated device positions; and
aupdating the map with one or more refined global landmark positions for the one or more localization landmarks based on the variable and a correlation between the one or more refined device positions and the one or more estimated relative landmark positions.
23. The system of claim 22 , wherein the determining the one or more refined device positions comprises minimizing a constraint based on estimated relative poses between sequential sensor poses associated with the one or more moving devices.