Semantic mapping for low-power augmented reality using dynamic vision sensor

1. A computer-implemented method comprising:

receiving, at a master platform from a first device, (i) first pose data associated with an image sensor of the first device and (ii) a first semantic map generated by the first device, the first semantic map comprising at least one simplified object representation in a coordinate space of the first device;

receiving, at the master platform from a second device, a) second pose data associated with an image sensor of the second device and (ii) a second semantic map generated by the second device, the second semantic map comprising at least one simplified object representation in a coordinate space of the second device;

identifying a shared simplified object representation common to the first semantic map and the second semantic map;

performing a transformation of the first semantic map from the coordinate space of the first device to a global coordinate system;

performing a transformation of the second semantic map from the coordinate space of the second device to the global coordinate system;

generating a time-stamped global consistent map, the time-stamped global consistent map comprising a union set of the at least one simplified object representation in the first semantic map and the at least one simplified object representation in the second semantic map; and

combining the first semantic map with the second semantic map based on the first pose data and the second pose data,

wherein one or more elements of the time-stamped global consistent map are assigned coordinates in the global coordinate system,

wherein the first pose data, the first semantic map, the second pose data, and the second semantic map are associated with a common time interval, and

wherein the shared simplified object representation comprises geometric primitives representing surfaces for positioning items of augmented reality (AR) content.

2. The computer-implemented method of claim 1 , further comprising:

applying a fusion algorithm to the time-stamped global consistent map to update a global stable semantic map.

3. The computer-implemented method of claim 2 , further comprising:

sending the updated global stable semantic map to the first device.

4. The computer-implemented method of claim 1 , further comprising:

assigning the first device and the second device to a device group of a plurality of device groups based on the shared simplified object representation; and

applying a fusion algorithm to time-stamped global consistent maps of each device group of the plurality of device groups to update a global stable semantic map.

5. The computer-implemented method of claim 1 , wherein the master platform is one or more of: (1) a peer device operating as a master device, (2) a server, or (3) a cloud computing platform.

6. The computer-implemented method of claim 1 , wherein the first pose data associated with the image sensor of the first device comprises image data obtained from a dynamic vision sensor (DVS).

7. The computer-implemented method of claim 6 , wherein the image data obtained from the dynamic vision sensor comprises a filtered event frame of DVS sensor data obtained over a predetermined interval.

8. An apparatus comprising:

a communication unit configured to:

receive, from a first device, (i) first pose data associated with an image sensor of the first device and (ii) a first semantic map generated by the first device, the first semantic map comprising at least one simplified object representation in a coordinate space of the first device; and

receive, from a second device, (i) second pose data associated with an image sensor of the second device and (ii) a second semantic map generated by the second device, the second semantic map comprising at least one simplified object representation in a coordinate space of the second device; and

a processor configured to:

identify a shared simplified object representation common to the first semantic map and the second semantic map;

perform a transformation of the first semantic map from the coordinate space of the first device to a global coordinate system;

perform a transformation of the second semantic map from the coordinate space of the second device to the global coordinate system;

generate a time-stamped global consistent map, the time-stamped global consistent map comprising a union set of the at least one simplified object representation in the first semantic map and the at least one simplified object representation in the second semantic map; and

combine the first semantic map with the second semantic map based on the first pose data and the second pose data,

wherein the processor is configured to assign one or more elements of the time-stamped global consistent map coordinates in the global coordinate system,

wherein the first pose data, the first semantic map, the second pose data, and the second semantic map are associated with a common time interval, and

wherein the shared simplified object representation comprises geometric primitives representing surfaces for positioning items of augmented reality (AR) content.

9. The apparatus of claim 8 , wherein the processor is further configured to:

apply a fusion algorithm to the time-stamped global consistent map to update a global stable semantic map.

10. The apparatus of claim 9 , wherein the processor is further configured to:

send, via the communication unit, the updated global stable semantic map to the first device.

11. The apparatus of claim 8 , wherein the processor is further configured to:

assign the first device and the second device to a device group of a plurality of device groups based on the shared simplified object representation; and

apply a fusion algorithm to time-stamped global consistent maps of each device group of the plurality of device groups to update a global stable semantic map.

12. The apparatus of claim 8 , wherein the apparatus is one or more of: (1) a peer device operating as a master device, (2) a server, or (3) a cloud computing platform.

13. The apparatus of claim 8 , wherein the first pose data associated with the image sensor of the first device comprises image data obtained from a dynamic vision sensor (DVS).

14. The apparatus of claim 13 , wherein the image data obtained from the dynamic vision sensor comprises a filtered event frame of DVS sensor data obtained over a predetermined interval.

15. A non-transitory computer-readable medium comprising program code that, when executed by a processor of an apparatus, causes the apparatus to:

receive, from a first device, (i) first pose data associated with an image sensor of the first device and (ii) a first semantic map generated by the first device, the first semantic map comprising at least one simplified object representation in a coordinate space of the first device;

receive, from a second device, (i) second pose data associated with an image sensor of the second device and (ii) a second semantic map generated by the second device, the second semantic map comprising at least one simplified object representation in a coordinate space of the second device;

identify a shared simplified object representation common to the first semantic map and the second semantic map;

perform a transformation of the first semantic map from the coordinate space of the first device to a global coordinate system;

perform a transformation of the second semantic map from the coordinate space of the second device to the global coordinate system;

generate a time-stamped global consistent map, the time-stamped global consistent map comprising a union set of the at least one simplified object representation in the first semantic map and the at least one simplified object representation in the second semantic map; and

combine the first semantic map with the second semantic map based on the first pose data and the second pose data,

wherein one or more elements of the time-stamped global consistent map are assigned coordinates in the global coordinate system,

wherein the first pose data, the first semantic map, the second pose data, and the second semantic map are associated with a common time interval, and

wherein the shared simplified object representation comprises geometric primitives representing surfaces for positioning items of augmented reality (AR) content.

16. The non-transitory computer-readable medium of claim 15 , wherein the program code, when executed by the processor, is further configured to cause the apparatus to:

apply a fusion algorithm to the time-stamped global consistent map to update a global stable semantic map.

17. The non-transitory computer-readable medium of claim 16 , wherein the program code, when executed by the processor, is further configured to cause the apparatus to:

send the updated global stable semantic map to the first device.

18. The non-transitory computer-readable medium of claim 15 , wherein the program code, when executed by the processor, is further configured to cause the apparatus to:

assign the first device and the second device to a device group of a plurality of device groups based on the shared simplified object representation; and

apply a fusion algorithm to time-stamped global consistent maps of each device group of the plurality of device groups to update a global stable semantic map.

19. The non-transitory computer-readable medium of claim 15 , wherein the first pose data associated with the image sensor of the first device comprises image data obtained from a dynamic vision sensor (DVS).

20. The non-transitory computer-readable medium of claim 19 , wherein the image data obtained from the dynamic vision sensor comprises a filtered event frame of DVS sensor data obtained over a predetermined interval.