Methods and systems to allow three-dimensional map sharing between heterogeneous computing systems, cross-localization, and sharing content in three-dimensional space
A method includes a computing system associated with a first artificial-reality device accessing sensor data of a real environment, generating a first plurality of feature descriptors for the real environment, and generating a second plurality of feature descriptors for the real environment based on the first plurality of feature descriptors, wherein the second plurality of feature descriptors have lower detail than the first plurality of feature descriptors. The computing system may further cause the first plurality of feature descriptors to be transmitted using a wireless connection, wherein the transmitted first plurality of feature descriptors are configured to be converted into a third plurality of feature descriptors different from the second plurality of feature descriptors. The third plurality of feature descriptors are configured to be used by a second artificial-reality device for tracking the real environment. The computing system tracks the real environment using the second plurality of feature descriptors.
1. A method comprising, by a computing system associated with a first device, wherein the first device is an artificial-reality device:
accessing sensor data of a real environment captured by the first device;
generating, based on the sensor data, a first plurality of feature descriptors for the real environment using a machine-learning model;
generating a second plurality of feature descriptors for the real environment based on the first plurality of feature descriptors, wherein the second plurality of feature descriptors have lower detail than the first plurality of feature descriptors;
transmitting the first plurality of feature descriptors using a wireless connection, wherein the transmitted first plurality of feature descriptors are configured to be converted into a third plurality of feature descriptors different from the second plurality of feature descriptors, the third plurality of feature descriptors being configured to be used by a second artificial-reality device for tracking the real environment; and
tracking the real environment using the second plurality of feature descriptors.
2. The method of claim 1 , wherein the sensor data is an image of the real environment captured by a camera worn by a user of the first artificial-reality device.
3. The method of claim 1 , wherein tracking the real environment using the second plurality of feature descriptors further comprises localizing the first artificial-reality device in the real environment.
4. The method of claim 1 , wherein the first artificial-reality device is a high-performant device relative to the second artificial-reality device, wherein the second plurality of feature descriptors have higher detail relative to the third plurality of feature descriptors.
5. The method of claim 1 , wherein the first artificial-reality device is a less-performant device relative to the second artificial-reality device, wherein the second plurality of feature descriptors have less detail relative to the third plurality of feature descriptors.
6. The method of claim 1 , wherein the first plurality of feature descriptors are transmitted to a server configured to convert the first plurality of feature descriptors into the third plurality of feature descriptors and transmit the third plurality of feature descriptors to the second artificial reality device.
7. The method of claim 1 , wherein the first plurality of feature descriptors are transmitted to the second artificial reality device, the second artificial reality device being configured to convert the first plurality of feature descriptors into the third plurality of feature descriptors.
8. The method of claim 1 , wherein the second plurality of feature descriptors are generated by a second machine learning model stored on the first artificial reality device.
9. The method of claim 1 , wherein the first plurality of feature descriptors are lossless feature descriptors.
10. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
access sensor data of a real environment captured by the first device;
generate, based on the sensor data, a first plurality of feature descriptors for the real environment using a machine-learning model;
generate a second plurality of feature descriptors for the real environment based on the first plurality of feature descriptors, wherein the second plurality of feature descriptors have lower detail than the first plurality of feature descriptors;
transmit the first plurality of feature descriptors using a wireless connection, wherein the transmitted first plurality of feature descriptors are configured to be converted into a third plurality of feature descriptors different from the second plurality of feature descriptors, the third plurality of feature descriptors being configured to be used by a second artificial-reality device for tracking the real environment; and
track the real environment using the second plurality of feature descriptors.
11. The media of claim 10 , wherein the sensor data is an image of the real environment captured by a camera worn by a user of the first artificial-reality device.
12. The media of claim 10 , wherein the first plurality of feature descriptors are transmitted to a server configured to convert the first plurality of feature descriptors into the third plurality of feature descriptors and transmit the third plurality of feature descriptors to the second artificial reality device.
13. The media of claim 10 , wherein the first plurality of feature descriptors are transmitted to the second artificial reality device, the second artificial reality device being configured to convert the first plurality of feature descriptors into the third plurality of feature descriptors.
14. The media of claim 10 , wherein the first artificial-reality device is a high-performant device relative to the second artificial-reality device, wherein the second plurality of feature descriptors have higher detail relative to the third plurality of feature descriptors.
15. The media of claim 10 , wherein the second plurality of feature descriptors are generated by a second machine learning model stored on the first artificial reality device.
16. A system comprising:
one or more processors; and
one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:
access sensor data of a real environment captured by the first device;
generate, based on the sensor data, a first plurality of feature descriptors for the real environment using a machine-learning model;
generate a second plurality of feature descriptors for the real environment based on the first plurality of feature descriptors, wherein the second plurality of feature descriptors have lower detail than the first plurality of feature descriptors;
transmit the first plurality of feature descriptors using a wireless connection, wherein the transmitted first plurality of feature descriptors are configured to be converted into a third plurality of feature descriptors different from the second plurality of feature descriptors, the third plurality of feature descriptors being configured to be used by a second artificial-reality device for tracking the real environment; and
track the real environment using the second plurality of feature descriptors.
17. The system of claim 16 , wherein the sensor data is an image of the real environment captured by a camera worn by a user of the first artificial-reality device.
18. The system of claim 16 , wherein the first plurality of feature descriptors are transmitted to a server configured to convert the first plurality of feature descriptors into the third plurality of feature descriptors and transmit the third plurality of feature descriptors to the second artificial reality device.
19. The system of claim 16 , wherein the first plurality of feature descriptors are transmitted to the second artificial reality device, the second artificial reality device being configured to convert the first plurality of feature descriptors into the third plurality of feature descriptors.
20. The system of claim 16 , wherein the first artificial-reality device is a high-performant device relative to the second artificial-reality device, wherein the second plurality of feature descriptors have higher detail relative to the third plurality of feature descriptors.