Cross reality system
An augmented reality viewing system is described. A local coordinate frame of local content is transformed to a world coordinate frame. A further transformation is made to a head coordinate frame and a further transformation is made to a camera coordinate frame that includes all pupil positions of an eye. One or more users may interact in separate sessions with a viewing system. If a canonical map is available, the earlier map is downloaded onto a viewing device of a user. The viewing device then generates another map and localizes the subsequent map to the canonical map.
1. An electronic system comprising:
an electronic device that includes:
a processor;
a computer-readable medium connected to the processor, the computer-readable medium comprising a first coordinate frame and a second coordinate frame different from the first coordinate frame;
a data channel to receive data representing virtual content, wherein the processor is configured to execute a coordinate frame transformer to transform a positioning of the virtual content from the first coordinate frame to the second coordinate frame; and
a display system adapted to display the virtual content based, at least in part, on the positioning of the virtual content in the second coordinate frame.
2. The electronic system of claim 1 , wherein the electronic device further includes:
an object detection device that detects positioning of an object, wherein the processor is further configured to execute:
a world surface determining routine to identify positioning of at least one point on a surface of the object;
a world frame determining routine to compute a world coordinate frame based on the at least one point; and
a world frame storing instruction to store the world coordinate frame on the computer-readable medium.
3. The electronic system of claim 2 , wherein the electronic device further includes:
a head-mountable frame, wherein the processor is further configured to execute:
a head frame determining routine to compute a head coordinate frame that changes upon movement of the head-mountable frame; and
a head frame storing instruction to store the head coordinate frame on the computer-readable medium.
4. The electronic system of claim 3 , wherein the coordinate frame transformer is configured to transform the world coordinate frame to the head coordinate frame.
5. The electronic system of claim 3 , wherein the electronic device further includes:
an inertial measurement unit secured to the head-mountable frame that detects movement of the head-mountable frame, wherein the head frame determining routine computes the head coordinate frame based on a measurement of the inertial measurement unit.
6. The electronic system of claim 3 , wherein:
the electronic device further includes a movement tracking camera secured to the head-mountable frame,
the movement tracking camera detects movement of the head-mountable frame, and
the head frame determining routine computes the head coordinate frame based on images captured by the movement tracking camera.
7. The electronic system of claim 1 , wherein:
the computer-readable medium further includes a camera coordinate frame including a plurality of eye positions of an eye that moves relative to the head-mountable frame, wherein the camera coordinate frame is the second coordinate frame, and
the coordinate frame transformer comprises transforming the head coordinate frame to the camera coordinate frame.
8. An electronic system comprising:
a device comprising one or more sensors configured to capture data about one or more objects in a scene, the data being in a first coordinate frame; and
a computer-readable medium comprising computer executable instructions for specifying a location of virtual content in the scene based at least in part on information derived from the data in the first coordinate frame, wherein the location of the virtual content is specified in a second coordinate frame different from the first coordinate frame.
9. The electronic system of claim 8 , wherein:
the first coordinate frame is a first pose of the electronic system when the electronic system is powered on for capturing the data.
10. The electronic system of claim 8 , wherein:
the first coordinate frame has an origin determined based, at least in part, on dimensions of the electronic system and one or more poses of the one or more sensors of the electronic system when capturing the data.
11. The electronic system of claim 8 , comprising:
at least one processor configured to execute additional computer executable instructions to provide the virtual content, wherein the additional computer executable instructions comprise instructions for:
determining the first coordinate frame based, at least in part, on the one or more objects in the scene; and
transforming the specified location of the virtual content in the second coordinate frame to the first coordinate frame.
12. The electronic system of claim 11 , wherein the first coordinate frame is determined based, at least in part, on one or more nodes on an outer surface of a bounding box that encloses the virtual content.
13. The electronic system of claim 11 , wherein specifying a location of the virtual content in the scene based at least in part on information derived from the data in the first coordinate frame comprises determining a location of the device in a coordinate frame used by an application.
14. An electronic system comprising:
a portable device comprising a display and one or more sensors configured to capture data, with respect to a device coordinate frame, about a 3D environment; and
a computer-readable medium comprising computer executable instructions for:
obtaining a location of a virtual object in a stored map of the 3D environment with respect to a stored coordinate frame of the 3D environment,
computing a transformation between the device coordinate frame and the stored coordinate frame, and
rendering the virtual object on the display at a location determined, at least in part, based on the transformation and the obtained location of the virtual object.
15. The electronic system of claim 14 , wherein the computer executable instructions further comprise instructions for:
identifying a plurality of features in the data about the 3D environment,
generating a plurality of key frames based, at least in part, on the plurality of features, and
generating a plurality of persistent coordinate frames (PCFs) based, at least in part, on the plurality of key frames.
16. The electronic system of claim 15 , wherein
the computer executable instructions further comprise instructions for:
converting each of the identified plurality of features to a descriptor, and
each key frame comprises one or more descriptors.
17. The electronic system of claim 15 , wherein a new PCF is generated when the portable device moves for a predetermined distance after the last PCF is generated.
18. The electronic system of claim 15 , wherein
the transformation between the device coordinate frame and the stored coordinate frame is computed based, at least in part, on the generated PCFs and PCFs in the stored map of the 3D environment.
19. The electronic system of claim 15 , wherein
the computer executable instructions further comprise instructions for:
generating a plurality of persistent poses, each persistent pose corresponding to one or more of the plurality of key frames, and
the plurality of PCFs are generated based, at least in part, on the plurality of persistent poses.
20. The electronic system of claim 19 , wherein a new persistent pose is generated when the portable device moves for a predetermined distance after the last persistent pose is generated.