Cross reality system
An augmented reality viewing system is described. A local coordinate frame of local content is transformed to a world coordinate frame. A further transformation is made to a head coordinate frame and a further transformation is made to a camera coordinate frame that includes all pupil positions of an eye. One or more users may interact in separate sessions with a viewing system. If a canonical map is available, the earlier map is downloaded onto a viewing device of a user. The viewing device then generates another map and localizes the subsequent map to the canonical map.
1. An electronic system comprising:
a first electronic device that includes:
a first processor;
a first computer-readable medium connected to the first processor, the first computer-readable medium comprising a first origin coordinate frame and a first destination coordinate frame different from the first origin coordinate frame;
a first data channel to receive data representing virtual content, wherein the first processor is configured to execute a first coordinate frame transformer to transform a positioning of the virtual content from the first origin coordinate frame to the first destination coordinate frame; and
a first display system adapted to display the virtual content based, at least in part, on the positioning of the virtual content in the first destination coordinate frame.
2. The electronic system of claim 1 , wherein the first electronic device further includes:
a first real object detection device that detects positioning of a real object, wherein the first processor is further configured to execute:
a first world surface determining routine to identify positioning of at least one point on a surface of the real object;
a first world frame determining routine to compute a first world coordinate frame based on the at least one point; and
a first world frame storing instruction to store the world coordinate frame on the computer-readable medium.
3. The electronic system of claim 2 , wherein the first real object detection device is a camera.
4. The electronic system of claim 2 , wherein the first real object detection device detects positioning of a plurality of real objects.
5. The electronic system of claim 2 , wherein the first world surface determining routine identifies positioning of a plurality of points on a surface of the real object.
6. The electronic system of claim 5 , wherein the first world frame determining routine computes the first world coordinate frame based on the plurality of points.
7. The electronic system of claim 2 , wherein the first electronic device further includes:
a first head-mountable frame, wherein the first processor is further configured to execute;
a first head frame determining routine to compute a first head coordinate frame that changes upon movement of the first head-mountable frame; and
a first head frame storing instruction to store the first head coordinate frame on the first computer-readable medium.
8. The electronic system of claim 7 , wherein the first coordinate frame transformer is configured to transform the first world coordinate frame to the first head coordinate frame.
9. The electronic system of claim 7 , wherein the first electronic device further includes:
a first inertial measurement unit secured to the first head-mountable frame that detects movement of the first head-mountable frame, wherein the first head frame determining routine computes the first head coordinate frame based on a measurement of the first inertial measurement unit.
10. The electronic system of claim 7 , wherein:
the first electronic device further includes a first movement tracking camera secured to the first head-mountable frame,
the first movement tracking camera detects movement of the first head-mountable frame, and
the first head frame determining routine computes the first head coordinate frame based on images captured by the first movement tracking camera.
11. The electronic system of claim 2 , wherein the first processor is configured to execute:
a first local frame determining routine to compute a first local coordinate frame of the virtual content; and
a first local frame storing instruction to store the local coordinate frame on the computer-readable medium.
12. The electronic system of claim 11 , wherein the first coordinate frame transformer comprises transforming the first local coordinate frame to the first world coordinate frame.
13. The electronic system of claim 1 , wherein:
the first computer-readable medium further includes a first camera coordinate frame including a plurality of eye positions of an eye that moves relative to the first head-mountable frame, and
the first coordinate frame transformer comprises transforming the first head coordinate frame to the first camera coordinate frame.
14. A method of operating an electronic system to render virtual content in a 3D environment comprising a portable device, the method comprising, with one or more processors:
providing a first coordinate frame based, at least in part, on a pose of the electronic system when the electronic system is powered on for a session;
providing a second coordinate frame based, at least in part, on a pose of a sensor of the electronic system when the sensor is capturing information about the 3D environment;
obtaining data representing the virtual content;
transforming a positioning of the virtual content from the first coordinate frame to the second coordinate frame; and
rendering the virtual content using the positioning of the virtual content in the second coordinate frame.
15. An electronic system comprising:
a device portable by a user comprising one or more sensors configured to capture sensor data about one or more physical objects in a scene, the sensor data being in a first coordinate frame; and
an application comprising computer executable instructions to specify a location of virtual content in the scene based at least in part on information derived from the sensor data in the first coordinate frame, wherein the application specifies the location of the virtual content in a second coordinate frame different from the first coordinate frame such that the displaying of the virtual content is independent of eye position and/or deformations of the device portable by the user that changes position of the one or more sensors.
16. The electronic system of claim 15 , wherein:
the first coordinate frame is a first pose of the electronic system when the electronic system is powered on for capturing the sensor data.
17. The electronic system of claim 15 , wherein:
the first coordinate frame has an origin determined based, at least in part, on dimensions of the electronic system and one or more poses of the one or more sensors of the electronic system when capturing the sensor data.
18. The electronic system of claim 15 , comprising:
at least one processor configured to execute additional computer executable instructions to provide the virtual content to the application, wherein the additional computer executable instructions comprise instructions for:
determining the first coordinate frame based, at least in part, on the one or more physical objects in the scene; and
transforming the specified location of the virtual content in the second coordinate frame to the first coordinate frame.
19. The electronic system of claim 18 , wherein the first coordinate frame is determined based, at least in part, on one or more nodes on an outer surface of a bounding box that encloses the virtual content.
20. The electronic system of claim 18 , wherein specifying a location of the virtual content in the scene based at least in part on information derived from the sensor data in the first coordinate frame comprises determining a location of the device portable by the user in a coordinate frame used by the application.