Systems and methods for augmented reality
View Patent ↗An augmented reality display system includes an electromagnetic field emitter to emit a known magnetic field in a known coordinate system. The system also includes an electromagnetic sensor to measure a parameter related to a magnetic flux at the electromagnetic sensor resulting from the known magnetic field. The system further includes a depth sensor to measure a distance in the known coordinate system. Moreover, the system includes a controller to determine pose information of the electromagnetic sensor relative to the electromagnetic field emitter in the known coordinate system based at least in part on the parameter related to the magnetic flux measured by the electromagnetic sensor and the distance measured by the depth sensor. In addition, the system includes a display system to display virtual content to a user based at least in part on the pose information of the electromagnetic sensor relative to the electromagnetic field emitter.
1. An augmented reality (AR) display system, comprising:
a hand-held controller comprising:
an electromagnetic field emitter to emit a known magnetic field in a known coordinate system, and
an inertial measurement (“IMU)” component that assists in determining positioning and/or orientation of the electromagnetic field emitter relative to other components;
an electromagnetic sensor to measure a parameter related to a magnetic flux at the electromagnetic sensor resulting from the known magnetic field, wherein the electromagnetic field emitter and the electromagnetic sensor are mobile;
a depth sensor to measure a distance in the known coordinate system;
a controller to determine pose information of the electromagnetic sensor relative to the electromagnetic field emitter in the known coordinate system based at least in part on the parameter related to the magnetic flux measured by the electromagnetic sensor and the distance measured by the depth sensor; and
a display system to display virtual content to a user based at least in part on the pose information of the electromagnetic sensor relative to the electromagnetic field emitter.
2. The AR display system of claim 1 , further comprising a world capture camera and a picture camera,
wherein the depth sensor comprises a depth camera having a first field of view (FOV),
wherein the world capture camera has a second FOV at least partially overlapping with the first FOV,
wherein the picture camera has a third FOV at least partially overlapping with the first FOV and the second FOV, and
wherein the depth camera, the world capture camera, and the picture camera are configured to capture respective first, second, and third images.
3. The AR display system of claim 2 , wherein the controller is programmed to segment the second and third images.
4. The AR display system of claim 3 , wherein the controller is programmed to fuse the second and third images after segmenting the second and third images to generate a fused image.
5. The AR display system of claim 1 , further comprising an additional localization resource to provide additional information, wherein the pose information of the electromagnetic sensor relative to the electromagnetic field emitter in the known coordinate system is determined based at least in part on the parameter related to the magnetic flux measured by the electromagnetic sensor, the distance measured by the depth sensor, and the additional information provided by the additional localization resource.
6. The AR display system of claim 1 , wherein the electromagnetic sensor is attached to a head set of the AR display system.
7. An augmented reality display system, comprising:
a hand-held component comprising an electromagnetic field emitter, and an inertial measurement (“IMU)” component that assists in determining positioning and/or orientation of the electromagnetic field emitter relative to other components, the electromagnetic field emitter emitting a magnetic field;
a head-mounted component having a display system that displays virtual content to a user, the head mounted component coupled to an electromagnetic sensor measuring a parameter related to a magnetic flux at the electromagnetic sensor resulting from the magnetic field, wherein a head pose of the head-mounted component in a known coordinate system is known, wherein the electromagnetic field emitter and the electromagnetic sensor are mobile;
a depth sensor measuring a distance in the known coordinate system; and
a controller communicatively coupled to the hand-held component, the head-mounted component, and the depth sensor, the controller receiving the parameter related to the magnetic flux at the electromagnetic sensor from the head mounted component and receiving the distance from the depth sensor,
wherein the controller determines a hand pose of the hand-held component based at least in part on the parameter related to the magnetic flux measured by the electromagnetic sensor and the distance measured by the depth sensor,
wherein the system modifies the virtual content displayed to the user based at least in part on the hand pose.