Augmenting human vision in limited-visibility environments
A Mixed-Reality (MR) device for augmenting human vision in limited-visibility environments (e.g., due to smoke 103 ) is provided. The MR device comprises a visible-range sensor for capturing a visible-range representation of a real-world scene surrounding the MR device with a range commensurate with human vision, an extended-range sensor for capturing an extended-range representation of the real-world scene with an extended range wider than the range commensurate with human vision, a display, and processing circuitry. The processing circuitry causes the MR device to be operative to generate a visible-range representation of edges of one or more physical objects ( 104 ) present in the real-world scene by extracting edges from the visible-range representation captured by the visible-range sensor, generate an extended-range representation of edges of the one or more physical objects ( 104 ) by extracting edges from the extended-range representation captured by the extended-range sensor, generate a delta representation of edges of the one or more physical objects ( 104 ), comprising edges present in the extended-range representation of edges but absent in the visible-range representation of edges (illustrated with thick lines in FIG. 2 D), and display the delta representation of edges on the display.
1 . A Mixed-Reality, MR, device, for augmenting human vision in limited-visibility environments, the MR device comprising:
a visible-range sensor for capturing a visible-range representation of a real-world scene surrounding the MR device with a range commensurate with human vision,
an extended-range sensor for capturing an extended-range representation of the real-world scene with an extended range wider than the range commensurate with human vision,
a display, and
processing circuitry causing the MR device to be operative to:
generate a visible-range representation of edges of one or more physical objects present in the real-world scene by extracting edges from the visible-range representation captured by the visible-range sensor,
generate an extended-range representation of edges of the one or more physical objects by extracting edges from the extended-range representation captured by the extended-range sensor,
generate a delta representation of edges of the one or more physical objects, comprising edges present in the extended-range representation of edges but absent in the visible-range representation of edges, and
display the delta representation of edges on the display.
2 . The MR device according to claim 1 , operative to generate the delta representation of edges of the one or more physical objects by subtracting the visible-range representation of edges from the extended-range representation of edges.
3 . The MR device according to claim 1 , wherein the display is a see-through display, and the delta representation of edges is displayed as an overlay onto the real-world scene as seen by a user viewing the display.
4 . The MR device according to claim 1 , wherein the display is an opaque display, and the MR device is further operative to display the visible-range representation of the real-world scene on the display.
5 . The MR device according to claim 1 , wherein the visible-range sensor is operative to capture the visible-range representation of the real-world scene using wavelengths within the visible range.
6 . The MR device according to claim 1 , wherein the extended-range sensor is operative to capture the extended-range representation of the real-world scene using wavelengths at least partly outside the visible range.
7 . The MR device according to claim 6 , wherein the extended-range sensor is an infrared-light sensor.
8 . The MR device according to claim 1 , wherein at least one of the visible-range sensor and the extended-range sensor is a digital camera.
9 . The MR device according to claim 1 , wherein the extended-range sensor is a Lidar or a radar sensor.
10 . The MR device according to claim 1 , being any one of: a hand-held device, a Head-Mounted Device, HMD, and a helmet-mounted device.
11 . A method of augmenting human vision in limited-visibility environments, the method being performed by a Mixed-Reality, MR, device and comprising:
capturing a visible-range representation of a real-world scene surrounding the MR device with a range commensurate with human vision, using a visible-range sensor comprised in the MR device,
capturing an extended-range representation of the real-world scene with an extended range wider than the range commensurate with human vision, using a an extended-range sensor comprised in the MR device,
generating a visible-range representation of edges of one or more physical objects present in the real-world scene by extracting edges from the visible-range representation captured by the visible-range sensor,
generating an extended-range representation of edges of the one or more physical objects by extracting edges from the extended-range representation captured by the extended-range sensor,
generating a delta representation of edges of the one or more physical objects, comprising edges present in the extended-range representation of edges but absent in the visible-range representation of edges, and
displaying the delta representation of edges on a display comprised in the MR device.
12 . The method according to claim 11 , wherein the delta representation of edges of the one or more physical objects is generated by subtracting the visible-range representation of edges from the extended-range representation of edges.
13 . The method according to claim 11 , wherein the display is a see-through display, and the delta representation of edges is displayed as an overlay onto the real-world scene as seen by a user viewing the display.
14 . The method according to claim 11 , wherein the display is an opaque display, and the method further comprises displaying the visible-range representation of the real-world scene on the display.
15 . The method according to claim 11 , wherein the visible-range representation of the real-world scene is captured using wavelengths within the visible range.
16 . The method according to claim 11 , wherein the extended-range representation of the real-world scene is captured using wavelengths at least partly outside the visible range.
17 . The method according to claim 16 , wherein the extended-range representation of the real-world scene is captured using infrared light.
18 . The method according to claim 11 , wherein at least one of the visible-range sensor and the extended-range sensor is a digital camera.
19 . The method according to claim 11 , wherein the extended-range sensor is a Lidar or a radar sensor.
20 . A non-transitory computer-readable medium storing a computer program comprising instructions which, when the computer program is executed by a Mixed-Reality, MR, device, causes the MR device to carry out the method according to claim 11 .