Virtual manipulation of hidden objects
An occluded object displayed on a head-mounted display (HMD) is manipulated by establishing a first view position relative to a real-world object, and receiving three-dimensional data associated with occluded objects, the occluded objects at least partially occluded by the real-world object at the first view position. A first image of at least a portion of the occluded objects is rendered on the HMD from the first view position at a geometrically correct location with respect to an image of the real-world object on the HMD using the three-dimensional data. At least one occluded object is selected from the rendered portion of the occluded objects, a second view position is established relative to the selected at least one occluded object, and a second image of the selected at least one occluded object is rendered on the HMD from the second view position using the three-dimensional data.
1. A method to virtually manipulate an occluded object shown through a display of a hybrid-reality system (DHR), the method comprising:
establishing a first view position for the DHR including a first view location and a first direction of view relative to a first real-world object;
receiving position data associated with one or more occluded real-world objects at the hybrid-reality system from a separate device over a computer network, the one or more occluded real-world objects at least partially occluded at the first view position by the first real-world object, the one or more occluded real-world objects including a hidden portion that is hidden from the first view position by the first real-world object;
rendering a first image of at least some of the hidden portion of the one or more occluded real-world objects on the DHR from the first view position, the at least some of the hidden portion of the one or more occluded real-world objects rendered at a geometrically correct location with respect to an image of the first real-world object on the DHR using the received position data;
manipulating a size and/or a position of a view-port superimposed on a second image that includes at least a portion of the first real-world object, wherein the first image is clipped to the view-port and presented on the DHR overlaid on the second image;
selecting at least one occluded object from the hidden portion of the one or more occluded real-world objects rendered in the first image based on the size and/or the position of the view-port.
2. The method of claim 1 , the manipulating comprising:
detecting a gesture of a user of the hybrid-reality system using a sensor coupled to the hybrid-reality system; and
manipulating the size and/or the position of the view-port based on the gesture.
3. The method of claim 1 , said selecting comprising receiving one or more commands from a user of the hybrid-reality system to select the at least one occluded object, wherein the at least one occluded object includes a first occluded object of the one or more occluded real-world objects rendered in the first image and excludes a second occluded object of the one or more occluded real-world objects rendered in the first image.
4. The method of claim 3 , the receiving one or more commands comprising:
detecting a gesture of the user using a sensor coupled to the hybrid-reality system; and
determining the one or more commands based on the gesture.
5. The method of claim 1 , further comprising establishing a second view position, including a second location and a second direction of view, relative to the at least one selected occluded object; and rendering a third image of the at least one selected occluded object on the DHR from the second view position.
6. The method of claim 5 , further comprising presenting the third image at a position on the DHR independent of a position of the second image.
7. The method of claim 5 , further comprising presenting the first image and the third image on the DHR simultaneously.
8. The method of claim 5 , further comprising detecting a change of location of the DHR to establish the second view position.
9. The method of claim 5 , further comprising receiving input from the user of the hybrid-reality system to establish the second view position.
10. The method of claim 9 , the receiving input comprising:
detecting a gesture of the user using a sensor coupled to the hybrid-reality system; and
determining the input based on the gesture.
11. The method of claim 5 , further comprising:
receiving input from a user of the hybrid-reality system over a period of time to establish subsequent view positions; and
generating subsequent images of the at least one selected occluded object on the DHR from the subsequent view positions in near real-time to create an interactive display on the DHR.
12. The method of claim 11 , the receiving input comprising:
detecting gestures of the user using a sensor coupled to the hybrid reality system; and
determining the input based on the gestures.
13. The method of claim 11 , the input comprising locations and/or orientations of the DHR over the period of time from sensors included in the DHR.
14. The method of claim 5 , further comprising:
determining a first location and a first orientation of the DHR using sensors in the DHR at a first time;
establishing the first view position based on the first location and the first orientation of the DHR;
establishing a virtual position of the at least one selected occluded object with respect to the first view location for a virtual presentation of the at least one selected occluded object; and
rendering the at least one selected object from the second view position at the virtual position in the third image.
15. The method of claim 13 , wherein the DHR comprises a head-mounted display (HMD).
16. The method of claim 14 , further comprising:
determining a second location and a second orientation of the DHR using the sensors in the DHR at a second time;
establishing the second view position based on the second location and the second orientation of the DHR.
17. The method of claim 1 , wherein the position data comprises a schematic, plan, blueprint, or map.
18. The method of claim 1 , wherein the position data is based on data from one or more sensors separate from the hybrid-reality system.
19. An article of manufacture comprising a tangible medium,
that is not a transitory propagating signal, encoding computer-readable instructions that, when applied to a computer system, instruct the computer system to perform a method comprising:
establishing a first view position for the DHR including a first view location and a first direction of view relative to a first real-world object;
receiving position data associated with one or more occluded real-world objects at the hybrid-reality system from a separate device over a computer network, the one or more occluded real-world objects at least partially occluded at the first view position by the first real-world object, the one or more occluded real-world objects including a hidden portion that is hidden from the first view position by the first real-world object;
rendering a first image of at least some of the hidden portion of the one or more occluded real-world objects on the DHR from the first view position, the at least some of the hidden portion of the one or more occluded real-world objects rendered at a geometrically correct location with respect to an image of the first real-world object on the DHR using the received position data;
manipulating a size and/or a position of a view-port superimposed on a second image that includes at least a portion of the first real-world object, wherein the first image is clipped to the view-port and presented on the DHR overlaid on the second image;
selecting at least one occluded object from the hidden portion of the one or more occluded real-world objects rendered in the first image based on the size and/or the position of the view-port.
20. A hybrid-reality system comprising:
a display;
a structure, coupled to the display and adapted to position the display in a field-of-view (FOV) of the user;
a network interface; and
a processor, coupled to the display and the network interface, the processor configured to:
establish a first view position for the display including a first view location and a first direction of view relative to a first real-world object;
receive position data associated with one or more occluded real-world objects from a separate device over a computer network, the one or more occluded real-world objects at least partially occluded at the first view position by the first real-world object, the one or more occluded real-world objects including a hidden portion that is hidden from the first view position by the first real-world object;
render a first image of at some of the hidden portion of the one or more occluded real-world objects on the display from the first view position, the at least some of the hidden portion of the one or more occluded real-world objects rendered at a geometrically correct location with respect to an image of the first real-world object on the display using the received position data;
manipulate a size and/or a position of a view-port superimposed on a second image that includes at least a portion of the first real-world object, wherein the first image is clipped to the view-port and presented on the DHR overlaid on the second image;
select at least one occluded object from the hidden portion of the one or more occluded real-world objects rendered in the first image based on the size and/or the position of the view-port.