Display-based interactive simulation with dynamic panorama
The subject invention provides a panoramic, interactive, seamless, intuitive, high-resolution and yet extensive simulation tool capable of simulating dynamic events that alter the panoramic view in response to the user's actions or inaction. The dynamic panorama simulation can generate and display novel scenes that are not limited to static “canned” content that has been previously photographed, rendered, recorded, captured or filmed thus providing great flexibility, contextual authenticity and liveliness and, in some instances, unanticipated and surprising but accurate developments in the simulation.
1. A non-transitory computer-readable medium storing code for controlling one or more processors to provide a display-based simulation, wherein the code includes instructions for:
executing the display-based simulation;
accessing a first image comprising a panoramic image of an environment to be simulated and dynamic and/or interactive images overlaying the panoramic image, wherein the dynamic and/or interactive images comprise elements of the panoramic image to be manipulated, modulated, and/or observed; and
displaying a viewing portion of the first image while the display-based simulation is being executed, wherein a dynamic image of the dynamic images reflects consequences of an action taken by a user as the consequences occur over time in the simulation environment, the dynamic image using visual cues to reflect the consequences.
2. The medium according to claim 1 , wherein the panoramic image comprises a set of photographs or images stitched together or a single photograph or image.
3. The medium according to claim 1 , wherein the panoramic image comprises an abstract image or a schematic image.
4. The medium according to claim 1 , wherein the panoramic image has warping and the dynamic and/or interactive images overlaying the panoramic image are warped corresponding to the warping of the panoramic image; and
wherein displaying the viewing portion of the first image comprises displaying a de-warped viewing portion to a user.
5. The medium according to claim 4 , wherein displaying a de-warped viewing portion to the user comprises instructions for:
removing or reducing visible warping in the viewing portion of the first image to create a second image comprising de-warped dynamic and/or interactive images and a de-warped portion of the panoramic image, wherein the second image provides the de-warped viewing portion.
6. The medium according to claim 5 , further comprising instructions for:
mapping the de-warped dynamic and/or interactive images of the second image to corresponding ones of the warped dynamic and/or interactive images of the first image; and
mapping the user's displayed cursor position in the second image to a corresponding cursor position in the first image.
7. The medium according to claim 6 , further comprising instructions for:
executing any pointing device commands generated by the user based on the mapped cursor position in the first image.
8. The medium according to claim 1 , further comprising instruction for:
displaying to a user indicia that a selected image of the dynamic and/or interactive images is interacted, wherein the indicia is available while a cursor is within an active area of the selected image.
9. The medium according to claim 1 , further comprising instructions to provide audio cues to reflect the consequences of the action taken by the user.
10. The medium according to claim 9 , wherein the instructions to provide audio cues comprise instructions to direct the audio cues to a particular audio output for orientated delivery of the audio cues.
11. The medium according to claim 1 , wherein the dynamic images comprise animations, motion pictures, or a combination of animations and motion pictures, the computer-readable medium further comprising instructions for accessing and displaying the animations, motion pictures, or the combination of animations and motion pictures when the animations, motion pictures, or the combination of animations and motion pictures are in the viewing portion.
12. The medium according to claim 1 , wherein the dynamic images comprise a video or a medical equipment data representation, the computer-readable medium further comprising instructions for accessing and displaying the video or the medical equipment data representation when the video or the medical equipment data representation is in the viewing portion.
13. The medium according to claim 12 , wherein the video comprises frames, wherein certain frames of the video that are associated with predetermined events are tagged with a tag comprising information, the computer-readable medium further comprising instructions for:
altering the simulation based on the information in a tag when a tagged frame is played.
14. The medium according to claim 1 , further comprising instructions utilizing mathematical models to modulate the dynamic and/or interactive images based on actions and inactions of a user.
15. A non-transitory computer-readable-medium storing code for controlling one or more processors to provide a display-based simulation, wherein the code includes instructions to:
execute the display-based simulation;
display a panoramic image, providing a background for a simulation environment of the display-based simulation;
display interactive images superimposed on regions of the panoramic image;
adjust a viewing portion of the panoramic image having the superimposed interactive images based on a defined area selected to be displayed to a user; and
display dynamic graphics superimposed on regions of the panoramic image, wherein the viewing portion is a viewing portion of the panoramic image having the superimposed interactive images and the dynamic graphics, wherein a dynamic image of the dynamic graphics reflects consequences of an action taken by the user as the consequences unfold over time in the simulation environment, the dynamic image using visual cues to reflect the consequences.
16. The medium according to claim 15 , further comprising instructions to:
provide a panning tool for selecting the defined area to be displayed to the user.
17. The medium according to claim 15 , further comprising instructions to display a zoomed-in image of a selected component of the panoramic image based on input from the user.
18. The medium according to claim 15 , further comprising instructions utilizing mathematical models to modulate the simulation environment based on actions and inactions of the user.
19. A method for providing a simulation to a user of an electronic device having a display monitor and a pointing device, the method comprising:
displaying a viewing portion of a panoramic image providing a background for a simulation environment of the simulation; and
displaying dynamic and/or interactive images overlaid on regions of the panoramic image while the simulation is executing, wherein a dynamic image of the dynamic images reflects consequences of an action taken by a user as the consequences occur over time in the simulation environment, the dynamic image using visual cues to reflect the consequences.
20. The method according to claim 19 , further comprising:
providing a panning tool for adjusting the viewing portion of the panoramic image having the overlaid dynamic and/or interactive images.
21. The method according to claim 19 , further comprising:
animating the dynamic and/or interactive images available in the viewing portion in accordance with actions of a user or outputs of mathematical models for modulating the simulation environment based on actions and inactions of the user.
22. The method according to claim 19 , wherein the displaying of the panoramic image and the dynamic and/or interactive images overlaying the panoramic image comprises displaying a de-warped viewing portion, the method further comprising:
mapping a real cursor position in the de-warped viewing portion to a corresponding cursor position in a warped first image comprising the panoramic image and all overlaid interactive and/or dynamic images.
23. The method according to claim 22 , further comprising:
executing pointing device commands generated by the user based on the mapped cursor position in the warped first image.
24. The method according to claim 19 , further comprising:
displaying a time control tool for adjusting a point in time within the display-based simulation.