Electronic system and method for three-dimensional mixed-reality space and experience construction and sharing
A novel 3D mixed-reality space and experience construction sharing system is configured to provide a novel part-holographic and part-physical object-based immersive user experience environment that enables an intermixture of computer-generated lifelike holographic objects and real objects to be synchronized and correlated to a particular physical space (i.e. as a “mixed-reality” (MR) environment) for vividly-interactive user experiences during a user's visit to the particular physical space. Moreover, the novel 3D mixed-reality space and experience construction sharing system accommodates a user interaction designer to construct and configure a mixed-reality (MR) environment and various potential user interactivities for a geographic landmark, a museum, or another tourist destination, and subsequently share the MR environment with other user interaction designers and users (e.g. tourists) who visit that tourist destination. Typically, the MR environment is visualized through a head-up display device or another portable electronic device executing an MR environment-visualization mobile application.
1. A three-dimensional (3D) mixed-reality space and experience construction sharing system comprising:
a dedicated physical studio space for 3D hologram generation, wherein the dedicated physical studio space includes a center stage, a plurality of cameras surrounding the center stage, and a 3D reconstruction electronic system integrating a memory unit and at least one of a CPU and a GPU, wherein the 3D reconstruction electronic system is configured to capture, calculate, reconstruct, and generate graphical transformation of a target object to create a user-controllable 3D holographic object;
a head-mounted display device connected to the 3D reconstruction electronic system;
a walk-through map creation engine and a 3D map database executed in a computer server connected to the 3D reconstruction electronic system and one or more portable electronic devices utilized by a mixed-reality experience designer and a mixed-reality experience viewer, wherein the walk-through map creation engine intelligently and automatically generates a 3D map of a designer-specified physical space with a mixed-reality artificial layer incorporating virtual coordinates correlated to the designer-specified physical space, when the mixed-reality experience designer walks through the designer-specified physical space while wearing the head-mounted display device connected to the walk-through map creation engine, and wherein the 3D map newly constructed by the walk-through map creation engine is subsequently stored in the 3D map database;
a user experience choreography engine and a 3D holographic database executed in the computer server, wherein the user experience choreography engine allows the mixed-reality experience designer to retrieve the 3D map from the 3D map database and select and position the user-controllable 3D holographic object retrieved from the 3D holographic database within the virtual coordinates of the mixed-reality artificial layer, which is graphically superimposed on the designer-specified physical space, and wherein the user experience choreography engine also provides a timed choreographic sequence of interactions in a mixed-reality experience scenario incorporated into multiple mixed-reality (MR) artificial layers, wherein each mixed-reality artificial layer embeds a custom time-sequenced interactivity among the user-controllable 3D holographic object, physical objects present in the designer-specified physical space, and the mixed-reality experience viewer immersed in the timed choreographic sequence of interactions in the mixed-reality experience scenario while also being present at the designer-specified physical space;
a mixed-reality experience designer creator module containing a mixed-reality experience construction interface, which is executed and displayed on the one or more portable electronic devices utilized by the mixed-reality experience designer to construct the timed choreographic sequence of interactions in the mixed-reality experience scenario for the mixed-reality experience viewer, wherein the mixed-reality experience designer module is connected to the walk-through map creation engine, the 3D map database, the user experience choreography engine, and the 3D holographic database; and
a mixed-reality experience viewer module connected to the mixed-reality experience designer creator module, and containing a viewer's mixed-reality visualization interface executed and displayed by the one or more portable electronic devices utilized by the mixed-reality experience viewer, wherein the mixed-reality experience viewer immerses in a mixed-reality environment that intermixes the user-controllable 3D holographic object on the virtual coordinates of the mixed-reality artificial layer with one or more physical objects present in the designer-specified physical space.
2. The three-dimensional (3D) mixed-reality space and experience construction sharing system of claim 1 , further comprising additional mixed-reality computer-generated objects and mixed-reality humanized holographic figures generated by the 3D reconstruction electronic system, wherein the additional mixed-reality computer-generated objects and the mixed-reality humanized holographic figures are utilized by the mixed-reality experience designer creator module for additional placements and choreographed interactivity constructions in the timed choreographic sequence of interactions in the mixed-reality experience scenario, which also integrates a physical presence of various nearby physical objects in the designer-specified physical space.
3. The three-dimensional (3D) mixed-reality space and experience construction sharing system of claim 1 , further comprising mixed-reality holographic app and service toolkits that enable the mixed-reality experience designer to create, modify, and manage a variety of 3D mixed-reality space and experience applications and service offerings, wherein the mixed-reality holographic app and the service toolkits are integrated into or operatively connected to the walk-through map creation engine and the user experience choreography engine.
4. The three-dimensional (3D) mixed-reality space and experience construction sharing system of claim 1 , further comprising a wireless data network connecting the dedicated physical studio space, the 3D reconstruction electronic system, the walk-through map creation engine, the 3D map database, the user experience choreography engine, the 3D holographic database, the mixed-reality experience designer creator module, and the mixed-reality experience viewer module.
5. The three-dimensional (3D) mixed-reality space and experience construction sharing system of claim 1 , wherein the one or more portable electronic devices utilized by the mixed-reality experience designer and the mixed-reality experience viewer are at least one of head-mounted display devices, smart phones, tablet computers, or notebook computers.
6. The three-dimensional (3D) mixed-reality space and experience construction sharing system of claim 1 , wherein the 3D reconstruction electronic system in the dedicated physical studio space is configured to perform silhouette extractions, 3D voxel generation, 3D mesh generation, and texture and detail-adding operations to create the user-controllable holographic object that resembles the target object.
7. The three-dimensional (3D) mixed-reality space and experience construction sharing system of claim 1 , wherein the user-controllable holographic object is a holographic tour guide, a holographic museum curator, or a holographic travel assistant, which is juxtaposed to the one or more physical objects present in the designer-specified physical space.
8. The three-dimensional (3D) mixed-reality space and experience construction sharing system of claim 7 , wherein the one or more physical objects are geographic landmarks, museum exhibits, artifacts, or points of interest, and wherein the designer-specified physical space is a popular tourist spot or a renowned travel destination that contains the geographic landmarks, the museum exhibits, the artifacts, or the points of interest.
9. The three-dimensional (3D) mixed-reality space and experience construction sharing system of claim 3 , wherein the variety of 3D mixed-reality space and experience applications and service offerings include holographic mixed-reality (HMR) tour guide applications to integrate holographic travel guides or holographic curators in the timed choreographic sequence of interactions in the mixed-reality experience scenario generated by the mixed-reality experience designer creator module for the mixed-reality experience viewer.
10. The three-dimensional (3D) mixed-reality space and experience construction sharing system of claim 1 , wherein choreographing of the user-controllable 3D holographic object within the virtual coordinates of the mixed-reality artificial layer involves defining a sequence and a roadmap of possible interactions between the user-controllable 3D holographic object and the mixed-reality experience viewer in the designer-specified physical space.
11. The three-dimensional (3D) mixed-reality space and experience construction sharing system of claim 1 , wherein the user-controllable 3D holographic object incorporates artificial intelligence to provide naturalized and dynamic question-and-answer and conversational capabilities between the user-controllable 3D holographic object and the mixed-reality experience viewer.