IP Library Granted Patent US 9,342,929
Granted Patent B2
US 9,342,929 · App. 13/747,328 · Granted May 17, 2016

Mixed reality experience sharing

Inventors: Nicholas Kamuda (Seattle, WA); Peter Tobias Kinnebrew (Seattle, WA)
Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
G06T19/006G02B27/00G06T15/04G06T17/00A63F2300/1093A63F2300/205A63F2300/407A63F2300/63A63F2300/695G06T2219/024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,342,929
App. No.
13/747,328
Granted
May 17, 2016
Kind
B2
Abstract

Embodiments that relate to presenting a textured shared world model of a physical environment are disclosed. One embodiment includes receiving geo-located crowd-sourced structural data items of the physical environment. The structural data items are stitched together to generate a 3D spatial shared world model. Geo-located crowd-sourced texture data items are also received and include time-stamped images or video. User input of a temporal filter parameter is used to temporally filter the texture data items. The temporally-filtered texture data items are applied to surfaces of the 3D spatial shared world model to generate a textured shared world model of the physical environment. The textured shared world model is then provided for display by a display device.

Claims (50)

1. A mixed reality experience sharing system for presenting a textured shared world model of a physical environment, the textured shared world model including a plurality of crowd-sourced structural data items that are geo-located at the physical environment, and a plurality of crowd-sourced texture data items that are geo-located at the physical environment, the mixed reality experience sharing system comprising:

a computing device comprising a processor; and

a mixed reality experience sharing program executed by the processor of the computing device, the mixed reality experience sharing program configured to:

receive the plurality of crowd-sourced structural data items including depth information;

stitch together the plurality of crowd-sourced structural data items to generate a 3D spatial shared world model of the physical environment;

receive the plurality of crowd-sourced texture data items from a plurality of sources, each of the texture data items comprising a time-stamped image or video, wherein the plurality of crowd-sourced texture data items comprise at least one hologram and metadata describing the hologram;

receive user input of a temporal filter parameter;

filter the plurality of crowd-sourced texture data items by the temporal filter parameter to generate a temporally-filtered subgroup of the crowd-sourced texture data items;

apply to surfaces of the 3D spatial shared world model of the physical environment the temporally-filtered subgroup of the crowd-sourced texture data items to generate the textured shared world model of the physical environment;

provide the textured shared world model of the physical environment to a display program for display by a display device; and

locate the hologram within the textured shared world model of the physical environment.

2. The mixed reality experience sharing system of claim 1 ,

wherein the temporal filter parameter corresponds to a predated time that is earlier than a current time.

3. The mixed reality experience sharing system of claim 2 , wherein the mixed reality experience sharing program is further configured to:

filter the plurality of crowd-sourced structural data items by the temporal filter parameter to generate a temporally-filtered subgroup of the crowd-sourced structural data items; and

stitch together the temporally-filtered subgroup of the crowd-sourced structural data items to generate the 3D spatial shared world model of the physical environment.

4. The mixed reality experience sharing system of claim 1 , wherein the mixed reality experience sharing program is further configured to receive the plurality of crowd-sourced texture data items from at least a user of the display device and a third party source.

5. The mixed reality experience sharing system of claim 1 , wherein each of the plurality of crowd-sourced texture data items is selected from the group consisting of light illuminating one or more of the crowd-sourced structural data items, an image of an object in the physical environment, a video of an object in the physical environment, and audio of the physical environment.

6. The mixed reality experience sharing system of claim 1 , wherein the mixed reality experience sharing program is further configured to prioritize the plurality of crowd-sourced structural data items according to one or more of a fidelity, an age, a trustworthiness of a source, and a social proximity of the source of the crowd-sourced structural data items.

7. The mixed reality experience sharing system of claim 1 , wherein the mixed reality experience sharing program is further configured to receive the plurality of crowd-sourced structural data items from at least one head-mounted display device.

8. The mixed reality experience sharing system of claim 1 , wherein the mixed reality experience sharing program is further configured to prioritize the plurality of crowd-sourced texture data items according to one or more of a trustworthiness of a source and a social proximity of the source of the crowd-sourced texture data items.

9. A method for presenting a textured shared world model of a physical environment via a display device, the method comprising:

receiving a plurality of crowd-sourced structural data items that are geo-located at the physical environment and include depth information;

stitching together the plurality of crowd-sourced structural data items to generate a 3D spatial shared world model of the physical environment;

receiving from a plurality of sources a plurality of crowd-sourced texture data items that are geo-located at the physical environment, each of the crowd-sourced texture data items comprising a time-stamped image or video, wherein the plurality of crowd-sourced texture data items comprise at least one hologram and metadata describing the hologram;

receiving user input of a temporal filter parameter;

filtering the plurality of crowd-sourced texture data items by the temporal filter parameter to generate a temporally-filtered subgroup of the crowd-sourced texture data items;

applying to surfaces of the 3D spatial shared world model of the physical environment the temporally-filtered subgroup of the crowd-sourced texture data items to generate the textured shared world model of the physical environment;

providing the textured shared world model of the physical environment to a display program for display by the display device; and

locating the hologram within the textured shared world model of the physical environment.

10. The method of claim 9 , wherein the temporal filter parameter corresponds to a predated time that is earlier than a current time.

11. The method of claim 10 , further comprising:

filtering the plurality of crowd-sourced structural data items by the temporal filter parameter to generate a temporally-filtered subgroup of the crowd-sourced structural data items; and

wherein stitching together the plurality of crowd-sourced structural data items comprises stitching together the temporally-filtered subgroup of the crowd-sourced structural data items to generate the 3D spatial shared world model of the physical environment.

12. The method of claim 9 , further comprising receiving the plurality of crowd-sourced texture data items from at least a user of the display device and a third party source.

13. The method of claim 9 , wherein each of the plurality of crowd-sourced texture data items are selected from the group consisting of light illuminating one or more of the crowd-sourced structural data items, an image of an object in the physical environment, a video of an object in the physical environment, and audio of the physical environment.

14. The method of claim 9 , further comprising prioritizing the plurality of crowd-sourced structural data items according to one or more of a fidelity, an age, a trustworthiness of a source, and a social proximity of the source of the crowd-sourced structural data items.

15. The method of claim 9 , further comprising prioritizing the plurality of crowd-sourced texture data items according to one or more of a trustworthiness of a source and a social proximity of the source of the crowd-sourced texture data items.

16. A mixed reality experience sharing system for presenting a textured shared world model of a physical environment, the textured shared world model including a plurality of crowd-sourced structural data items that are geo-located at the physical environment, and a plurality of crowd-sourced texture data items that are geo-located at the physical environment, the mixed reality experience sharing system comprising:

a computing device comprising a processor; and

a mixed reality experience sharing program executed by the processor of the computing device, the mixed reality experience sharing program configured to:

receive the plurality of crowd-sourced structural data items including depth information;

stitch together the plurality of crowd-sourced structural data items to generate a 3D spatial shared world model of the physical environment;

receive the plurality of crowd-sourced texture data items from a plurality of sources, each of the texture data items comprising a time-stamped image or video, wherein the plurality of crowd-sourced texture data items comprise at least one hologram and metadata describing the hologram;

receive user input of a temporal filter parameter corresponding to a predated time that is earlier than a current time;

filter the plurality of crowd-sourced texture data items by the temporal filter parameter to generate a temporally-filtered subgroup of the crowd-sourced texture data items;

apply to surfaces of the 3D spatial shared world model of the physical environment the temporally-filtered subgroup of the crowd-sourced texture data items to generate the textured shared world model of the physical environment;

provide the textured shared world model of the physical environment to a display program for display by a display device; and

locate the hologram within the textured shared world model of the physical environment.

17. The mixed reality experience sharing system of claim 16 , wherein the mixed reality experience sharing program is further configured to prioritize the plurality of crowd-sourced texture data items according to one or more of a trustworthiness of a source and a social proximity of the source of the crowd-sourced texture data items.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 20, 2015
From: KAMUDA, NICHOLAS; KINNEBREW, PETER TOBIAS
To: MICROSOFT CORPORATION
Reel/Frame 035683/0539 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 9, 2015
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 039025/0454 →
Continuity (1)
Related Publication 20140204077A1 · Jul 24, 2014