IP Library Granted Patent US 11,651,575
Granted Patent B2
US 11,651,575 · App. 17/718,295 · Granted May 16, 2023

Virtual work of expression within a virtual environment

Inventor: Stephen Constantinides (Baltimore, MD)
Assignee: You Map Inc.
G06T19/006G06V40/174
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,651,575
App. No.
17/718,295
Granted
May 16, 2023
Kind
B2
Abstract

A system and method for displaying a virtual three-dimensional environment, including: displaying at least a portion of the virtual three-dimensional environment in a head-mounted display, where the head-mounted display includes a retinal tracking device; enabling a user to interact with the virtual three-dimensional environment; displaying a virtual representation of a written work within the virtual three-dimensional environment in the head-mounted display; enabling the user to interact with the virtual representation of the written work; tracking the user's reading position in the representation of the written work via the retinal tracking device; determining the content of the representation of the written book at the user's reading position; and modifying the virtual three-dimensional environment based at least in part on the content at the user's reading position.

Claims (70)

1. A wearable device for augmented reality, comprising:

a computer processor;

a movement sensor;

a hardware display component configured to:

overlay at least a portion of a virtual three-dimensional environment onto a real-world environment; and

display a virtual representation of a narrative in the virtual three-dimensional environment, wherein the narrative comprises a literary component;

a content engine executing on the computer processor and configured to:

receive a selection of the narrative by a first user of the wearable device;

launch the virtual representation of the narrative using the hardware display component in response to the selection;

receive an indication of a physical movement of the first user by the movement sensor;

determine a user interaction with the virtual representation of the narrative based on the physical movement of the first user;

perform a transformation of the virtual representation in response to the user interaction, wherein the transformation comprises advancing a position of the narrative.

2. The system of claim 1 , wherein the content engine is further configured to:

connect multiple users to the virtual three-dimensional environment in a single session;

display a first avatar representing the first user and a second avatar representing a second user in the virtual three-dimensional environment;

enable the first user and the second user to communicate using text and audio for at least the duration of the single session.

3. The system of claim 1 , wherein the content engine is further configured to:

identify metadata associated with the narrative;

identify and display a content item of the narrative in the virtual representation;

dynamically retrieve a plurality of context data associated with the content item from at least one external data provider; and

display at least a portion of the plurality of context data in the virtual representation.

4. The system of claim 3 , wherein the narrative is a movie and the content item is a location, and wherein the context data comprises geographical information about the location.

5. The system of claim 1 , wherein the content engine is further configured to:

advance and regress the position of the narrative in response to user interactions; and

enable multiple user interactions with the virtual three-dimensional environment at a single position in the narrative.

6. The system of claim 1 , wherein the narrative comprises multiple storylines, and wherein the content engine is further configured to:

receive, from the user, selection of a storyline decision at the position of the narrative; and

advance the position of the narrative along a storyline corresponding to the storyline decision.

7. The system of claim 1 , wherein the narrative is an instructional narrative for at least one selected from a group consisting of: (i) performing a medical procedure, (ii) performing repair of a physical object, and (iii) assembling a physical apparatus.

8. The system of claim 1 , wherein the narrative is a supplementary guide for an artistic endeavor, and wherein the position of the narrative represents a timeline of the creative endeavor.

9. The system of claim 1 , further comprising a haptic feedback device configured to provide sensory feedback corresponding to one or more events of the narrative as the first user advances the position of the narrative.

10. The system of claim 1 , wherein the narrative is a book, and wherein the content engine is further configured to overlay a text content of the book in the virtual representation.

11. A method for augmented reality, comprising:

receiving a selection of a narrative by a first user of a wearable device;

overlaying, by the wearable device, at least a portion of a virtual three-dimensional environment onto a real-world environment;

displaying a virtual representation of the narrative in the virtual three-dimensional environment, wherein the narrative comprises a literary component;

launching the virtual representation of the narrative using a hardware display component of the wearable device in response to the selection;

receiving an indication of a physical movement of the first user by a movement sensor of the wearable device;

determining a user interaction with the virtual representation of the narrative based on the physical movement of the first user; and

performing a transformation of the virtual representation in response to the user interaction, wherein the transformation comprises advancing a position of the narrative.

12. The method of claim 11 , further comprising:

connecting multiple users to the virtual three-dimensional environment in a single session;

displaying a first avatar representing the first user and a second avatar representing a second user in the virtual three-dimensional environment;

enabling the first user and the second user to communicate using text and audio for at least the duration of the single session.

13. The method of claim 11 , further comprising:

identifying metadata associated with the narrative;

identifying and displaying a content item of the narrative in the virtual representation;

dynamically retrieving a plurality of context data associated with the content item from at least one external data provider; and

displaying at least a portion of the plurality of context data in the virtual representation.

14. The method of claim 13 , wherein the narrative is a movie and the content item is a location, and wherein the context data comprises geographical information about the location.

15. The method of claim 11 , further comprising:

advancing and regressing the position of the narrative in response to user interactions; and

enabling multiple user interactions with the virtual three-dimensional environment at a single position in the narrative.

16. The method of claim 11 , wherein the narrative comprises multiple storylines, and wherein the method further comprises:

receiving, from the user, selection of a storyline decision at the position of the narrative; and

advancing the position of the narrative along a storyline corresponding to the storyline decision.

17. The method of claim 11 , wherein the narrative is an instructional narrative for at least one selected from a group consisting of: (i) performing a medical procedure, (ii) performing repair of a physical object, and (iii) assembling a physical apparatus.

18. The method of claim 11 , wherein the narrative is a supplementary guide for an artistic endeavor, and wherein the position of the narrative represents a timeline of the creative endeavor.

19. A non-transitory computer-readable storage medium comprising a plurality of instructions for augmented reality, the plurality of instructions configured to execute on at least one computer processor to enable the at least one computer processor to:

receive a selection of a narrative by a first user of a wearable device;

overlaying, by the wearable device, at least a portion of a virtual three-dimensional environment onto a real-world environment;

display a virtual representation of the narrative in the virtual three-dimensional environment, wherein the narrative comprises a literary component;

launch the virtual representation of the narrative using a hardware display component of the wearable device in response to the selection;

receive an indication of a physical movement of the first user by a movement sensor of the wearable device;

determine a user interaction with the virtual representation of the narrative based on the physical movement of the first user; and

perform a transformation of the virtual representation in response to the user interaction, wherein the transformation comprises advancing a position of the narrative.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the plurality of instructions are configured to execute on the at least one computer processor to further enable the at least one computer processor to:

connect multiple users to the virtual three-dimensional environment in a single session;

display a first avatar representing the first user and a second avatar representing a second user in the virtual three-dimensional environment;

enable the first user and the second user to communicate using text and audio for at least the duration of the single session.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 27, 2026
From: CONSTANTINIDES, STEPHEN
To: YOU MAP INC.
Reel/Frame 075402/0074 →
Continuity (7)
Continuation 17063635 · Oct 5, 2020
Continuation 16252706 · Jan 20, 2019
Continuation 15587128 · May 4, 2017
Continuation 15007162 · Jan 26, 2016
Provisional Application 62109533 · Jan 29, 2015
Provisional Application 62107313 · Jan 23, 2015
Related Publication 20220292787A1 · Sep 15, 2022