IP Library › Granted Patent US 11,055,916
Granted Patent B2
US 11,055,916 · App. 16/231,035 · Granted Jul 6, 2021

Virtualizing content

Inventors: Clarence Chui (Los Altos Hills, CA); Marc Mignard (San Jose, CA)
Assignee: Outward, Inc.
G06T19/006G06K9/00201G06T11/00H04N5/23229H04N7/0125H04N21/234318H04N21/4223H04N21/8126
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,055,916
App. No.
16/231,035
Granted
Jul 6, 2021
Kind
B2
Abstract

Techniques for virtualizing content are disclosed. One or more objects comprising source video content are determined. The one or more objects comprising the source video content are virtualized by mapping each to and representing each with a corresponding database object. Data comprising the corresponding database objects is provided for rendering the source video content instead of any original pixel information of the source video content so that a virtualized version of the source video content is rendered.

Claims (47)

1. A method, comprising:

identifying elements in source content comprising a scene having one or more objects;

mapping identified elements in the source content to corresponding existing database objects including mapping an identified element in the source content that comprises a partial object to a whole or complete existing database object by matching the partial object to the whole or complete existing database object;

virtualizing the source content by representing the source content completely using existing database objects to which identified elements are mapped, wherein in the virtualized version of the source content the partial object in the source content is represented by the whole or complete existing database object to which the partial object is mapped; and

storing a specification of the virtualized version of the source content as a representation of the source content for distribution instead of the original source content;

wherein the virtualized version of the source content comprises a re-imagined version of the source content that does not include original pixel information of the source content, wherein the virtualized version of the source content comprises an automatically augmented version of the source content that includes content that was not present in the original source content but that is included in the virtualized version of the source content to enhance viewing experience, and wherein the specification of the virtualized version of the source content is provided instead of the source content in response to a request for the source content in order to reduce communication bandwidth because providing the specification of the virtualized version of the source content uses less communication bandwidth than providing the source content.

2. The method of claim 1 , further comprising determining camera pose with respect to the source content and providing the same camera pose with respect to the virtualized version of the source content.

3. The method of claim 1 , wherein a specification of an existing database object comprises a mesh of vertices and their connectivities.

4. The method of claim 1 , wherein a specification of an existing database object comprises surface normal vectors.

5. The method of claim 1 , wherein mapping comprises selecting an existing database object that best matches an identified element.

6. The method of claim 1 , wherein mapping comprises purposely selecting a mismatching existing database object for an identified element.

7. The method of claim 1 , wherein a virtual space comprising the virtualized version of the source content comprises a larger space than that encompassed by the source content.

8. The method of claim 1 , further comprising tracking and mapping motion of an identified element to motion of the corresponding existing database object in the virtualized version of the source content.

9. The method of claim 1 , wherein the source content is two-dimensional and the virtualized version of the source content is three-dimensional.

10. The method of claim 1 , wherein a specification of an existing database object comprises a model of optical behavior of that object.

11. A system, comprising:

a processor configured to:

identify elements in source content comprising a scene having one or more objects;

map identified elements in the source content to corresponding existing database objects including mapping an identified element in the source content that comprises a partial object to a whole or complete existing database object by matching the partial object to the whole or complete existing database object;

virtualize the source content by representing the source content completely using existing database objects to which identified elements are mapped, wherein in the virtualized version of the source content the partial object in the source content is represented by the whole or complete existing database object to which the partial object is mapped; and

store a specification of the virtualized version of the source content as a representation of the source content for distribution instead of the original source content;

wherein the virtualized version of the source content comprises a re-imagined version of the source content that does not include original pixel information of the source content, wherein the virtualized version of the source content comprises an automatically augmented version of the source content that includes content that was not present in the original source content but that is included in the virtualized version of the source content to enhance viewing experience, and wherein the specification of the virtualized version of the source content is provided instead of the source content in response to a request for the source content in order to reduce communication bandwidth because providing the specification of the virtualized version of the source content uses less communication bandwidth than providing the source content; and

a memory coupled to the processor and configured to provide the processor with instructions.

12. The system of claim 11 , wherein the processor is further configured to determine camera pose with respect to the source content and provide the same camera pose with respect to the virtualized version of the source content.

13. The system of claim 11 , wherein a specification of an existing database object comprises a mesh of vertices and their connectivities.

14. The system of claim 11 , wherein to map comprises to select an existing database object that best matches an identified element.

15. The system of claim 11 , wherein to map comprises to purposely select a mismatching existing database object for an identified element.

16. The system of claim 11 , wherein a virtual space comprising the virtualized version of the source content comprises a larger space than that encompassed by the source content.

17. The system of claim 11 , wherein the processor is further configured to track and map motion of an identified element to motion of the corresponding existing database object in the virtualized version of the source content.

18. The system of claim 11 , wherein the source content is two-dimensional and the virtualized version of the source content is three-dimensional.

19. The system of claim 11 , wherein a specification of an existing database object comprises surface normal vectors.

20. A computer program product embodied in a non-transitory computer usable storage medium, comprising computer instructions for:

identifying elements in source content comprising a scene having one or more objects;

mapping identified elements in the source content to corresponding existing database objects including mapping an identified element in the source content that comprises a partial object to a whole or complete existing database object by matching the partial object to the whole or complete existing database object;

virtualizing the source content by representing the source content completely using existing database objects to which identified elements are mapped, wherein in the virtualized version of the source content the partial object in the source content is represented by the whole or complete existing database object to which the partial object is mapped; and

storing a specification of the virtualized version of the source content as a representation of the source content for distribution instead of the original source content;

wherein the virtualized version of the source content comprises a re-imagined version of the source content that does not include original pixel information of the source content, wherein the virtualized version of the source content comprises an automatically augmented version of the source content that includes content that was not present in the original source content but that is included in the virtualized version of the source content to enhance viewing experience, and wherein the specification of the virtualized version of the source content is provided instead of the source content in response to a request for the source content in order to reduce communication bandwidth because providing the specification of the virtualized version of the source content uses less communication bandwidth than providing the source content.

21. The system of claim 11 , wherein a specification of an existing database object comprises a model of optical behavior of that object.

22. The computer program product of claim 20 , further comprising computer instructions for determining camera pose with respect to the source content and providing the same camera pose with respect to the virtualized version of the source content.

23. The computer program product of claim 20 , wherein a specification of an existing database object comprises a mesh of vertices and their connectivities.

24. The computer program product of claim 20 , wherein a specification of an existing database object comprises surface normal vectors.

25. The computer program product of claim 20 , wherein mapping comprises selecting an existing database object that best matches an identified element.

26. The computer program product of claim 20 , wherein mapping comprises purposely selecting a mismatching existing database object for an identified element.

27. The computer program product of claim 20 , wherein a virtual space comprising the virtualized version of the source content comprises a larger space than that encompassed by the source content.

28. The computer program product of claim 20 , further comprising computer instructions for tracking and mapping motion of an identified element to motion of the corresponding existing database object in the virtualized version of the source content.

29. The computer program product of claim 20 , wherein the source content is two-dimensional and the virtualized version of the source content is three-dimensional.

30. The computer program product of claim 20 , wherein a specification of an existing database object comprises a model of optical behavior of that object.

Continuity (4)
Continuation 14741295 · Jun 16, 2015
Continuation 14069040 · Oct 31, 2013
Provisional Application 61720857 · Oct 31, 2012
Related Publication 20190221042A1 · Jul 18, 2019