IP Library › Granted Patent US 10,417,828
Granted Patent B2
US 10,417,828 · App. 15/689,274 · Granted Sep 17, 2019

Systems and methods for contextually augmented video creation and sharing

Inventors: Gila Kamhi (Zichron Yaakov, IL); Barak Hurwitz (Kibbutz Alonim, IL); Vladimir Kouperman (Haifa, IL); Kobi Nistel (Haifa, IL)
Assignee: INTEL CORPORATION
G06T19/006G06K9/00671
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,417,828
App. No.
15/689,274
Granted
Sep 17, 2019
Kind
B2
Abstract

An augmented reality (AR) device includes a 3D video camera to capture video images and corresponding depth information, a display device to display the video data, and an AR module to add a virtual 3D model to the displayed video data. A depth mapping module generates a 3D map based on the depth information, a dynamic scene recognition and tracking module processes the video images and the 3D map to detect and track a target object within a field of view of the 3D video camera, and an augmented video rendering module renders an augmented video of the virtual 3D model dynamically interacting with the target object. The augmented video is displayed on the display device in real time. The AR device may further include a context module to select the virtual 3D model based on context data comprising a current location of the augmented reality device.

Claims (51)

1. An augmented reality device, comprising:

memory to store video data comprising video images; and

a processor to add a virtual three dimensional (3D) model to the video data, the processor comprising instructions to:

generate a 3D map based on depth information corresponding to the video data;

process the video images and the 3D map to detect and track a physical target object in the video images;

select the virtual 3D model based on a mood of a user and context data comprising a current geographic location of the augmented reality device, wherein the selected virtual 3D model is associated with the current geographic location; and

render an augmented video of the virtual 3D model and the tracked physical target object, the rendered augmented video showing the virtual 3D model initiating and performing an interaction between the virtual 3D model and the tracked physical target object, the interaction being based on the mood of the user.

2. The augmented reality device of claim 1 , wherein the context data further comprises user profile information including one or more data fields comprising birthdate, anniversary, animal preference, historic person preference, current celebrity preference, and genre preference.

3. The augmented reality device of claim 2 , wherein the processor further comprises instructions to select the virtual 3D model based on the user profile information.

4. The augmented reality device of claim 1 , wherein the processor further comprises instructions to select the virtual 3D model based on a mood indicator selected by a user.

5. The augmented reality device of claim 1 , wherein the virtual 3D model comprises:

model context data to associate the virtual 3D model with at least one of a location, person, and event; and

target object data associated with the model context data, the target object data to define a specific object or object type for the processor to detect and track as the target object.

6. The augmented reality device of claim 5 , wherein the virtual 3D model further comprises:

one or more graphic objects to animate in the rendered augmented video; and

interaction rules to define an interaction between the one or more graphic objects and the target object in the rendered augmented video.

7. The augmented reality device of claim 1 , wherein:

the processor further comprises instructions to communicate through a communication network; and

the processor further comprises instructions to, automatically or in response to user input, provide the augmented video through the communication network to a video-sharing service or social media service.

8. A method for rendering augmented video, the method comprising:

detecting and tracking a physical target object in video data comprising video images;

determining context information including a location of the mobile device; and

selecting a virtual three-dimensional (3D) model based on a mood of the user and the context information, wherein the location of the mobile device is provided by an external source;

rendering, based on a 3D map corresponding to depth information associated with the video data, an augmented video of the virtual 3D model and the tracked physical target object, the rendered augmented video showing the virtual 3D model initiating and performing an interaction between the virtual 3D model and the tracked physical target object, the interaction being based on the mood of the user; and

displaying the augmented video in real time on a display device.

9. The method of claim 8 , wherein the context information further includes a user profile comprising one or more data fields including birthdate, anniversary, animal preference, historic person preference, current celebrity preference, and genre preference.

10. The method of claim 9 , further comprising selecting the virtual 3D model based on the user profile.

11. The method of claim 9 , where the context information further includes an indication of a user's mood, and wherein the method further comprises selecting the virtual 3D model based on the indication of the user's mood.

12. The method of claim 8 , wherein the virtual 3D model comprises:

model context data to associate the virtual 3D model with at least one of a location, person, and event; and

target object data associated with the model context data, the target object data to define a specific object or object type to be detected and tracked as the target object.

13. The method of claim 12 , wherein the virtual 3D model further comprises:

one or more graphic objects to animate in the rendered augmented video; and

interaction rules to define an interaction between the one or more graphic objects and the target object in the rendered augmented video.

14. The method of claim 8 , wherein the augmented video comprises an augmented micro video, and wherein the method further comprises automatically communicating the augmented micro video through a communication network to at least one of a video-sharing service and a social media service.

15. At least one non-transitory computer-readable storage medium having stored thereon instructions that, when executed by a processor, cause the processor to perform operations comprising:

detecting and tracking a physical target object in video data comprising video images;

determining context information including a location of the augmented reality device;

selecting a virtual three-dimensional (3D) model based on a mood of a user and the context information;

rendering, based on a 3D map corresponding to depth information associated with the video data, an augmented video of the virtual 3D model and the tracked physical target object, the rendered augmented video showing the virtual 3D model initiating and performing an interaction between the virtual 3D model and the tracked physical target object, the interaction being based on the mood of the user; and

displaying the augmented video in real time.

16. The at least one non-transitory computer-readable storage medium of claim 15 , wherein the context information further includes a user profile comprising one or more data fields including birthdate, anniversary, animal preference, historic person preference, current celebrity preference, and genre preference.

17. The at least one non-transitory computer-readable storage medium of claim 16 , wherein the operations further comprise selecting the virtual 3D model based on the user profile.

18. The least one non-transitory computer-readable storage medium of claim 16 , where the context information further includes an indication of a user's mood, and wherein the operations further comprise selecting the virtual 3D model based on the indication of the user's mood.

19. The at least one non-transitory computer-readable storage medium of claim 15 , wherein the virtual 3D model comprises:

model context data to associate the virtual 3D model with at least one of a location, person, and event; and

target object data associated with the model context data, the target object data to define a specific object or object type to be detected and tracked as the target object.

20. The at least one non-transitory computer-readable storage medium of claim 19 , wherein the virtual 3D model further comprises:

one or more graphic objects to animate in the rendered augmented video; and

interaction rules to define an interaction between the one or more graphic objects and the target object in the rendered augmented video.

21. The at least one non-transitory computer-readable storage medium of claim 15 , wherein the augmented video comprises an augmented micro video, and wherein the operations further comprise automatically communicating the augmented micro video through a communication network to at least one of a video-sharing service and a social media service.

Continuity (2)
Continuation 14581468 · Dec 23, 2014
Related Publication 20180047214A1 · Feb 15, 2018
Cited By (1)
US 12,732,696