IP Library Granted Patent US 11,494,994
Granted Patent B2
US 11,494,994 · App. 17/075,159 · Granted Nov 8, 2022

Apparatus, method, and system for presentation of multimedia content including augmented reality content

Inventors: Leon Chen (Austin, TX); Brian Keffer (Austin, TX); Jocelyn Seever (Austin, TX); Greg Tamargo (Austin, TX)
Assignee: TIFF'S TREATS HOLDINGS, INC.
G06T19/006G06T5/002G06T5/20G06T7/70G06T15/50H04M1/72439G06Q50/28G06T2207/10016G06T2207/30204G06T2207/30244H04N5/232933
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,494,994
App. No.
17/075,159
Granted
Nov 8, 2022
Kind
B2
Abstract

The present disclosure relates to methods, devices, and systems for presentation of multimedia content. In an aspect of the present disclosure, a method includes receiving, at a processor of a mobile device, a plurality of images from a camera, at least one image of the plurality of images including an object. The method also includes displaying a camera video stream based on the plurality of images, displaying three dimensional (3D) augmented reality content in the camera video content, and displaying video content based on the 3D augmented reality content. The video content is associated with the object.

Claims (93)

1. A method comprising:

receiving, at a processor of a mobile device, video content including a plurality of frames;

displaying, at a display of the mobile device after receiving the video content, a camera video stream generated by an image capture device of the mobile device, the camera video stream including an object;

positioning, based on a coordinate system that is based on a detected marker in the camera video stream, a two dimensional (2D) virtual object on a surface of the object;

positioning, based on a camera pose, a second virtual object adjacent to the 2D virtual object and between the image capture device and a virtual axis that extends through an opening in the 2D virtual object; and

presenting a portion of a first augmented reality (AR) object in the camera video stream based on the coordinate system such that the portion of the first AR object is occluded at least by the second virtual object.

2. The method of claim 1 , further comprising:

displaying a first virtual object in the camera video stream, the first virtual object associated with presentation of a version of the video content in the camera video stream;

presenting a virtual light source associated with the first virtual object, the virtual light source positioned to correspond with presentation of the version of the video content; and

presenting the version of the video content based on the first virtual object and concurrently with the virtual light source;

wherein:

the object comprises a real world object;

the first virtual object comprises a film projector or a monitor; and/or

the version of the video content comprises the video content or a modified version of the video content.

3. The method of claim 2 , further comprising:

displaying a third virtual object in the camera video stream;

wherein, when the first virtual object comprises the film projector:

the virtual light source is positioned at the first virtual object;

the third virtual object comprises a projector screen; and

the version of the video content is projected from the film projector toward the third virtual object.

4. The method of claim 1 , wherein:

the 2D virtual object comprises a first 2D virtual plane that is coplanar with the surface of the object; and

the second virtual object comprises a second 2D virtual plane that includes a first edge that is coplanar with the first 2D virtual plane.

5. The method of claim 1 , wherein:

the portion of the first AR object comprises an occluded portion of the first AR object that is occluded by the 2D virtual object and the second virtual object;

a second portion of the first AR object comprises a non-occluded portion of the first AR object that is not occluded by the 2D virtual object or the second virtual object;

the second portion is positioned in a first direction from the 2D virtual object;

the portion is positioned in a second direction from the 2D virtual object that is opposite to the first direction; and

a field of view of the image capture device is directed at least partially in the second direction from the 2D virtual object.

6. The method of claim 1 , further comprising:

determining the camera pose associated with the image capture device;

determining the opening in the 2D virtual object; and

determining the portion of the first AR object based on the second virtual object or a combination of the 2D virtual object, the opening, and the second virtual object.

7. The method of claim 1 , further comprising:

based on the camera pose, determining first positional information associated with the 2D virtual object and second positional information associated with the second virtual object;

performing a comparison based on AR positional information of the first AR object, the first positional information, and the second positional information;

identifying the portion of the first AR object as overlapping with the 2D virtual object or the second virtual object based on the comparison; and

occluding the portion of the first AR object.

8. The method of claim 1 , further comprising:

positioning, based on the coordinate system and based on the opening, the virtual axis that extends through the opening and perpendicular to the surface of the object; and

based on movement of the image capture device to a second camera pose, rotating the second virtual object about the virtual axis such that the second virtual object remains between the image capture device and the virtual axis.

9. The method of claim 1 , further comprising:

determining a first portion of a second AR object and a second portion of the second AR object based at least on the 2D virtual object, the opening, or both; and

wherein:

the first portion of the second AR object comprises a non-occluded portion of the second AR object;

the second portion of the second AR object comprises an occluded portion of the second AR object; and

the portion of the first AR object of the first AR object is determined based on the second AR object, the 2D virtual object, the opening, the second virtual object, or a combination thereof.

10. The method of claim 9 , wherein:

the second portion of the first AR object comprises a non-occluded portion of the first AR object that is not occluded by the 2D virtual object, the second virtual object, and the second AR object, and

the portion of the first AR object comprises an occluded portion of the first AR object that is occluded by the 2D virtual object, the second virtual object, or the second AR object,

the first AR object comprises a three-dimensional (3D) AR character, and

the second AR object comprises a 3D AR object extending from the opening.

11. The method of claim 1 , wherein positioning the second virtual object comprises interposing the second virtual object between the image capture device and the virtual axis.

12. A device comprising:

an image capture device;

a processor coupled to the image capture device; and

a memory storing one or more instructions that, when executed by the processor, cause the processor to:

receive a plurality of images from the image capture device, at least one image of the plurality of images including an object;

initiate display of a camera video stream based on the plurality of images;

position, based on a coordinate system, a two dimensional (2D) virtual object on a surface of the object;

position, based on a camera pose, a second virtual object adjacent to the 2D virtual object and between the image capture device and a virtual axis that extends through an opening in the 2D virtual object; and

display, in the camera video stream, portion of a first augmented reality (AR) object based on the coordinate system such that the portion of the first AR object is occluded at least by the second virtual object.

13. The device of claim 12 , further comprising:

a display coupled to the processor; and

wherein the image capture device comprises a camera.

14. The device of claim 12 , wherein the one or more instructions, when executed by the processor, further cause the processor to:

receive video content from a server;

display, in the camera video stream, the video content; and

initiate storage at the memory of the camera video stream presented via the display, wherein the video content comprises personalized video data submitted to the server by a third party.

15. The device of claim 12 , wherein:

the portion comprises an occluded portion of the first AR object that is occluded by the second virtual object and not the 2D virtual object;

a second portion of the first AR object comprises a non-occluded portion of the first AR object;

the second portion is positioned in a first direction from the 2D virtual object;

the portion is positioned in a second direction from the 2D virtual object that is opposite to the first direction; and

a field of view of the image capture device is directed entirely in the second direction from the 2D virtual object.

16. The device of claim 12 , further comprising a network interface coupled to the processor, wherein the network interface comprises:

a transmitter coupled to the processor, the transmitter configured to transmit one or more wireless signals; and/or

a receiver coupled to the processor, the receiver configured to receive one or more wireless signals.

17. A computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

displaying, at a display of a mobile device, a camera video stream generated by an image capture device of the mobile device, the camera video stream including an object;

positioning, based on a coordinate system that is based on a detected marker in the camera video stream, a first virtual object, at least a portion of the first virtual object extending along at least a majority of a first surface of the object;

positioning, based on a camera pose, a second virtual object adjacent to the first virtual object and between the image capture device and a virtual axis that extends through an opening in the first virtual object; and

displaying a portion of a first augmented reality (AR) object in the camera video stream such that the portion of the first AR object is occluded at least by the second virtual object.

18. The computer-readable storage medium of claim 17 , wherein the operations further comprise:

determining the camera pose of the image capture device; and

determining the portion of the first AR object based on the second virtual object or a combination of the first virtual object and the second virtual object,

wherein the portion of the first AR object comprises an entirety of the first AR object.

19. The computer-readable storage medium of claim 17 , wherein wherein the instructions further comprise:

displaying, in a first frame of the camera video stream associated with the camera pose, a non-occluded portion of the first AR object at a first time; and

displaying, in a second frame of the camera video stream associated with a second camera pose, the non-occluded portion of the first AR object at a second time, the non-occluded portion of the first AR object at the second time being larger than the non-occluded portion of the first AR object at the first time.

20. The computer-readable storage medium of claim 17 , wherein the operations further comprise:

determining the opening in the first virtual object, the first virtual object comprising a first two dimensional (2D) virtual plane that is coplanar with a surface of the object, wherein the second virtual object comprises a second 2D virtual plane that includes a first edge that is coplanar with the first 2D virtual plane; and

rotating, based on a change in the camera pose, the second 2D virtual plane about the virtual axis such that the second 2D virtual plane remains between the image capture device and the virtual axis after the change in the camera pose.

Assignments (4)
SECURITY INTEREST Recorded Jan 19, 2026
From: TIFF’S TREATS HOLDINGS, INC.
To: CIP ADMINISTRATIVE, LLC
Reel/Frame 074426/0227 →
RELEASE OF SECURITY INTEREST Recorded Jan 15, 2026
From: SALEM INVESTMENT PARTNERS IV, LIMITED PARTNERSHIP
To: TIFF'S TREATS HOLDINGS, INC.
Reel/Frame 073482/0455 →
SECURITY INTEREST Recorded Jun 28, 2024
From: TIFF'S TREATS HOLDINGS, INC.
To: SALEM INVESTMENT PARTNERS IV, LIMITED PARTNERSHIP
Reel/Frame 067965/0792 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 20, 2021
From: CHEN, LEON; KEFFER, BRIAN; SEEVER, JOCELYN; TAMARGO, GREG
To: TIFF'S TREATS HOLDINGS, INC.
Reel/Frame 054972/0651 →
Continuity (4)
Continuation 16404104 · May 6, 2019
Provisional Application 62732938 · Sep 18, 2018
Provisional Application 62676748 · May 25, 2018
Related Publication 20210043006A1 · Feb 11, 2021
Cited By (1)
US 12,647,507