IP Library Granted Patent US 10,528,998
Granted Patent B2
US 10,528,998 · App. 15/950,716 · Granted Jan 7, 2020

Systems and methods for presenting information related to products or services being shown on a second display device on a first display device using augmented reality technology

Inventor: Joel LaMontagne (Huntington Beach, CA)
Assignee: Trivver, Inc.
G06Q30/0623G06Q30/0643G06T19/006H04L67/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,528,998
App. No.
15/950,716
Filed
Apr 11, 2018
Granted
Jan 7, 2020
Kind
B2
Art Unit
2613
USPC
345/633
Abstract

Systems and methods for presenting information, related to products or services being shown on a second display device, on a first display device using augmented reality technology are disclosed. Exemplary implementations may: receive a request, by a server, from a processing system coupled to the first display device, to display information related to a product or service on an augmented reality marker present in a real-time or near real-time image generated or captured in a viewpoint of an image capturing system; determine the product or service being shown on the second display device; and transmit information related to the product or service.

Claims (51)

1. A system comprising:

one or more hardware processors configured by machine-readable instructions to:

receive a request, by a server, from a processing system coupled to a first display device to display information related to a product or service on an augmented reality marker present in a real-time or near real-time image captured in a viewpoint of an image capturing system, wherein the processing system detects feature descriptors of the real-time or near real-time image to identify the augmented reality marker using at least one of corner detection, blob detection, edge detection or thresholding algorithm;

determine a timestamp value at which the request is received from the processing system coupled to the first display device;

determine another timestamp value related to a product or service scheduled to be displayed on a show or channel on the second displaying device; and

transmit information related to the product or service to the first display device when the another timestamp value is near or equal to the timestamp value;

wherein the processing system augments the real-time or near real-time image comprising the augmented reality marker with information related to the product or service.

2. The system of claim 1 , wherein the augmented reality marker is identified at a location at which brightness of the real-time or near real-time image changes sharply or has discontinuities, and wherein the processing system displays the information related to the product or service, including a three dimensional (3D) artifact, at the augmented reality marker on the first display device.

3. The system of claim 2 , wherein displaying the 3D artifact on the first display device includes:

receive the real-time or near real-time image from the image capturing system;

determine the augmented reality marker by at least one of determining occurrences of gradient orientation in localized portions of the real-time or near real-time image or identifying points in the real time or near real-time image; and

receive the 3D artifact, the 3D artifact including metadata to indicate a center point of the 3D artifact and a rendering location with reference to the augmented reality marker;

wherein the 3D artifact is augmented at the augmented reality marker.

4. The system of claim 3 , wherein the center point of the 3D artifact is determined using a two dimensional grid of coordinates at the location of the 3D artifact.

5. The system of claim 3 , wherein the rendering location can be expressed in a coordinate system of the first display device, and wherein the metadata further comprises a position and orientation of the 3D artifact in the coordinate system of the first display device using matrix transformation.

6. The system of claim 1 , wherein to determine the product or service being shown on the second display device, the one or more hardware processors are further configured by machine-readable instructions to:

retrieve a catalog related to the show or program channel, the catalog comprising details about the products or services being displayed on the second display device; and

identify the product or service being displayed on the second display device.

7. A method, comprising:

receiving a request, by a server, from a processing system coupled to the first display device, to display information related to a product or service on an augmented reality marker present in a real-time or near real-time image captured in a viewpoint of an image capturing system, wherein the processing system detects feature descriptors of the real-time or near real-time image to identify the augmented reality marker using at least one of corner detection, blob detection, edge detection or thresholding algorithm;

determining a timestamp value at which the request is received from the processing system coupled to the first display device;

determining another timestamp value related to a product or service scheduled to be displayed on a show or channel on the second displaying device; and

transmitting information related to the product or service to the first display device when the another timestamp value is near or equal to the timestamp value;

wherein the processing system augments the image comprising the augmented reality marker with information related to the product or service and displays the information related to the product or service.

8. The method of claim 7 , wherein the augmented reality marker is identified at a location at which brightness of the real-time or near real-time image changes sharply or has discontinuities, and wherein the processing system displays the information related to the product or service, including a three dimensional (3D) artifact, at the augmented reality marker on the first display device.

9. The method of claim 8 , wherein displaying the 3D artifact on the first display device includes:

receiving the real-time or near real-time image from the image capturing system;

determining the augmented reality marker by at least one of determining occurrences of gradient orientation in localized portions of the real-time or near real-time image or identifying points in the real time or near real-time image; and

receiving the 3D artifact, the 3D artifact including metadata to indicate a center point of the 3D artifact and a rendering location with reference to the augmented reality marker;

wherein the 3D artifact is augmented at the augmented reality marker.

10. The method of claim 9 , wherein the center point of the 3D artifact is determined using a two dimensional grid of coordinates at the location of the 3D artifact.

11. The method of claim 9 , wherein the rendering location can be expressed in a coordinate system of the first display device, and wherein the metadata further comprises a position and orientation of the 3D artifact in the coordinate system of the first display device using matrix transformation.

12. The method of claim 7 , wherein determining the product or service being shown on the second display device comprises:

retrieving a catalog related to the show or program channel, the catalog comprising details about the products or services being displayed on the second display device; and

identifying the product or service being displayed on the second display device.

13. A non-transitory computer-readable storage medium comprising instructions which when executed by one or more processors performs a method comprising:

receiving a request from a processing system coupled to the first display device, to display information related to a product or service on an augmented reality marker present in a real-time or near real-time image captured in a viewpoint of an image capturing system, wherein the processing system detects feature descriptors of the real-time or near real-time image to identify the augmented reality marker using at least one of corner detection, blob detection, edge detection or thresholding algorithm;

determining a timestamp value at which the request is received from the processing system coupled to the first display device;

determining another timestamp value related to a product or service scheduled to be displayed on a show or channel on the second displaying device; and

transmitting information related to the product or service to the first display device;

wherein the processing system augments the image comprising the augmented reality marker with information related to the product or service and displays the information related to the product or service, and

wherein the method is performed by a server computer.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the augmented reality marker is identified at a location at which brightness of the real-time or near real-time image changes sharply or has discontinuities, and wherein the processing system displays the information related to the product or service, including a three dimensional (3D) artifact, at the augmented reality marker on the first display device.

15. The non-transitory computer-readable storage medium of claim 13 , wherein displaying the 3D artifact on the first display device includes:

receiving the real-time or near real-time image from the image capturing system;

determining the augmented reality marker by at least one of determining occurrences of gradient orientation in localized portions of the real-time or near real-time image or identifying points in the real time or near real-time image; and

receiving the 3D artifact, the 3D artifact including metadata to indicate a center point of the 3D artifact and a rendering location with reference to the augmented reality marker, wherein the 3D artifact is augmented at the augmented reality marker.

16. The non-transitory computer-readable storage medium of claim 13 , wherein the image capturing system is capable of capturing a color image data to an image sensor using at least one of a CCD or CMOS, and a filter for obtaining a color image.

17. The non-transitory computer-readable storage medium of claim 13 , wherein determining the product or service being shown on the second display device comprises:

retrieving a catalog related to the show or program channel, the catalog comprising details about the products or services being displayed on the second display device; and

identifying the product or service being displayed on the second display device.

Assignments (1)
SECURITY INTEREST Recorded Sep 7, 2022
From: TRIVVER, INC.
To: NEXT WORLD TRUST; BURKHALTER FAMILY TRUST
Reel/Frame 061019/0015 →
Continuity (1)
Related Publication 20190318404A1 · Oct 17, 2019