IP Library Granted Patent US 11,277,658
Granted Patent B1
US 11,277,658 · App. 17/408,065 · Granted Mar 15, 2022

Integrating overlaid digital content into displayed data via graphics processing circuitry

Inventors: Dharmendra Etwaru (Sparta, NJ); Mike Sutcliff (Inlet Beach, FL)
Assignee: BEAM, INC.
H04N21/4316G06F16/783G06F16/7867G06T1/20H04N21/8146H04N21/8352
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,277,658
App. No.
17/408,065
Granted
Mar 15, 2022
Kind
B1
Abstract

An apparatus, method, and computer readable medium that include accessing a frame buffer of a graphics processing unit (GPU), analyzing, in the frame buffer of the GPU, a frame representing a section of a stream of displayed data that is being displayed by an apparatus, identifying a reference patch that includes a unique identifier associated with an available area in which secondary digital content is insertable in the displayed data that is being displayed by the apparatus, decoding the encoded data of the unique identifier, retrieving the secondary digital content from the remote device based on the unique identifier, and overlaying the secondary digital content into the displayed data in accordance with the available area, the screen position, and the size identified by the unique identifier.

Claims (61)

1. An apparatus, comprising:

processing circuitry, including a graphics processing unit (GPU), configured to

access a frame buffer of the GPU,

analyze, in the frame buffer of the GPU, a frame representing a section of a stream of displayed data that is being displayed by the apparatus,

based on the analyzed frame, identify a reference patch that includes a unique identifier associated with an available area in which secondary digital content is insertable in the displayed data that is being displayed by the apparatus, the unique identifier including encoded data that identifies the secondary digital content, a location address of the secondary digital content at a remote device, a screen position within the available area at which the secondary digital content is insertable in the displayed data, and a size of the secondary digital content when inserted in the displayed data,

decode the encoded data of the unique identifier,

after decoding the encoded data of the unique identifier, retrieve the secondary digital content from the remote device based on the unique identifier, and

after retrieving the secondary digital content from the remote device, overlay the secondary digital content into the displayed data in accordance with the available area, the screen position, and the size identified by the unique identifier.

2. The apparatus of claim 1 , wherein the processing circuitry is configured to analyze the frame by

processing attributes of each pixel included in the frame, and

detecting groups of pixels included in the frame that have a predetermined pattern of pixel luma and chroma manipulation.

3. The apparatus of claim 1 , wherein the processing circuitry is configured to identify the reference patch by

detecting a predetermined pattern of pixel luma and chroma manipulation included in the reference patch, the predetermined pattern corresponding to a size of the available area, and

detecting a predetermined edging pattern of pixel luma and chroma manipulation included in the reference patch, the predetermined edging pattern corresponding to a perimeter of the available area.

4. The apparatus of claim 3 , wherein the processing circuitry is configured to, prior to overlaying the secondary digital content into the displayed data, adjust, based on the predetermined pattern and the predetermined edging pattern, the size and a perimeter of the secondary digital content.

5. The apparatus of claim 1 , wherein the processing circuitry is configured to

retrieve metadata associated with the secondary digital content, and

set, based on the metadata, an access control parameter indicating a geographic location, a user, a date and time, a duration, or a type of display for which the secondary digital content is viewable when overlaid into the displayed data.

6. The apparatus of claim 1 , wherein the processing circuitry is configured to identify the reference patch based on a confidence level, the confidence level being high when a predetermined pattern of pixel luma and chroma manipulation and a predetermined edging pattern of pixel luma and chroma manipulation are detected in the reference patch.

7. A method, comprising:

accessing a frame buffer of a graphics processing unit (GPU);

analyzing, in the frame buffer of the GPU, a frame representing a section of a stream of displayed data that is being displayed by an apparatus;

based on the analyzed frame, identifying a reference patch that includes a unique identifier associated with an available area in which secondary digital content is insertable in the displayed data that is being displayed by the apparatus, the unique identifier including encoded data that identifies the secondary digital content, a location address of the secondary digital content at a remote device, a screen position within the available area at which the secondary digital content is insertable in the displayed data, and a size of the secondary digital content when inserted in the displayed data;

decoding the encoded data of the unique identifier;

after decoding the encoded data of the unique identifier, retrieving the secondary digital content from the remote device based on the unique identifier; and

after retrieving the secondary digital content from the remote device, overlaying the secondary digital content into the displayed data in accordance with the available area, the screen position, and the size identified by the unique identifier.

8. The method of claim 7 , wherein the analyzing of the frame further comprises

processing attributes of each pixel included in the frame; and

detecting groups of pixels included in the frame that have a predetermined pattern of pixel luma and chroma manipulation.

9. The method of claim 7 , wherein the identifying of the reference patch further comprises

detecting a predetermined pattern of pixel luma and chroma manipulation included in the reference patch, the predetermined pattern corresponding to a size of the available area; and

detecting a predetermined edging pattern of pixel luma and chroma manipulation included in the reference patch, the predetermined edging pattern corresponding to a perimeter of the available area.

10. The method of claim 9 , further comprising prior to overlaying the secondary digital content into the displayed data, adjusting, based on the predetermined pattern and the predetermined edging pattern, the size and a perimeter of the secondary digital content.

11. The method of claim 7 , wherein the identifying of the reference patch is based on a confidence level, the confidence level being high when a predetermined pattern of pixel luma and chroma manipulation and a predetermined edging pattern of pixel luma and chroma manipulation are detected in the reference patch.

12. The method of claim 7 , further comprising

continuously analyzing subsequent frames of the stream; and

adjusting the overlay of the secondary digital content based on changes of the displayed data in the subsequent frames of the stream.

13. The method of claim 7 , further comprising

retrieving metadata associated with the secondary digital content; and

setting, based on the metadata, an access control parameter indicating a geographic location, a user, a date and time, a duration, or a type of display for which the secondary digital content is viewable when overlaid into the displayed data.

14. A non-transitory computer-readable storage medium for storing computer-readable instructions that, when executed by a computer, cause the computer to perform a method, the method comprising:

accessing a frame buffer of a graphics processing unit (GPU);

analyzing, in the frame buffer of the GPU, a frame representing a section of a stream of displayed data that is being displayed by an apparatus;

based on the analyzed frame, identifying a reference patch that includes a unique identifier associated with an available area in which secondary digital content is insertable in the displayed data that is being displayed by the apparatus, the unique identifier including encoded data that identifies the secondary digital content, a location address of the secondary digital content at a remote device, a screen position within the available area at which the secondary digital content is insertable in the displayed data, and a size of the secondary digital content when inserted in the displayed data;

decoding the encoded data of the unique identifier;

after decoding the encoded data of the unique identifier, retrieving the secondary digital content from the remote device based on the unique identifier; and

after retrieving the secondary digital content from the remote device, overlaying the secondary digital content into the displayed data in accordance with the available area, the screen position, and the size identified by the unique identifier.

15. The non-transitory computer-readable storage medium according to claim 14 , wherein the analyzing of the frame further comprises

processing attributes of each pixel included in the frame; and

detecting groups of pixels included in the frame that have a predetermined pattern of pixel luma and chroma manipulation.

16. The non-transitory computer-readable storage medium according to claim 14 , wherein the identifying of the reference patch further comprises

detecting a predetermined pattern of pixel luma and chroma manipulation included in the reference patch, the predetermined pattern corresponding to a size of the available area; and

detecting a predetermined edging pattern of pixel luma and chroma manipulation included in the reference patch, the predetermined edging pattern corresponding to a perimeter of the available area.

17. The non-transitory computer-readable storage medium according to claim 16 , further comprising prior to overlaying the secondary digital content into the displayed data, adjusting, based on the predetermined pattern and the predetermined edging pattern, the size and a perimeter of the secondary digital content.

18. The non-transitory computer-readable storage medium according to claim 14 , wherein the identifying of the reference patch is based on a confidence level, the confidence level being high when a predetermined pattern of pixel luma and chroma manipulation and a predetermined edging pattern of pixel luma and chroma manipulation are detected in the reference patch.

19. The non-transitory computer-readable storage medium according to claim 15 , further comprising

continuously analyzing subsequent frames of the stream; and

adjusting the overlay of the secondary digital content based on changes of the displayed data in the subsequent frames of the stream.

20. The non-transitory computer-readable storage medium according to claim 14 , further comprising

retrieving metadata associated with the secondary digital content; and

setting, based on the metadata, an access control parameter indicating a geographic location, a user, a date and time, a duration, or a type of display for which the secondary digital content is viewable when overlaid into the displayed data.

Assignments (2)
CHANGE OF NAME Recorded Jul 15, 2022
From: BEAM, INC.
To: MOBEUS INDUSTRIES, INC.
Reel/Frame 060678/0149 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 20, 2021
From: ETWARU, DHARMENDRA; SUTCLIFF, MIKE
To: BEAM, INC.
Reel/Frame 057244/0450 →
Continuity (5)
Provisional Application 63213326 · Jun 22, 2021
Provisional Application 63189434 · May 17, 2021
Provisional Application 63182391 · Apr 30, 2021
Provisional Application 63172640 · Apr 8, 2021
Provisional Application 63068878 · Aug 21, 2020