IP Library Granted Patent US 11,044,521
Granted Patent B2
US 11,044,521 · App. 16/625,576 · Granted Jun 22, 2021

Modifying digital video content

Inventors: Lakshmi Dabbiru (Sunnyvale, CA); Indusekar Ponnapa Reddy (Fremont, CA)
Assignee: Google LLC
H04N21/44016G11B27/34H04N21/23418H04N21/435H04N21/437H04N21/8126H04N21/8455
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,044,521
App. No.
16/625,576
Granted
Jun 22, 2021
Kind
B2
Abstract

Systems and methods of combining digital video are described. A system can receive digital video comprising frames. The system can analyze the frames to identify a subset of consecutive frames that have a static portion defined by a spatial region with matching pixels across two or more frames. The system can tag the digital video with an indication of the subset and a trigger that causes transmission, at a temporal offset during playback prior to the subset, of a request for content used to overwrite the static portion. The system can receive the request from the client device, the request transmitted by the client device based on the temporal offset during playback prior to the subset. The system can select and transmit a content item responsive to the request to cause the client device to render the content item at the static portion of the subset.

Claims (85)

1. A method of combining digital video content, comprising:

receiving, by a data processing system from a publisher device, a digital video content item comprising a plurality of frames;

determining, by a video signal processor executed by the data processing system, a frame rate of the digital video content item;

analyzing, by the video signal processor, the plurality of frames to identify a first subset of consecutive frames from the plurality of frames having a first static portion, the first static portion defined by a spatial region with matching pixels across two or more frames;

determining, by the video signal processor, based on a number of frames in the first subset of consecutive frames and the frame rate, a first duration of the first static portion;

comparing, by the video signal processor, the first duration of the first static portion with a threshold;

tagging, by the video signal processor responsive to the first duration of the first static portion being greater than or equal to the threshold, the digital video content item with an indication of the first subset of consecutive frames and a trigger that causes transmission, at a temporal offset during playback prior to the first subset of consecutive frames, of a request for content used to overwrite the first static portion;

providing, by the video signal processor to a client device, the digital video content item with the indication and the trigger, receipt of the digital video content item causing the client device to render the digital video content item, parse the indication to identify the first subset of consecutive frames, and transmit, at the temporal offset during playback prior to the first subset of consecutive frames, the request for content responsive to execution of the trigger;

receiving, by the video signal processor, the request for content from the client device, the request transmitted by the client device based on the temporal offset during playback prior to the first subset of consecutive frames;

selecting, by a content selector executed by the data processing system, a content item responsive to the request for content received from the client device; and

transmitting, by the data processing system to the client device, the content item, receipt of the content item causing the client device to render the content item at the first static portion of the first subset of consecutive frames.

2. The method of claim 1 further comprising

analyzing, by the data processing system, the plurality of frames to identify a second static portion in the first subset of consecutive frames;

identifying, by the data processing system, a first parameter associated with the first static portion and a second parameter associated with the second static portion; and

selecting, based on a comparison of the first parameter and the second parameter, the first static portion for tagging.

3. The method of claim 2 , wherein the first parameter comprises at least one of a background color, color gradient, size, or location; and the second parameter comprises at least one of a background color, color gradient, size, or location.

4. The method of claim 1 , further comprising:

analyzing, by the data processing system, the plurality of frames to identify a second static portion in the first subset of consecutive frames;

identifying, by the data processing system, a second duration of the second static portion; and

selecting, by the data processing system, the first static portion for tagging based on the first duration exceeding the second duration.

5. The method of claim 1 , further comprising:

identifying, by the data processing system, a second static portion in the digital video content item;

requesting, by the data processing system from the publisher device, authorization to overwrite the second static portion;

receiving, by the data processing system, a negative authorization from the publisher device for the second static portion; and

locking, by the data processing system, the second static portion to prevent overwriting of the second static portion, comprising;

identifying, by the data processing system, the first static portion based on pixel variation across consecutive frames in the plurality of frames.

6. The method of claim 1 , comprising;

identifying, by the data processing system, the first static portion based on pixel variation across consecutive frames in the plurality of frames.

7. The method of claim 1 , further comprising:

receiving, by the data processing system, an indication of a background color from the publisher device to control identification of the first static portion; and

identifying, by the data processing system using the indication of the background color, the first static portion based on a pixel characteristic across consecutive frames in the plurality of frames.

8. The method of claim 1 , further comprising:

identifying, by the data processing system, one or more characteristics associated with the first static portion, the one or more characteristics comprising at least one of text, font, face, object type, or color; and

selecting, by the data processing system, the content item based on the one or more characteristics associated with the first static portion.

9. The method of claim 1 , further comprising:

identifying, by the data processing system, an amount of the digital video content item that is preloaded into memory of the client device; and

setting, by the data processing system, the temporal offset based at least in part on the amount to cause the client device to transmit the request for content prior to the amount being preloaded into the memory of the client device.

10. The method of claim 1 , further comprising:

merging, by the data processing system, the content item at the first static portion of the first subset of consecutive frames of the digital video content item; and

transmitting, by the data processing system to the client device, the first subset of consecutive frames of the digital video content item merged with the content item to cause the client device to render the content item.

11. A system for combining digital video content, comprising:

a data processing system comprising at least one processor and memory, the data processing system executing a video signal processor and a content selector;

wherein the video signal processor is configured to:

receive, from a publisher device, a digital video content item comprising a plurality of frames;

determine a frame rate of the digital video content item;

analyze the plurality of frames to identify a first subset of consecutive frames from the plurality of frames having a first static portion, the first static portion defined by a spatial region with matching pixels across two or more frames;

determine, based on a number of frames in the first subset of consecutive frames and the frame rate, a first duration of the first static portion;

compare the first duration of the first static portion with a threshold;

tag, responsive to the first duration of the first static portion being greater than or equal to the threshold, the digital video content item with an indication of the first subset of consecutive frames and a trigger that causes transmission, at a temporal offset during playback prior to the first subset of consecutive frames, of a request for content used to overwrite the first static portion; and

provide, to a client device, the digital video content item with the indication and the trigger, receipt of the digital video content item causing the client device to render the digital video content item, parse the indication to identify the first subset of consecutive frames, and transmit, at the temporal offset during playback prior to the first subset of consecutive frames, the request for content responsive to execution of the trigger;

wherein the content selector component is configured to:

receive the request for content from the client device, the request transmitted by the client device based on the temporal offset during playback prior to the first subset of consecutive frames; and

select a content item responsive to the request for content received from the client device; and

the data processing system is further configured to:

transmit, to the client device, the content item, receipt of the content item causing the client device to render the content item at the first static portion of the first subset of consecutive frames.

12. The system of claim 11 , wherein the video signal processor is further configured to:

analyze the plurality of frames to identify a second static portion in the first subset of consecutive frames;

identify a first parameter associated with the first static portion and a second parameter associated with the second static portion; and

select, based on a comparison of the first parameter and the second parameter, the first static portion for tagging.

13. The system of claim 12 , wherein the first parameter comprises at least one of a background color, color gradient, size, or location; and the second parameter comprises at least one of a background color, color gradient, size, or location.

14. The system of claim 11 , wherein the video signal processor is further configured to:

analyze the plurality of frames to identify a second static portion in the first subset of consecutive frames;

identify a second duration of the second static portion; and

select the first static portion for tagging based on the first duration exceeding the second duration.

15. The system of claim 11 , wherein the video signal processor is further configured to:

identify a second static portion in the digital video content item;

request, from the publisher device, authorization to overwrite the second static portion;

receive a negative authorization from the publisher device for the second static portion; and

lock the second static portion to prevent overwriting of the second static portion with content selected by the data processing system.

16. The system of claim 11 , wherein the video signal processor is further configured to:

identify the first static portion based on pixel variation across consecutive frames in the plurality of frames.

17. The system of claim 11 , wherein the video signal processor is further configured to:

receive an indication of a background color from the publisher device to control identification of the first static portion; and

identify, using the indication of the background color, the first static portion based on a pixel characteristic across consecutive frames in the plurality of frames.

18. The system of claim 11 , wherein the content selector is further configured to:

identify one or more characteristics associated with the first static portion, the one or more characteristics comprising at least one of text, font, face, object type, or color; and

select the content item based on the one or more characteristics associated with the first static portion.

19. The system of claim 11 , wherein the video signal processor is further configured to:

identify an amount of the digital video content item that is preloaded into memory of the client device; and

set the temporal offset based on the amount to cause the client device to transmit the request for content prior to the amount being preloaded into the memory of the client device.

20. The system of claim 11 , wherein the video signal processor is further configured to:

merge the content item at the first static portion of the first subset of consecutive frames of the digital video content item; and

transmit, to the client device, the first subset of consecutive frames of the digital video content item merged with the content item to cause the client device to render the content item.

21. The system of claim 11 , wherein the video signal processor is further configured to:

use logistic regression to analyze the plurality of frames to identify the first static portion defined by the spatial region with matching pixels across two or more frames.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 21, 2020
From: DABBIRU, LAKSHMI; REDDY, INDUSEKAR PONNAPA
To: GOOGLE LLC
Reel/Frame 051893/0192 →
Continuity (1)
Related Publication 20210084368A1 · Mar 18, 2021
Cited By (1)
US 12,341,737