IP Library Granted Patent US 11,348,199
Granted Patent B2
US 11,348,199 · App. 16/921,321 · Granted May 31, 2022

Modifying graphics rendering by transcoding a serialized command stream

Inventor: Matthew James Sottek (Austin, TX)
Assignee: Roku, Inc.
G06T1/60G09G5/39
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,348,199
App. No.
16/921,321
Granted
May 31, 2022
Kind
B2
Abstract

Disclosed herein are system, method, and computer program product embodiments for modifying graphics rendering by transcoding a serialized command stream. An embodiment operates by receiving a command configured to instruct an API to render a graphics element. The embodiment further operates by generating, based on the command, a transcoded command configured to instruct the API to render a modified graphics element by applying a set of modification factors to a portion of the command. Subsequently, the embodiment operates by transmitting the transcoded command to the API.

Claims (82)

1. An apparatus, comprising:

a memory; and

at least one processor coupled to the memory and configured to:

receive a first set of commands configured to instruct an application programming interface (API) to render a first graphics element;

receive a second set of commands configured to instruct the API to render a second graphics element;

generate, based on the first set of commands and a first set of modification factors and without storing the first graphics element in an off-screen buffer, a first set of transcoded commands configured to instruct the API to render a first modified graphics element;

generate, based on the second set of commands and a second set of modification factors and without storing the second graphics element in the off-screen buffer, a second set of transcoded commands configured to instruct the API to render a second modified graphics element;

receive a request to generate a frame; and

in response to receipt of the request, generate the frame based on the first set of transcoded commands and the second set of transcoded commands, wherein the frame comprises the first graphics element and the second graphics element.

2. The apparatus of claim 1 , wherein:

the first set of commands were generated by an application and transmitted by the application to the API;

the second set of commands were generated by the application and transmitted by the application to the API; and

the at least one processor is further configured to:

receive the first set of commands after transmission of the first set of commands by the application and before any processing of the first set of commands by the API; and

receive the second set of commands after transmission of the second set of commands by the application and before any processing of the second set of commands by the API.

3. The apparatus of claim 2 , wherein to receive the first set of commands and the second set of commands, the at least one processor is configured to:

intercept the first set of commands after transmission of the first set of commands by the application and before any receipt of the first set of commands by the API; and

intercept the second set of commands after transmission of the second set of commands by the application and before any receipt of the second set of commands by the API.

4. The apparatus of claim 2 , wherein to receive the first set of commands and the second set of commands, the at least one processor is configured to:

capture the first set of commands after receipt of the first set of commands by the API and before any processing of the first set of commands by the API; and

capture the second set of commands after receipt of the second set of commands by the API and before any processing of the second set of commands by the API.

5. The apparatus of claim 2 , the at least one processor further configured to:

prevent receipt of the first set of commands by the API; and

prevent receipt of the second set of commands by the API.

6. The apparatus of claim 1 , the at least one processor further configured to:

generate a serialized set of commands comprising the first set of commands and the second set of commands.

7. The apparatus of claim 6 , the at least one processor further configured to:

generate a serialized set of transcoded commands based on the serialized set of commands,

wherein the serialized set of transcoded commands comprises the first set of transcoded commands and the second set of transcoded commands.

8. The apparatus of claim 1 , wherein:

the first set of commands comprises:

on-screen data associated with the first graphics element; and

off-screen data associated with the first graphics element; and

to generate the first set of transcoded commands, the at least one processor is configured to:

apply the first set of modification factors to the on-screen data associated with the first graphics element; and

not apply the first set of modification factors to the off-screen data associated with the first graphics element.

9. The apparatus of claim 8 , wherein:

the off-screen data comprises source data associated with the first graphics element; and

the on-screen data comprises destination data associated with the first graphics element.

10. The apparatus of claim 1 , wherein:

the first graphics element comprises a rectangle; and

the first modified graphics element comprises a modified rectangle.

11. The apparatus of claim 1 , wherein:

the first graphics element comprises a font; and

the first modified graphics element comprises a modified font.

12. A computer-implemented method for modifying graphics rendering, comprising:

receiving, by at least one processor, a first set of commands configured to instruct an application programming interface (API) to render a first graphics element;

receiving, by the at least one processor, a second set of commands configured to instruct the API to render a second graphics element;

generating, by the at least one processor based on the first set of commands and without storing the first graphics element in an off-screen buffer, a first set of transcoded commands configured to instruct the API to render a first modified graphics element by applying a first set of modification factors to a first portion of the first set of commands;

generating, by the at least one processor based on the second set of commands and without storing the second graphics element in the off-screen buffer, a second set of transcoded commands configured to instruct the API to render a second modified graphics element by applying a second set of modification factors to a second portion of the second set of commands;

receiving, by the at least one processor, a request to generate a frame; and

in response to receiving the request, generating, by the at least one processor, the frame based on the first set of transcoded commands and the second set of transcoded commands, wherein the frame comprises the first graphics element and the second graphics element.

13. The computer-implemented method of claim 12 , wherein:

the first set of commands were generated by an application and transmitted by the application to the API;

the second set of commands were generated by the application and transmitted by the application to the API; and

the receiving further comprises:

receiving, by the at least one processor, the first set of commands after transmission of the first set of commands by the application and before any processing of the first set of commands by the API; and

receiving, by the at least one processor, the second set of commands after transmission of the second set of commands by the application and before any processing of the second set of commands by the API.

14. The computer-implemented method of claim 13 , the receiving further comprising:

intercepting, by the at least one processor, the first set of commands after transmission of the first set of commands by the application and before any receipt of the first set of commands by the API; and

intercepting, by the at least one processor, the second set of commands after transmission of the second set of commands by the application and before any receipt of the second set of commands by the API.

15. The computer-implemented method of claim 13 , the receiving further comprising:

capturing, by the at least one processor, the first set of commands after receipt of the first set of commands by the API and before any processing of the first set of commands by the API; and

capturing, by the at least one processor, the second set of commands after receipt of the second set of commands by the API and before any processing of the second set of commands by the API.

16. The computer-implemented method of claim 13 , further comprising:

preventing, by the at least one processor, receipt of the first set of commands by the API; and

preventing, by the at least one processor, receipt of the second set of commands by the API.

17. The computer-implemented method of claim 12 , further comprising:

generating, by the at least one processor, a serialized set of commands comprising the first set of commands and the second set of commands.

18. The computer-implemented method of claim 17 , further comprising:

generating, by the at least one processor, a serialized set of transcoded commands based on the serialized set of commands,

wherein the serialized set of transcoded commands comprises the first set of transcoded commands and the second set of transcoded commands.

19. A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:

receiving a first set of commands configured to instruct an application programming interface (API) to render a first graphics element;

receiving a second set of commands configured to instruct the API to render a second graphics element;

generating, based on the first set of commands and without storing the first graphics element in an off-screen buffer, a first set of transcoded commands configured to instruct the API to render a first modified graphics element by applying a first set of modification factors to a first portion of the first set of commands;

generating, based on the second set of commands and without storing the second graphics element in the off-screen buffer, a second set of transcoded commands configured to instruct the API to render a second modified graphics element by applying a second set of modification factors to a second portion of the second set of commands;

receiving a request to generate a frame; and

in response to receiving the request, generating the frame based on the first set of transcoded commands and the second set of transcoded commands, wherein the frame comprises the first graphics element and the second graphics element.

20. The non-transitory computer-readable medium of claim 19 , wherein:

the first set of modification factors comprises a first subset of scaling factors; and

the second set of modification factors comprises a second subset of scaling factors.

Assignments (2)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 7, 2020
From: SOTTEK, MATTHEW JAMES
To: ROKU, INC.
Reel/Frame 053138/0994 →
Continuity (1)
Related Publication 20220005149A1 · Jan 6, 2022