Text extraction to separate encoding of text and images for streaming during periods of low connectivity

1 . A method, comprising:

determining, based on a low-connectivity of a network, to encode a scene of an image frame independent of a text of the image frame rather than encoding the scene and the text together;

processing, using a machine learning model, the image frame to identify a first set of commands of one or more draw calls that are used to render the text in the image frame;

identifying a second set of commands of the one or more draw calls that are used to render the scene in the image frame based on the identification of the first set of commands of the one or more draw calls by the machine learning model;

executing the first set of commands to render the text separately from rendering the scene;

executing the second set of commands to render the scene independent of rendering the text;

encoding the scene that is rendered independent of the text, the text remaining unencoded; and

separately streaming, via the network, to a client device the unencoded text and the encoded scene.

2 . The method of claim 1 , wherein the client device reconstructs the image frame by decoding the encoded scene to generate the scene that is rendered and overlaying the text onto the scene that is rendered.

3 . The method of claim 2 , further comprising:

including timing information with the scene that is rendered; and

including the timing information with the text.

4 . The method of claim 1 , further comprising:

identifying when network connectivity between a server and the client device is below a quality of service (QoS) threshold.

5 . The method of claim 1 , further comprising:

streaming the text raw.

6 . The method of claim 1 , wherein the text is one of:

player status information; and

closed-captioning; and

player communication; and

game generated text; and

chat communication.

7 . The method of claim 1 , further comprising:

rendering a user interface associated with the text that is identified,

wherein the first set of commands is used to render the user interface and the text in combination,

wherein the user interface and the text in combination are streamed to the client device.

8 . The method of claim 7 , wherein the user interface includes one of:

a character status interface; and

a thought bubble; and

a chat forum.

9 . The method of claim 1 , further comprising:

applying an artificial intelligence (AI) model to identify the first set of commands that are used to render the text.

10 . The method of claim 1 , further comprising:

storing the first set of commands in a text command buffer for execution by a first GPU to render the text; and

storing the second set of commands in an image command buffer by a second GPU to render the scene.

11 . A non-transitory computer-readable medium storing a computer program for performing a method, the computer-readable medium comprising:

programming instructions for determining, based on a low-connectivity of a network, to encode a scene of an image frame independent of a text of the image frame rather than encoding the scene and the text together;

program instructions for processing, using a machine learning model, the image frame to identify a first set of commands of one or more draw calls that are used to render the text in the image frame;

program instructions for identifying a second set of commands of the one or more draw calls that are used to render the scene in the image frame based on the identification of the first set of commands of the one or more draw calls by the machine learning model;

program instructions for executing the first set of commands to render the text separately from rendering the scene;

program instructions for executing the second set of commands to render the scene independent of rendering the text;

program instructions for encoding the scene that is rendered independent of the text, the text remaining unencoded; and

program instructions for separately streaming, via the network, to a client device the unencoded text and the encoded scene,

wherein the client device reconstructs the image frame by decoding the encoded scene to generate the scene that is rendered and overlaying the unencoded text onto the scene that is rendered.

12 . The non-transitory computer-readable medium of claim 11 , further comprising:

program instructions for identifying when network connectivity between a server and the client device is below a quality of service (QoS) threshold.

13 . The non-transitory computer-readable medium of claim 11 , wherein in the method the text is one of:

player status information; and

closed-captioning; and

player communication; and

game generated text; and

chat communication.

14 . The non-transitory computer-readable medium of claim 11 , further comprising:

program instructions for rendering a user interface associated with the text that is identified,

wherein the first set of commands is used to render the user interface and the text in combination,

wherein the user interface and the text in combination are streamed to the client device.

15 . The non-transitory computer-readable medium of claim 11 , further comprising:

program instructions for storing the first set of commands in a text command buffer for execution by a first GPU to render the text; and

program instructions for storing the second set of commands in an image command buffer by a second GPU to render the scene.

16 . A computer system comprising:

a processor,

memory coupled to the processor and having stored therein instructions that, if executed by the computer system, cause the computer system to execute a method, comprising:

determining, based on a low-connectivity of a network, to encode a scene of an image frame independent of a text of the image frame rather than encoding the scene and the text together;

processing, using a machine learning model, the image frame to identify a first set of commands of one or more draw calls that are used to render the text in the image frame;

identifying a second set of commands of the one or more draw calls that are used to render the scene in the image frame based on the identification of the first set of commands of the one or more draw calls by the machine learning model;

executing the first set of commands to render the text;

executing the second set of commands to render the scene independent of rendering the text;

encoding the scene that is rendered independent of the text, the text remaining unencoded; and

separately streaming, via the network, to a client device the unencoded text and the encoded scene,

wherein the client device reconstructs the image frame by decoding the encoded scene to generate the scene that is rendered and overlaying the unencoded text onto the scene that is rendered.

17 . The computer system of claim 16 , the method further comprising:

identifying when network connectivity between the server and the client device is below a quality of service (QoS) threshold.

18 . The computer system of claim 16 , wherein in the method the text is one of:

player status information; and

closed-captioning; and

player communication; and

game generated text; and

chat communication.

19 . The computer system of claim 16 , the method further comprising:

rendering a user interface associated with the text that is identified,

wherein the first set of commands is used to render the user interface and the text in combination,

wherein the user interface and the text in combination are streamed to the client device.

20 . The computer system of claim 16 , the method further comprising:

storing the first set of commands in a text command buffer for execution by a first GPU to render the text; and

storing the second set of commands in an image command buffer by a second GPU to render the scene.