Channel-aware semantic coding

1. A method performed by one or more processors, the method comprising:

determining a quality level for a channel for a time period in which a video frame is being transmitted over the channel;

determining, based on the quality level, one or more semantic elements to include in a semantic transcript stream (STS);

encoding the video frame with the one or more semantic elements of the STS; and

preparing the encoded video frame for transmission to a remote device.

2. The method of claim 1 , wherein determining the quality level comprises:

monitoring channel features comprising one or more of a size of upload grants, a measurement for a handover, or a reference symbol; and

generating a prediction of an upload data rate for transmitting the video frame.

3. The method of claim 1 , wherein determining one or more semantic elements to include comprises determining a number of annotations for extracting from the video frame.

4. The method of claim 1 , wherein determining one or more semantic elements to include comprises determining an amount of bandwidth available for semantic elements;

assigning each semantic element a priority; and

including higher priority semantic elements until the amount of bandwidth available is exhausted.

5. The method of claim 1 , further comprising:

assigning a privacy marker to the one or more semantic elements, the privacy marker requiring the remote device to have a corresponding permission to access a semantic element of the one or more semantic elements.

6. The method of claim 5 , wherein the semantic element comprises an identifier of a person or object.

7. The method of claim 5 , further comprising performing end-to-end encryption of the one or more semantic elements associated with the privacy marker.

8. The method of claim 1 , wherein the determined channel quality level is a predicted channel quality level, the method further comprising:

receiving an actual channel quality level at a time for transmitting the encoded video frame;

determining that the actual channel quality level has a reduced quality relative to the predicted channel quality level;

removing one or more semantic elements from the encoded video frame, based on one or more priority rules or by changing one or more configurable frame rates for semantic elements; and

preparing the encoded video frame for transmission without the removed one or more semantic elements.

9. The method of claim 1 , wherein the encoded video frame comprises data of the STS only without video frame data.

10. The method of claim 1 , wherein the encoded video frame comprises data of the STS with a full compressed video frame or by attaching one or more portions of the video frame.

11. One or more processors configured

to perform operations comprising:

determining a quality level for a channel for a time period in which a video frame is being transmitted over the channel;

determining, based on the quality level, one or more semantic elements to include in a semantic transcript stream (STS);

encoding the video frame with the one or more semantic elements of the STS; and

preparing the encoded video frame for transmission to a remote device.

12. The one or more processors of claim 11 , wherein determining the quality level comprises:

monitoring channel features comprising one or more of a size of upload grants, a measurement for a handover, or a reference symbol; and

generating a prediction of an upload data rate for transmitting the video frame.

13. The one or more processors of claim 11 , wherein determining one or more semantic elements to include comprises determining a number of annotations for extracting from the video frame.

14. The one or more processors of claim 11 , wherein determining one or more semantic elements to include comprises determining an amount of bandwidth available for semantic elements;

assigning each semantic element a priority; and

including higher priority semantic elements until the amount of bandwidth available is exhausted.

15. The one or more processors of claim 11 , the operations further comprising:

assigning a privacy marker to the one or more semantic elements, the privacy marker requiring the remote device to have a corresponding permission to access a semantic element of the one or more semantic elements.

16. The one or more processors of claim 15 , wherein the semantic element comprises an identifier of a person or object.

17. The one or more processors of claim 15 , further comprising performing end-to-end encryption of the one or more semantic elements associated with the privacy marker.

18. The one or more processors of claim 11 , wherein the determined channel quality level is a predicted channel quality level, the operations further comprising:

receiving an actual channel quality level at a time for transmitting the encoded video frame;

determining that the actual channel quality level has a reduced quality relative to the predicted channel quality level;

removing one or more semantic elements from the encoded video frame, based on one or more priority rules or by changing one or more configurable frame rates for semantic elements; and

preparing the encoded video frame for transmission without the removed one or more semantic elements.

19. The one or more processors of claim 11 , wherein the encoded video frame comprises data of the STS only without video frame data.

20. The one or more processors of claim 11 , wherein the encoded video frame comprises data of the STS with a full compressed video frame or by attaching one or more portions of the video frame.