Systems and methods for encoding and decoding video with memory-efficient prediction mode selection
A method of memory-efficient prediction mode selection includes receiving, by an encoder, a coded bitstream including a current frame, determining, by the encoder, costs of a first prediction mode and a second prediction mode, wherein determining further comprises determining, for the first prediction mode, a first bit cost and a first memory cost and determining, for the second prediction mode, a second bit cost and a second memory cost, selecting, by the encoder, a current prediction mode of the first prediction mode and the second prediction mode as a function of the first bit cost, first memory cost, second bit cost, and second memory cost, and encoding, by the encoder, the current frame using the current prediction mode. The prediction mode may be informed by at least one parameter received from the decoder.
1 . A method of encoding features of a video signal in a system for machine video consumption comprising:
receiving an input video signal;
performing feature extraction on the received input video signal to generate a feature signal comprising a least one feature in a current video frame;
determining costs of a first prediction mode and a second prediction mode, wherein determining further comprises:
determining, for the first prediction mode, a first bit cost and a first memory cost associated with encoding the feature signal; and
determining, for the second prediction mode, a second bit cost and a second memory cost associated with encoding the feature signal;
selecting a current prediction mode of the first prediction mode and the second prediction mode as a function of the first bit cost, first memory cost, second bit cost, and second memory cost; and
encoding the feature signal of the current frame using the current prediction mode.
2 . The encoding method of claim 1 , wherein determining the first memory cost further comprises retrieving a stored value representing the first memory cost.
3 . The encoding method of claim 1 , wherein determining the second memory cost further comprises retrieving a stored value representing the second memory cost.
4 . The encoding method of claim 1 , wherein determining the first memory cost further comprises receiving processor architecture data from a decoder, and determining the first memory cost from the processor architecture data.
5 . The encoding method of claim 1 , wherein determining the second memory cost further comprises receiving processor architecture data from a decoder, and determining the second memory cost from the processor architecture data.
6 . The encoding method of claim 1 , wherein selecting further comprises determining a threshold value based on the first memory cost and the second memory cost, and selecting as a function of the threshold value.
7 . The encoding method of claim 6 , wherein selecting further comprises comparing a difference between the first bit cost and the second bit cost to the threshold value.
8 . The encoding method of claim 1 , wherein the first prediction mode is vertical intra prediction and the second prediction mode is horizontal intra prediction.
9 . The encoding method of claim 8 , wherein selecting a current prediction mode includes determining a difference in bit cost between vertical intra prediction and horizontal intra prediction and selecting vertical intra prediction when the difference is less than a predetermined threshold value.
10 . The encoding method of claim 1 , wherein feature extraction is performed using a partial convolutional neural network trained using a machine model for a machine task performed at a decoder site.
11 . The encoding method of claim 10 , wherein encoding the feature signal is performed using a VVC compliant encoding method.