IP Library › Granted Patent US 12666021
Granted Patent B2
US 12666021 · App. 18/640,520 · Granted Jun 23, 2026

Determining regions of interest using learned image codec for machines

Inventors: Honglei Zhang (Tampere, FI); Francesco Cricrì (Tampere, FI); Alireza Aminlou (Tampere, FI); Miska Matias Hannuksela (Tampere, FI); Nam Hai Le (Tampere, FI); Jukka Ilari Ahonen (Tampere, FI); Hamed Rezazadegan Tavakoli (Espoo, FI)
Assignee: Nokia Technologies Oy
H04N19/119H04N19/167H04N19/176H04N19/124H04N19/154
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12666021
App. No.
18/640,520
Granted
Jun 23, 2026
Kind
B2
Abstract

Various embodiments describe an apparatus, a method, and a computer program product. An example apparatus includes at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: encoding an input picture by using a first encoder or first encoding parameters; encoding the input picture by using a second encoder or second encoding parameters; generating a first reconstructed picture based on the encoding of the input picture by using the first encoder or the first encoding parameters; and generating a second reconstructed picture based on the encoding of the input picture by using the second encoder or the second encoding parameters.

Claims (101)

1 . An apparatus comprising at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform:

encoding an input picture using a neural network based learned image codec; and

deriving a first reconstructed picture based on encoding;

wherein the input picture is encoded by a neural network learned image codec by using a setting for high quality reconstruction to generate the first reconstructed picture and a first bitstream, and wherein the apparatus is further caused to perform:

encoding the input picture by using the neural network based learned image codec by using a setting for low-quality reconstruction;

generating a second reconstructed picture based on encoding;

generating a second bitstream based on the second reconstructed picture;

partitioning the second reconstructed picture into blocks;

calculating bits per pixel (BPP) values or relative BPP values of a block in the first reconstructed picture and the second reconstructed picture;

calculating a difference of BPP values or a relative difference of the relative BPP values by subtracting a BPP value or a relative BPP value of the second reconstructed picture from the BPP value or the relative BPP value of the first reconstructed picture; and

categorizing the block as a foreground region when the difference of the BPP value or the relative difference of the BPP value is greater than a predefined threshold value and/or categorizing the block as a background region when the difference of the BPP value or the relative difference of the BPP value is smaller than the predefined threshold value.

2 . The apparatus of claim 1 , wherein the apparatus is further caused to perform:

partitioning the first reconstructed picture into blocks;

for each block in the first reconstructed picture, computing a quality measurement of the first reconstructed picture with regard to the input picture;

categorizing the block as a background region when the quality measurement is greater than a threshold value; and/or

categorizing the block as a foreground region when the quality measurement is lower than the threshold value.

3 . The apparatus of claim 2 , wherein the quality measurement of the first reconstructed picture is calculated in a feature domain.

4 . The apparatus of claim 1 , wherein the apparatus is further caused to perform:

deriving or generating a first bitstream based on the first reconstructed picture; and

determining a length of bits used to represent a block from the generated bitstream.

5 . The apparatus of claim 4 , wherein the apparatus is further caused to perform:

calculating bits per pixel (BPP) by dividing the length of bits by a number of pixels in the block;

categorizing the block as a foreground region when the BPP of the block is greater than a predefined threshold;

categorizing the block as a background region when the BPP of the block is smaller than the predefined threshold;

calculating a relative BPP of the block by subtracting a BPP value of the first reconstructed picture from the BPP value of the block;

categorizing the block as the foreground region when the relative BPP of the block is greater than the predefined threshold;

categorizing the block as the background region when the relative BPP of the block is smaller than the predefined threshold;

calculating a BPP ratio by dividing the BPP value of the block by a BPP of first reconstructed picture; and

categorizing the block as the foreground region when the BPP ratio of the block is greater than the predefined threshold and/or categorizing the block as the background region when the BPP ratio of the block is smaller than the predefined threshold.

6 . A method comprising:

encoding an input picture using a neural network based learned image codec; and

deriving a first reconstructed picture based on encoding;

wherein the input picture is encoded by a neural network learned image codec by using a setting for high quality reconstruction to generate the first reconstructed picture and a first bitstream, and wherein the method further comprises:

encoding the input picture by using the neural network based learned image codec by using a setting for low-quality reconstruction;

generating a second reconstructed picture based on encoding;

generating a second bitstream based on the second reconstructed picture;

partitioning the second reconstructed picture into blocks;

calculating bits per pixel (BPP) values or relative BPP values of a block in the first reconstructed picture and the second reconstructed picture; and

calculating a difference of BPP values or a relative difference of the relative BPP values by subtracting a BPP value or a relative BPP value of the second reconstructed picture from the BPP value or the relative BPP value of the first reconstructed picture;

categorizing the block as a foreground region when the difference of the BPP value or the relative difference of the BPP value is greater than a predefined threshold value and/or categorizing the block as a background region when the difference of the BPP value or the relative difference of the BPP value is smaller than the predefined threshold value.

7 . The method of claim 6 further comprising:

partitioning the first reconstructed picture into blocks;

for each block in the first reconstructed picture, computing a quality measurement of the first reconstructed picture with regard to the input picture;

categorizing the block as a background region when the quality measurement is greater than a threshold value; and/or

categorizing the block as a foreground region when the quality measurement is lower than the threshold value.

8 . The method of claim 7 , wherein the quality measurement of the first reconstructed picture is calculated in a feature domain.

9 . The method of claim 6 further comprising:

deriving or generating a first bitstream based on the first reconstructed picture; and

determining a length of bits used to represent a block from the generated bitstream.

10 . The method of claim 9 further comprising:

calculating bits per pixel (BPP) by dividing the length of bits by a number of pixels in the block;

categorizing the block as a foreground region when the BPP of the block is greater than a predefined threshold;

categorizing the block as a background region when the BPP of the block is smaller than the predefined threshold;

calculating a relative BPP of the block by subtracting a BPP value of the first reconstructed picture from the BPP value of the block;

categorizing the block as the foreground region when the relative BPP of the block is greater than the predefined threshold;

categorizing the block as the background region when the relative BPP of the block is smaller than the predefined threshold;

calculating a BPP ratio by dividing the BPP value of the block by a BPP of first reconstructed picture; and

categorizing the block as the foreground region when the BPP ratio of the block is greater than the predefined threshold and/or categorizing the block as the background region when the BPP ratio of the block is smaller than the predefined threshold.

11 . An apparatus comprising at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform:

receiving a first bitstream; and

generating a first reconstructed picture from the first bitstream, wherein an input picture is encoded using a neural network based learned image codec, and wherein the first reconstructed picture is derived based on encoding;

wherein the input picture is encoded by a neural network learned image codec by using a setting for high quality reconstruction to generate the first reconstructed picture and the first bitstream, and wherein the input picture is encoded by using the neural network based learned image codec by using a setting for low-quality reconstruction to generate a second reconstructed picture, and wherein the apparatus is further caused to perform:

receiving a second bitstream comprising the second reconstructed picture;

generating the second reconstructed picture from the second bitstream;

partitioning the second reconstructed picture into blocks;

calculating bits per pixel (BPP) values or relative BPP values of a block in the first reconstructed picture and the second reconstructed picture;

calculating a difference of BPP values or a relative difference of the relative BPP values by subtracting a BPP value or a relative BPP value of the second reconstructed picture from the BPP value or the relative BPP value of the first reconstructed picture; and

categorizing the block as a foreground region when the difference of the BPP value or the relative difference of the BPP value is greater than a predefined threshold value; and/or categorizing the block as a background region when the difference of the BPP value or the relative difference of the BPP value is smaller than the predefined threshold value.

12 . The apparatus of claim 11 , wherein the apparatus is further caused to perform:

partitioning the first reconstructed picture into blocks;

for each block in the first reconstructed picture, computing a quality measurement of the first reconstructed picture with regard to the input picture;

categorizing the block as a background region when the quality measurement is greater than a threshold value; and/or

categorizing the block as a foreground region when the quality measurement is lower than the threshold value.

13 . The apparatus of claim 12 , wherein the quality measurement of the first reconstructed picture is calculated in a feature domain.

14 . The apparatus of claim 11 , wherein the apparatus is further caused to perform:

deriving or generating the first bitstream based on the first reconstructed picture; and

determining a length of bits used to represent a block from the generated bitstream.

15 . The apparatus of claim 14 , wherein the apparatus is further caused to perform:

calculating bits per pixel (BPP) by dividing the length of bits by a number of pixels in the block;

categorizing the block as a foreground region when the BPP of the block is greater than a predefined threshold;

categorizing the block as a background region when the BPP of the block is smaller than the predefined threshold;

calculating a relative BPP of the block by subtracting a BPP value of the first reconstructed picture from the BPP value of the block;

categorizing the block as the foreground region when the relative BPP of the block is greater than the predefined threshold;

categorizing the block as the background region when the relative BPP of the block is smaller than the predefined threshold;

calculating a BPP ratio by dividing the BPP value of the block by a BPP of first reconstructed picture; and

categorizing the block as the foreground region when the BPP ratio of the block is greater than the predefined threshold and/or categorizing the block as the background region when the BPP ratio of the block is smaller than the predefined threshold.

16 . A method comprising:

receiving a first bitstream; and

generating a first reconstructed picture from the first bitstream, wherein an input picture is encoded using a neural network based learned image codec, and wherein the first reconstructed picture is derived based on encoding;

wherein the input picture is encoded by a neural network learned image codec by using a setting for high quality reconstruction to generate the first reconstructed picture and the first bitstream, and wherein the input picture is encoded by using the neural network based learned image codec by using a setting for low-quality reconstruction to generate a second reconstructed picture, and wherein the method further comprises:

receiving a second bitstream comprising the second reconstructed picture;

generating the second reconstructed picture from the second bitstream;

partitioning the second reconstructed picture into blocks;

calculating bits per pixel (BPP) values or relative BPP values of a block in the first reconstructed picture and the second reconstructed picture;

calculating a difference of BPP values or a relative difference of the relative BPP values by subtracting a BPP value or a relative BPP value of the second reconstructed picture from the BPP value or the relative BPP value of the first reconstructed picture; and

categorizing the block as a foreground region when the difference of the BPP value or the relative difference of the BPP value is greater than a predefined threshold value and/or categorizing the block as a background region when the difference of the BPP value or the relative difference of the BPP value is smaller than the predefined threshold value.

17 . The method of claim 16 further comprising:

partitioning the first reconstructed picture into blocks;

for each block in the first reconstructed picture, computing a quality measurement of the first reconstructed picture with regard to the input picture;

categorizing the block as a background region when the quality measurement is greater than a threshold value; and/or

categorizing the block as a foreground region when the quality measurement is lower than the threshold value.