Device for processing image and method for operating same using neural network-based compression ratio prediction

1 . A method of processing an image, the method comprising:

obtaining a feature map for detecting a target object in an image;

predicting a compression ratio for recognizing the target object in the image via a neural network model by inputting the image and the feature map into the neural network model; and

generating a bitstream by compressing the image based on the compression ratio and encoding the compressed image,

wherein the neural network model that determines a compression ratio for recognizing a reference target object in a training image, is trained based on

a degree of overlap between a first region of interest (ROI) including a detected reference target object in the training image and a second ROI including the detected reference target object in a reconstructed image, the degree of overlap being calculated based on first position coordinate values of a bounding box of the first ROI in the training image and second position coordinate values of a bounding box of the second ROI in the reconstructed image, and

a degree of similarity between a first feature vector extracted from the reference target object in the training image and a second feature vector extracted from the reference target object in the reconstructed image.

2 . The method of claim 1 , wherein the neural network model is trained using training data that comprise a plurality of training images, a plurality of feature maps extracted from the plurality of training images, and a plurality of ground-truth compression ratios for recognizing reference target objects respectively in the plurality of training images.

3 . The method of claim 2 , wherein

the training data is generated by:

generating the reconstructed image by compressing the training image at a preset compression ratio, encoding the compressed training image, and decoding the encoded compressed training image, for each of the plurality of training images;

performing an objection detection operation to obtain the detected reference target object in the training image and extracting the first feature vector from the detected reference target object;

performing the object detection operation to obtain the detected reference target object in the reconstructed image and extracting the second feature vector from the detected reference target object in the reconstructed image; and

determining a ground-truth compression ratio for recognizing the detected reference target object in the training image, based on the degree of overlap between the first ROI and the second ROI, and the degree of similarity between the first feature vector and the second feature vector.

4 . The method of claim 3 , wherein

the determining of the ground-truth compression ratio comprises:

calculating the degree of overlap between the first ROI and the second ROI;

calculating the degree of similarity between the first feature vector and the second feature vector; and

comparing the calculated degree of overlap with a preset first threshold and comparing the calculated degree of similarity with a preset second threshold.

5 . The method of claim 4 , wherein the determining of the ground-truth compression ratio comprises changing the compression ratio to be lower than the preset compression ratio when the degree of overlap is less than or equal to the first threshold or the degree of similarity is less than or equal to the second threshold as a result of the comparing.

6 . The method of claim 4 , wherein the determining of the ground-truth compression ratio comprises determining the compression ratio used to generate the reconstructed image as a final compression ratio when the degree of overlap exceeds the first threshold and the degree of similarity exceeds the second threshold.

7 . The method of claim 3 , wherein the training data is generated further by:

obtaining the plurality of feature maps and information about a plurality of maximum compression ratios; and

storing, in a database, the plurality of training images, the plurality of feature maps, and the information about the plurality of maximum compression ratios.

8 . The method of claim 1 , wherein the neural network model is trained using a training data storage, in which a plurality of first ROIs from a plurality of raw images, a plurality of feature maps from the plurality of raw images, and a plurality of compression ratio values that are determined based on the degree of overlap and the degree of similarity are stored in association with each corresponding raw image.

9 . The method of claim 8 , wherein, in the training data storage, the plurality of first ROIs, the plurality of feature maps, and the plurality of compression ratio values are stored as key-value pairs, and

each key is composed of a corresponding raw image and a corresponding feature map, and each value is a corresponding compression ratio associated with said corresponding raw image.

10 . A device for processing an image, the device comprising:

at least one camera;

a communication interface configured to connect the device to a server or another device for data communication;

a memory storing a program including one or more instructions; and

a processor configured to execute the one or more instructions of the program stored in the memory to:

receive an image via the at least one camera or the communication interface;

obtain a feature map for detecting a target object in the received image;

predict a compression ratio for recognizing the target object in the image via a neural network model by inputting the image and the feature map to the neural network model; and

generate a bitstream by compressing the image using the compression ratio and encoding the compressed image,

wherein the neural network model that determines a compression ratio for recognizing a reference target object in a training image, is trained based on

a degree of overlap between a first region of interest (ROI) including a detected reference target object in the training image and a second ROI including the detected reference target object in a reconstructed image, the degree of overlap being calculated based on first position coordinate values of a bounding box of the first ROI in the training image and second position coordinate values of a bounding box of the second ROI in the reconstructed image, and

a degree of similarity between a first feature vector extracted from the reference target object in the training image and a second feature vector extracted from the reference target object in the reconstructed image.

11 . The device of claim 10 , wherein the neural network model is trained using training data that comprise a plurality of training images, a plurality of feature maps extracted from the plurality of training images, and a plurality of ground-truth compression ratios for recognizing reference target objects respectively in the plurality of training images.

12 . The device of claim 11 , wherein model parameters of the neural network model are obtained via a training process based on a loss between a plurality of predicted compression ratios and the plurality of ground-truth compression ratios for the plurality of training images.

13 . The device of claim 11 , wherein

the processor is further configured to:

generate the reconstructed image by compressing the training image at a preset compression ratio, encoding the compressed training image, and decoding the encoded compressed training image, for each of the plurality of training images;

performing an objection detection operation to obtain the detected reference target object in the training image and extract the first feature vector from the detected reference target object;

performing the object detection operation to obtain the detected reference target object in the reconstructed image and extract the second feature vector from the detected reference target object in the reconstructed image; and

determine a ground-truth compression ratio for recognizing the detected reference target object in the training image, based on the degree of overlap between the first ROI and the second ROI, and the degree of similarity between the first feature vector and the second feature vector.

14 . The device of claim 13 , wherein

the processor is further configured to

calculate the degree of overlap between the first ROI and the second ROI,

calculate the degree of similarity between the first feature vector and the second feature vector, and

compare the calculated degree of overlap with a preset first threshold and compare the calculated degree of similarity with a preset second threshold.

15 . The device of claim 14 , wherein the processor is further configured to change the compression ratio to be lower than the preset compression ratio when the degree of overlap is less than or equal to the first threshold or the degree of similarity is less than or equal to the second threshold as a result of the comparing.

16 . The device of claim 14 , wherein the processor is further configured to determine the compression ratio used to generate the reconstructed image as a final compression ratio when the degree of overlap exceeds the first threshold and the degree of similarity exceeds the second threshold.

17 . A non-transitory computer-readable storage medium storing a program that is executable by at least one processor to perform an image processing method comprising:

obtaining a feature map for detecting a target object in an image;

outputting a compression ratio for recognizing the target object in the image by inputting the image and the feature map to a neural network model composed of pre-trained model parameters; and

generating a bitstream by compressing the image using the output compression ratio and encoding the compressed image,

wherein the neural network model that determines a compression ratio for recognizing a reference target object in a training image, is trained based on

a degree of overlap between a first region of interest (ROI) including a detected reference target object in the training image and a second ROI including the detected reference target object in a reconstructed image, the degree of overlap being calculated based on first position coordinate values of a bounding box of the first ROI in the training image and second position coordinate values of a bounding box of the second ROI in the reconstructed image, and

a degree of similarity between a first feature vector extracted from the reference target object in the training image and a second feature vector extracted from the reference target object in the reconstructed image.