Method and device for video analysis based on image correction learning model

1 . A method performed by an apparatus of a vehicle, the method comprising:

via a tuning parameter learning model for image correction, generating, by a processor of the apparatus and based on received video data, a tuning parameter for adjusting image signal processing (ISP) for correcting the received video data;

correcting, based on the tuning parameter, the received video data;

identifying, via a video recognition model, at least one object in at least one image corresponding to the corrected video data; and

controlling, based on the identified at least one object, autonomous driving of the vehicle,

wherein the tuning parameter learning model comprises a regression analysis model in which a factor is differently set according to a video analysis task associated with the corrected video data.

2 . The method of claim 1 , wherein the tuning parameter learning model is trained by using a ground truth parameter,

wherein the ground truth parameter is determined based on an output of the video recognition model,

wherein the output of the video recognition model is based on a result of a video analysis task associated with the corrected video data, and

wherein the output of the video recognition model is generated using learning video data corrected via the ISP.

3 . The method of claim 2 , wherein the ground truth parameter comprises a tuning parameter that is determined by a performance evaluation index for evaluating the result of the video analysis task.

4 . The method of claim 2 , wherein the tuning parameter for adjusting the ISP comprises a plurality of tuning parameters for adjusting the ISP, and

wherein the corrected learning video data comprises a plurality pieces of corrected learning video data generated based on a combination of the plurality of tuning parameters.

5 . The method of claim 4 , wherein the ground truth parameter is determined based on a combination of a plurality of tuning parameters applied to the ISP, and

wherein the combination of a plurality of tuning parameters applied to the ISP corresponds to an optimal value of a performance evaluation index for evaluating the result of the video analysis task.

6 . The method of claim 1 , wherein the tuning parameter learning model comprises the video recognition model to which a tuning head network configured as a deep learning model is added for generating the tuning parameter.

7 . The method of claim 6 , wherein a weight of the video recognition model is frozen by the tuning parameter learning model, and

wherein the tuning parameter learning model is trained based on a weight of the tuning head network being updated.

8 . The method of claim 6 , wherein the tuning head network has a factor that is differently set according to a video analysis task associated with the corrected video data, and

wherein the received video data is determined based on at least one image frame captured by at least one camera of the vehicle.

9 . The method of claim 6 , wherein the identifying the at least one object comprises:

determining a first video analysis task associated with the corrected video data; and

identifying the at least one object by using the video recognition model via an analysis head network, wherein the analysis head network is configured to provide a result associated with the video recognition model for the first video analysis task.

10 . An apparatus of a vehicle, the apparatus comprising:

a memory storing at least one instruction; and

a processor configured to execute the at least one instruction,

wherein the at least one instruction is configured to cause, when executed by the processor, the apparatus to:

via a tuning parameter learning model for image correction, generate, based on received video data, a tuning parameter for adjusting image signal processing (ISP) for correcting the received video data,

correct, based on the tuning parameter, the received video data,

identify, via a video recognition model, at least one object in at least one image corresponding to the corrected video data, and

control, based on the identified at least one object, autonomous driving of the vehicle,

wherein the tuning parameter learning model comprises a regression analysis model in which a factor is differently set according to a video analysis task associated with the corrected video data.

11 . The apparatus of claim 10 , wherein the at least one instruction is configured to cause, when executed by the processor, the apparatus to:

generate, using learning video data corrected through the ISP, an output of the video recognition model, wherein the output of the video recognition model is based on a result of a video analysis task associated with the corrected video data,

determine, based on the output of the video recognition model, a ground truth parameter, and

train, by using the ground truth parameter, the tuning parameter learning model.

12 . The apparatus of claim 11 , wherein the ground truth parameter comprises a tuning parameter that is determined by a performance evaluation index for evaluating the result of the video analysis task.

13 . The apparatus of claim 11 , wherein the tuning parameter for adjusting the ISP comprises a plurality of tuning parameters for adjusting the ISP, and

wherein the corrected learning video data comprises a plurality pieces of corrected learning video data generated based on a combination of the plurality of tuning parameters.

14 . The apparatus of claim 13 , wherein the ground truth parameter is determined based on a combination of a plurality of tuning parameters applied to the ISP, and

wherein the combination of a plurality of tuning parameters applied to the ISP corresponds to an optimal value of a performance evaluation index for evaluating the result of the video analysis task.

15 . The apparatus of claim 10 , wherein the tuning parameter is configured to be generated and received from a camera that is coupled to the apparatus and is equipped with the regression analysis model.

16 . The apparatus of claim 10 , wherein the tuning parameter learning model comprises the video recognition model to which a tuning head network configured as a deep learning model is added for generating the tuning parameter.

17 . The apparatus of claim 16 , wherein the tuning parameter learning model is configured to freeze a weight of the video recognition model, and

wherein the tuning parameter learning model is trained based on a weight of the tuning head network being updated.

18 . The apparatus of claim 16 , wherein the tuning head network has a factor that is differently set according to a video analysis task associated with the corrected video data, and

wherein the received video data is configured to be determined based on at least one image frame captured by at least one camera of the vehicle.

19 . An apparatus comprising:

a memory storing at least one instruction; and

a processor configured to execute the at least one instruction, wherein the at least one instruction is configured to cause, when executed by the processor, the apparatus to:

via a tuning parameter learning model for image correction, generate, based on received video data, a tuning parameter for adjusting image signal processing (ISP) for correcting the received video data,

correct, based on the tuning parameter, the received video data,

identify, via a video recognition model, at least one object in at least one image corresponding to the corrected video data, and

control, based on the identified at least one object, an operation of a vehicle,

wherein a tuning head network is associated with the tuning parameter learning model for generating the tuning parameter,

wherein the tuning head network has a factor that is differently set according to a video analysis task associated with the corrected video data, and

wherein the received video data is configured to be determined based on at least one image frame captured by at least one camera of the vehicle.

20 . The apparatus of claim 19 , wherein the tuning parameter learning model is configured to freeze a weight of the video recognition model, and

wherein the tuning parameter learning model is trained based on a weight of the tuning head network being updated.