Method for training caption rephrasing model to allow edge-case data to be generated and learning device using the same

1 . A method for training a caption rephrasing model to allow edge-case data to be generated, comprising steps of:

(a) in response to acquiring at least one image caption, inputting, by a learning device, the image caption to the caption rephrasing model to thereby allow the caption rephrasing model to generate a first rephrased caption to an n-th rephrased caption, wherein n is an integer equal to or larger than 2;

(b) inputting, by the learning device, each of the first rephrased caption to the n-th rephrased caption to an image generating model, to thereby allow the image generating model to generate a first generated image and a first pseudo-label corresponding to the first rephrased caption to an n-th generated image and an n-th pseudo-label corresponding to the n-th rephrased caption, and generating, by the learning device, a first identification loss of an identification model for the first generated image to an n-th identification loss of the identification model for the n-th generated image; and

(c) fine-tuning, by the learning device, the caption rephrasing model through a preference learning using one of (i) a preference data set generated by referring to the first identification loss to the n-th identification loss and (ii) a group normalized reward generated by referring to the first identification loss to the n-th identification loss.

2 . The method of claim 1 , wherein, at the step of (b), the learning device inputs each of the first generated image to the n-th generated image to the identification model to thereby allow the identification model to generate a first identification result acquired from identifying the first generated image to an n-th identification result acquired from identifying the n-th generated image, and the learning device generates the first identification loss to the n-th identification loss, wherein the first identification loss is generated by referring to the first identification result and the first pseudo-label, and wherein the n-th identification loss is generated by referring to the n-th identification result and the n-th pseudo-label.

3 . The method of claim 1 , wherein, at the step of (c), the learning device performs one of (i) a sub-process of selecting a preferred rephrased caption and a non-preferred rephrased caption among the first rephrased caption to the n-th rephrased caption by referring to the first identification loss to the n-th identification loss to thereby generate the preference data set, generating a first preference loss by using at least one of the preferred rephrased caption and the non-preferred rephrased caption of the preference data set, and fine-tuning the caption rephrasing model by referring to the first preference loss and (ii) a sub-process of normalizing the first identification loss to the n-th identification loss by using an average and a standard deviation of the first identification loss to the n-th identification loss to thereby generate the group normalized reward, generating a 2_1-st preference loss corresponding to the first rephrased caption to a 2_n-th preference loss corresponding to the n-th rephrased caption by using the group normalized reward, and fine-tuning the caption rephrasing model by referring to the 2_1-st preference loss to the 2_n-th preference loss.

4 . The method of claim 1 , wherein, at the step of (c), the learning device performs fine-tuning on the caption rephrasing model through any one of (i) a full fine-tuning technique, which updates respective parameters of intermediate layers of the caption rephrasing model and (ii) a PEFT (Parameter Efficient Fine Tuning) technique, which updates parameters of at least one specific intermediate layer among intermediate layers of the caption rephrasing model.

5 . The method of claim 4 , wherein, in case the caption rephrasing model is fine-tuned through the PEFT (Parameter Efficient Fine Tuning) technique, with a low rank adaptor added in parallel to the specific intermediate layer,

wherein, at the step of (a), in order for the learning device to input the image caption to the caption rephrasing model to thereby allow the caption rephrasing model to rephrase the image caption and thus to generate the first rephrased caption to the n-th rephrased caption, the learning device allows the caption rephrasing model to input a prior hidden state of a prior intermediate layer positioned at a front end of the specific intermediate layer respectively to the specific intermediate layer and the low rank adaptor, to thereby concatenate a result of applying an operation of the specific intermediate layer to the prior hidden state and a result of applying an operation of the low rank adaptor to the prior hidden state and thus generate a specific hidden state corresponding to the specific intermediate layer, and

wherein, at the step of (c), in a backpropagation using preference losses according to the preference learning, the learning device only allows parameters of the low rank adaptor to be updated, and thus updates parameters of the specific intermediate layer by using the updated parameters of the low rank adaptor.

6 . The method of claim 5 , wherein the specific intermediate layer includes a specific intermediate layer parameter matrix of d×k dimension, and the low rank adaptor includes a first low rank parameter matrix of d×r dimension and a second low rank parameter matrix of r×k dimension, wherein d and k are respective integers equal to or larger than 1, and r is an integer equal to or larger than 1 while being smaller than d and k,

wherein the learning device performs inner product of the updated first low rank parameter matrix and the updated second low rank parameter matrix to thereby generate a reference parameter matrix of d×k dimension, and updates the specific intermediate layer parameter matrix by using the reference parameter matrix.

7 . The method of claim 1 , wherein, at the step of (a), the learning device inputs at least one original image to a captioning model to thereby allow the captioning model to perform captioning on the original image and thus to generate an image caption for the original image.

8 . The method of claim 7 , wherein the original image is an image annotated with at least one ground truth label corresponding to the original image.

9 . The method of claim 1 , wherein each of the first pseudo-label to the n-th pseudo-label includes class information and position information for at least one object in each of the first generated image to the n-th generated image.

10 . A learning device for training a caption rephrasing model to allow edge-case data to be generated, comprising:

at least one memory which saves instructions for training the caption rephrasing model to allow the edge-case data to be generated; and

at least one processor configured to execute an operation for training the caption rephrasing model to allow the edge-case data to be generated according to the instructions saved in the memory to perform processes of: (I) in response to acquiring at least one image caption, inputting the image caption to the caption rephrasing model to thereby allow the caption rephrasing model to generate a first rephrased caption to an n-th rephrased caption, wherein n is an integer equal to or larger than 2; (II) inputting each of the first rephrased caption to the n-th rephrased caption to an image generating model, to thereby allow the image generating model to generate a first generated image and a first pseudo-label corresponding to the first rephrased caption to an n-th generated image and an n-th pseudo-label corresponding to the n-th rephrased caption, and generating a first identification loss of an identification model for the first generated image to an n-th identification loss of the identification model for the n-th generated image; and (III) fine-tuning the caption rephrasing model through a preference learning using one of (i) a preference data set generated by referring to the first identification loss to the n-th identification loss and (ii) a group normalized reward generated by referring to the first identification loss to the n-th identification loss.

11 . The learning device of claim 10 , wherein, at the process of (II), the processor inputs each of the first generated image to the n-th generated image to the identification model to thereby allow the identification model to generate a first identification result acquired from identifying the first generated image to an n-th identification result acquired from identifying the n-th generated image, and the processor generates the first identification loss to the n-th identification loss, wherein the first identification loss is generated by referring to the first identification result and the first pseudo-label, and wherein the n-th identification loss is generated by referring to the n-th identification result and the n-th pseudo-label.

12 . The learning device of claim 10 , wherein, at the process of (III), the processor performs one of (i) a sub-process of selecting a preferred rephrased caption and a non-preferred rephrased caption among the first rephrased caption to the n-th rephrased caption by referring to the first identification loss to the n-th identification loss to thereby generate the preference data set, generating a first preference loss by using at least one of the preferred rephrased caption and the non-preferred rephrased caption of the preference data set, and fine-tuning the caption rephrasing model by referring to the first preference loss and (ii) a sub-process of normalizing the first identification loss to the n-th identification loss by using an average and a standard deviation of the first identification loss to the n-th identification loss to thereby generate the group normalized reward, generating a 2_1-st preference loss corresponding to the first rephrased caption to a 2_n-th preference loss corresponding to the n-th rephrased caption by using the group normalized reward, and fine-tuning the caption rephrasing model by referring to the 2_1-st preference loss to the 2_n-th preference loss.

13 . The learning device of claim 10 , wherein, at the process of (III), the processor performs fine-tuning on the caption rephrasing model through any one of (i) a full fine-tuning technique, which updates respective parameters of intermediate layers of the caption rephrasing model and (ii) a PEFT (Parameter Efficient Fine Tuning) technique, which updates parameters of at least one specific intermediate layer among intermediate layers of the caption rephrasing model.

14 . The learning device of claim 13 , wherein, in case the caption rephrasing model is fine-tuned through the PEFT (Parameter Efficient Fine Tuning) technique, with a low rank adaptor added in parallel to the specific intermediate layer,

wherein, at the process of (I), in order for the processor to input the image caption to the caption rephrasing model to thereby allow the caption rephrasing model to rephrase the image caption and thus to generate the first rephrased caption to the n-th rephrased caption, the processor allows the caption rephrasing model to input a prior hidden state of a prior intermediate layer positioned at a front end of the specific intermediate layer respectively to the specific intermediate layer and the low rank adaptor, to thereby concatenate a result of applying an operation of the specific intermediate layer to the prior hidden state and a result of applying an operation of the low rank adaptor to the prior hidden state and thus generate a specific hidden state corresponding to the specific intermediate layer, and

wherein, at the process of (III), in a backpropagation using preference losses according to the preference learning, the processor only allows parameters of the low rank adaptor to be updated, and thus updates parameters of the specific intermediate layer by using the updated parameters of the low rank adaptor.

15 . The learning device of claim 14 , wherein the specific intermediate layer includes a specific intermediate layer parameter matrix of d×k dimension, and the low rank adaptor includes a first low rank parameter matrix of d×r dimension and a second low rank parameter matrix of r×k dimension, wherein d and k are respective integers equal to or larger than 1, and r is an integer equal to or larger than 1 while being smaller than d and k,

wherein the processor performs inner product of the updated first low rank parameter matrix and the updated second low rank parameter matrix to thereby generate a reference parameter matrix of d×k dimension, and updates the specific intermediate layer parameter matrix by using the reference parameter matrix.

16 . The learning device of claim 10 , wherein, at the process of (I), the processor inputs at least one original image to a captioning model to thereby allow the captioning model to perform captioning on the original image and thus to generate an image caption for the original image.

17 . The learning device of claim 16 , wherein the original image is an image annotated with at least one ground truth label corresponding to the original image.

18 . The learning device of claim 10 , wherein each of the first pseudo-label to the n-th pseudo-label includes class information and position information for at least one object in each of the first generated image to the n-th generated image.