Learning apparatus and learning method
A learning apparatus for performing machine learning includes an acquisition unit configured to acquire teaching data including input data and correct answer data, the input data including an input image that contains a reference object and an input text that relatively designates a target position by referring to the reference object; a generation unit configured to input the input data to a model to generate output data for specifying the target position, a reference position that is a position of the reference object, and a positional relationship of the target position with respect to the reference position; and an update unit configured to update a parameter of the model to reduce a loss obtained by inputting the output data and the correct answer data to a loss function. The loss function is based on at least two errors of a first error between the target position specified by the output data and the target position specified by the correct answer data, a second error between the reference position specified by the output data and the reference position specified by the correct answer data, and a third error between the positional relationship specified by the output data and the positional relationship specified by the correct answer data.
1 . A learning apparatus for performing machine learning, comprising a memory storing instructions and a processor configured to execute the instructions to:
acquire teaching data including input data and correct answer data, the input data including an input image that contains a reference object and an input text that relatively designates a target position by referring to the reference object;
input the input data to a model to generate output data for specifying the target position, a reference position that is a position of the reference object, and a positional relationship of the target position with respect to the reference position; and
update a parameter of the model to reduce a loss obtained by inputting the output data and the correct answer data to a loss function,
wherein the loss function is based on at least two errors of
a first error between the target position specified by the output data and the target position specified by the correct answer data,
a second error between the reference position specified by the output data and the reference position specified by the correct answer data, and
a third error between the positional relationship specified by the output data and the positional relationship specified by the correct answer data, and
wherein the model includes
an image encoding layer for encoding the input image, and
a text encoding layer for encoding the input text,
a part of features determined by the text encoding layer is input to the image encoding layer, and
a part of features determined by the image encoding layer is input to the text encoding layer.
2 . The learning apparatus according to claim 1 , wherein the input image includes an image imaged by a camera of a vehicle.
3 . The learning apparatus according to claim 1 , wherein the input text is expressed by a natural language.
4 . The learning apparatus according to claim 1 , wherein the loss function is based on at least the first error.
5 . The learning apparatus according to claim 1 , wherein the loss function is based on all of the first error, the second error, and the third error.
6 . The learning apparatus according to claim 1 , wherein the image encoding layer and the text encoding layer have an identical layer structure.
7 . The learning apparatus according to claim 1 ,
wherein the machine learning of the model is first machine learning, and
a parameter of the model at a start point in time of training is a parameter of the image encoding layer determined by second machine learning with the input image as input data and a position of the reference object as correct answer data.
8 . A learning apparatus for performing machine learning, comprising a memory storing instructions and a processor configured to execute the instructions to:
acquire teaching data including input data and correct answer data, the input data including an input image that contains a reference object and an input text that relatively designates a target position by referring to the reference object;
generate output data for specifying the target position, a reference position that is a position of the reference object, and a positional relationship of the target position with respect to the reference position by inputting the input data to a model; and
update a parameter of the model to reduce a loss obtained by inputting the output data and the correct answer data to a loss function,
wherein the loss function is based on an error between the positional relationship specified by the output data and the positional relationship specified by the correct answer data, and
wherein the model includes
an image encoding layer for encoding the input image, and
a text encoding layer for encoding the input text,
a part of features determined by the text encoding layer is input to the image encoding layer, and
a part of features determined by the image encoding layer is input to the text encoding layer.
9 . A non-transitory computer readable storage medium for storing a program causing a computer to function as the learning apparatus according to claim 1 .
10 . A learning method of performing machine learning, comprising:
acquiring teaching data including input data and correct answer data, the input data including an input image that contains a reference object and an input text that relatively designates a target position by referring to the reference object;
inputting the input data to a model to generate output data for specifying the target position, a reference position that is a position of the reference object, and a positional relationship of the target position with respect to the reference position; and
updating a parameter of the model to reduce a loss obtained by inputting the output data and the correct answer data to a loss function,
wherein the loss function is based on at least two errors of
a first error between the target position specified by the output data and the target position specified by the correct answer data,
a second error between the reference position specified by the output data and the reference position specified by the correct answer data, and
a third error between the positional relationship specified by the output data and the positional relationship specified by the correct answer data, or
wherein the loss function is based on the third error, and
wherein the model includes
an image encoding layer for encoding the input image, and
a text encoding layer for encoding the input text,
a part of features determined by the text encoding layer is input to the image encoding layer, and
a part of features determined by the image encoding layer is input to the text encoding layer.