Learning device, learning method, tracking device, and storage medium
A learning device 1 X includes an acquisition means 15 X, an estimation result matching means 16 X, and a learning means 18 X. The acquisition means 15 X acquires tracking training data in which first and second training images in time series, tracking target position information regarding position or posture of a tracking target shown in each first and second training images, and identification information of the tracking target are associated. The estimation result matching means 16 X compares the tracking target position information with posture information indicating posture of the tracking target estimated from the first and second training images and associates the posture information with the identification information. The learning means 18 X learns an inference engine, which infers correspondence information indicating correspondence relation of the tracking target between the training images when information based on the posture information is inputted, the correspondence information based on the posture information and the identification information.
1. A learning device comprising:
at least one memory configured to store instructions; and
at least one processor configured to execute the instructions to:
acquire tracking training data in which a first training image and a second training image which are training images captured in time series, tracking target object position information regarding a position or a posture of a tracking target object shown in each of the first training image and the second training image, and identification information of the tracking target object are associated;
compare the tracking target object position information with posture information indicative of an estimated posture of the tracking target object estimated from each of the first training image and the second training image and
associate the posture information with the identification information; and
learn an inference engine based on the posture information and the identification information,
the inference engine being configured to infer correspondence information when information based on the posture information is inputted to the inference engine,
the correspondence information indicating correspondence relation of the tracking target object between the first training image and the second training image.
2. The learning device according to claim 1 ,
wherein the at least one processor is configured to further execute the instructions to convert the posture information into feature information which is information indicating a position of each feature point of the tracking target objects detected from the training images captured in time series, and
wherein the at least one processor is configured to execute the instructions to input the feature information to the inference engine as the information based on the posture information.
3. The learning device according to claim 2 ,
wherein the at least one processor is configured to execute the instructions to generate the feature information according to a format that is based on:
the number of detections of the tracking target objects in the first training image;
the number of detections of the tracking target objects in the second training image; and
the number of the feature points.
4. The learning device according to claim 1 ,
wherein the inference engine is a neural network with a convolution layer.
5. The learning device according to claim 1 ,
wherein the correspondence information indicates a matrix including each element indicating a probability that each of the tracking target objects in the first training image corresponds to each of the tracking target objects in the second training image.
6. The learning device according to claim 5 ,
wherein the matrix further comprises a row or a column indicating a probability of appearance or disappearance of the tracking target object in the first training image and the second training image.
7. The learning device according to claim 6 ,
wherein the at least one processor is configured to execute the instructions to learn the inference engine so that a sum of elements for each row or each column of the matrix is set to be a predetermined value.
8. The learning device according to claim 5 ,
wherein the matrix includes:
a first channel including each element indicating the probability that each of the tracking target objects in the first training image corresponds to each of the tracking target objects in the second training image; and
a second channel including each element indicating a probability that each of the tracking target objects in the first training image does not correspond to each of the tracking target objects in the second training image, and
wherein the at least one processor is configured to execute the instructions to learn the inference engine so that a sum of the each element in channel direction is set to be a predetermined value.
9. The learning device according to claim 1 ,
wherein the tracking target object position information is
information indicating an existence area of each tracking target object in the first training image and the second training image, or
information indicating a position of each feature point of each tracking target object in the first training image and the second training image.
10. The learning device according to claim 1 ,
wherein the at least one processor is configured to further execute the instructions to generate the posture information by estimating the posture of each tracking target object shown in the first training image and the second training image based on the first training image and the second training image.
11. A tracking device comprising:
at least one memory configured to store instructions; and
at least one processor configured to execute the instructions to:
acquire a first captured image and a second captured image captured in time series;
generate, based on the first captured image and the second captured image, posture information indicative of an estimation result of a posture of a tracking target object in each of the first captured image and the second captured image; and
generate correspondence information based on the posture information and an inference engine, the inference engine being configured to infer the correspondence information when information based on the posture information is inputted to the inference engine, the correspondence information indicating a correspondence relation of the tracking target object between the first captured image and the second captured image.
12. The tracking device according to claim 11 ,
wherein the at least one processor is configured to further execute the instructions to manage identification information assigned to the tracking target object based on the correspondence information.
13. A learning method executed by a computer, the learning method comprising:
acquiring tracking training data in which a first training image and a second training image which are training images captured in time series, tracking target object position information regarding a position or a posture of a tracking target object shown in each of the first training image and the second training image, and identification information of the tracking target object are associated;
comparing the tracking target object position information with posture information indicative of an estimated posture of the tracking target object estimated from each of the first training image and the second training image and associating the posture information with the identification information; and
learning an inference engine based on the posture information and the identification information,
the inference engine being configured to infer correspondence information when information based on the posture information is inputted to the inference engine,
the correspondence information indicating correspondence relation of the tracking target object between the first training image and the second training image.