Computer-readable recording medium storing region detection program, apparatus, and method
Avoiding False Negatives in vision-based multi-view gymnast detection presents a significant challenge. Detection results with False Negatives can further impact subsequent processing, such as multi-view 3D pose estimation. Therefore, interpolating missing bounding boxes is a desirable solution. Assuming that calibrated camera parameters are known, our method interpolates missing bounding boxes when a gymnast's 2D bounding boxes are detected in more than two views but are absent in others. This method primarily involves three steps: 1) Inferring the vertical 3D body center line using detected cross-view 2D bounding boxes and camera parameters through 2D-to-3D projection; 2) Obtaining the average 3D gymnast scale from pre-acquired data, and then calculating the 3D horizontal scale based on the 3D vertical scale obtained in step 1; 3) Interpolating the missing 2D bounding boxes using the inferred 3D vertical line and horizontal scale through 3D-to-2D projection.
1 . A non-transitory computer-readable recording medium storing a region detection program for causing a computer to execute a process comprising:
acquiring images each which is captured by each of a plurality of imaging apparatuses that are arranged in the same horizontal plane and capture the respective images of a person from respective different directions;
detecting a region indicating the person from each of the images by inputting the images to a machine learning model which is generated in advance by a machine learning so as to detect the region indicating the person; and
interpolating, based on a first region of the person which is detected from a first image in which the region indicating the person is detected by the machine learning model in the images and a parameter of each of the plurality of imaging apparatuses, a second region indicating the person in a second image in which the region indicating the person is not detected by the machine learning model in the images, a width of the second region being estimated based on a height of the first region, and a height of the second region estimated by converting an end point of a vertical center line of the first region into a coordinate of an end point of a vertical center line of the person in a three dimensional space based on the parameter of an imaging apparatus which captures the first image and converting the coordinate into a coordinate in the second image based on the parameter of an imaging apparatus which captures the second image, and statistical information regarding a posture of the person, and the statistical information regarding a posture of the person being a mean of a sum of a height and a width of a rectangular parallelepiped surrounding the person in each of a plurality of different postures of the person.
2 . The non-transitory computer-readable recording medium according to claim 1 , wherein
a length of the vertical center line of the person in the three dimensional space is set as a height of the person in the three dimensional space, a difference between the mean indicated by the statistical information and the height of the person in the three dimensional space is estimated as a width of the person in the three dimensional space and the width of the second region is estimated based on a ratio of the height and the width of the person in the three dimensional space and the height of the second region.
3 . The non-transitory computer-readable recording medium according to claim 1 , wherein
the plurality of imaging apparatuses are arranged in a same vertical plane, and
a height of the second region is estimated based on a width of the first region, a width of the second region, and statistical information regarding a posture of the person.
4 . A region detection apparatus comprising:
a memory; and
a processor coupled to the memory and configured to:
acquire images each which is captured by each of a plurality of imaging apparatuses that are arranged in the same horizontal plane and capture the respective images of a person from respective different directions;
detected a region indicating the person from each of the images by inputting the images to a machine learning model which is generated in advance by a machine learning so as to detect the region indicating the person; and
interpolate, based on a first region of the person which is detected from a first image in which the region indicating the person is detected by the machine learning model in the images and a parameter of each of the plurality of imaging apparatuses, a second region indicating the person in a second image in which the region indicating the person is not detected by the machine learning model in the images, a width of the second region being estimated based on a height of the first region, and a height of the second region estimated by converting an end point of a vertical center line of the first region into a coordinate of an end point of a vertical center line of the person in a three dimensional space based on the parameter of an imaging apparatus which captures the first image and converting the coordinate into a coordinate in the second image based on the parameter of an imaging apparatus which captures the second image, and statistical information regarding a posture of the person, and the statistical information regarding a posture of the person being a mean of a sum of a height and a width of a rectangular parallelepiped surrounding the person in each of a plurality of different postures of the person.
5 . The region detection apparatus according to claim 4 , wherein
a length of the vertical center line of the person in the three dimensional space is set as a height of the person in the three dimensional space, a difference between the mean indicated by the statistical information and the height of the person in the three dimensional space is estimated as a width of the person in the three dimensional space and the width of the second region is estimated based on a ratio of the height and the width of the person in the three dimensional space and the height of the second region.
6 . The region detection apparatus according to claim 4 , wherein
the plurality of imaging apparatuses are arranged in a same vertical plane, and
a height of the second region is estimated based on a width of the first region, a width of the second region, and statistical information regarding a posture of the person.
7 . A region detection method for executing a process comprising:
acquiring images each which is captured by each of a plurality of imaging apparatuses that are arranged in the same horizontal plane and capture the respective images of a person from respective different directions;
detecting a region indicating the person from each of the images by inputting the images to a machine learning model which is generated in advance by a machine learning so as to detect the region indicating the person; and
interpolating, based on a first region of the person which is detected from a first image in which the region indicating the person is detected by the machine learning model in the images and a parameter of each of the plurality of imaging apparatuses, a second region indicating the person in a second image in which the region indicating the person is not detected by the machine learning model in the images, a width of the second region being estimated based on a height of the first region, and a height of the second region estimated by converting an end point of a vertical center line of the first region into a coordinate of an end point of a vertical center line of the person in a three dimensional space based on the parameter of an imaging apparatus which captures the first image and converting the coordinate into a coordinate in the second image based on the parameter of an imaging apparatus which captures the second image, and statistical information regarding a posture of the person, and the statistical information regarding a posture of the person being a mean of a sum of a height and a width of a rectangular parallelepiped surrounding the person in each of a plurality of different postures of the person.