Image processing device, image processing method, and program, for high-definition creation of an image on a screen
A processor of an image processing device is configured to: acquire three-dimensional coordinates of a three-dimensional distance-measuring sensor system defined in a three-dimensional coordinate system applied to a three-dimensional distance-measuring sensor, the three-dimensional coordinates of the three-dimensional distance-measuring sensor system being capable of specifying positions of a plurality of measurement points; acquire three-dimensional coordinates of an imaging apparatus system, on the basis of the three-dimensional coordinates of the three-dimensional distance-measuring sensor system; convert the three-dimensional coordinates of the imaging apparatus system into two-dimensional coordinates of the imaging apparatus system, which are capable of specifying a position in a captured image; convert the three-dimensional coordinates of the imaging apparatus system into two-dimensional coordinates of a display system, which are capable of specifying a position on a screen; and assign a pixel constituting the captured image to an interpolation position specified through an interpolating method using the two-dimensional coordinates of the imaging apparatus system and the two-dimensional coordinates of the display system, on the screen.
1 . An image processing device comprising:
a processor; and
a memory that is built in or connected to the processor,
wherein a positional relationship between a three-dimensional distance-measuring sensor and an imaging apparatus having a higher sampling resolution than the three-dimensional distance-measuring sensor is given, and
the processor is configured to:
acquire three-dimensional coordinates of a three-dimensional distance-measuring sensor defined in a three-dimensional coordinate system applied to the three-dimensional distance-measuring sensor, on the basis of distance measurement results of the three-dimensional distance-measuring sensor for a plurality of measurement points, the three-dimensional coordinates of the three-dimensional distance-measuring sensor being capable of specifying positions of the plurality of measurement points;
acquire three-dimensional coordinates of an imaging apparatus defined in a three-dimensional coordinate system applied to the imaging apparatus, on the basis of the three-dimensional coordinates of the three-dimensional distance-measuring sensor;
convert the three-dimensional coordinates of the imaging apparatus into two-dimensional coordinates of the imaging apparatus, which are capable of specifying a position in a captured image which is captured through imaging performed by the imaging apparatus;
convert the three-dimensional coordinates of the imaging apparatus into two-dimensional coordinates of a display, which are capable of specifying a position on a screen;
assign a pixel constituting the captured image to an interpolation position specified through an interpolating method using the two-dimensional coordinates of the imaging apparatus and the two-dimensional coordinates of the display, on the screen;
acquire the three-dimensional coordinates of the imaging apparatus by converting the three-dimensional coordinates of the three-dimensional distance-measuring sensor into the three-dimensional coordinates of the imaging apparatus;
generate a polygonal patch on the basis of the three-dimensional coordinates of the imaging apparatus; and
the three-dimensional coordinates of the imaging apparatus define a position of an intersection of the polygonal patch,
wherein the interpolation position is a position corresponding to a position other than the intersection of the polygonal patch, on the screen, and
the processor is configured to create a three-dimensional image by assigning a pixel at a position corresponding to the position of the intersection of the polygonal patch, among a plurality of pixels included in the captured image, to an on-screen intersection position corresponding to the position of the intersection of the polygonal patch, on the screen.
2 . The image processing device according to claim 1 ,
wherein the polygonal patch is defined by a triangular mesh or a quadrangular mesh.
3 . The image processing device according to claim 1 ,
wherein the processor is configured to acquire the three-dimensional coordinates of the imaging apparatus by calculating the three-dimensional coordinates of the imaging apparatus on the basis of a feature point of a subject included as an image between images of a plurality of frames captured by imaging the subject with the imaging apparatus from different positions.
4 . The image processing device according to claim 1 ,
wherein the position on the screen is a position on a screen of a display.
5 . The image processing device according to claim 1 ,
wherein the memory holds association information in which the three-dimensional coordinates of the three-dimensional distance-measuring sensor and the two-dimensional coordinates of the imaging apparatus are associated with each other.
6 . The image processing device according to claim 5 ,
wherein the processor is configured to assign the pixel constituting the captured image to the interpolation position with reference to the association information.
7 . An image processing method comprising:
acquiring three-dimensional coordinates of a three-dimensional distance-measuring sensor defined in a three-dimensional coordinate system applied to a three-dimensional distance-measuring sensor, on the basis of distance measurement results of the three-dimensional distance-measuring sensor for a plurality of measurement points, the three-dimensional coordinates of the three-dimensional distance-measuring sensor being capable of specifying positions of the plurality of measurement points, on condition that a positional relationship between the three-dimensional distance-measuring sensor and an imaging apparatus having a higher sampling resolution than the three-dimensional distance-measuring sensor is given;
acquiring three-dimensional coordinates of an imaging apparatus defined in a three-dimensional coordinate system applied to the imaging apparatus, on the basis of the three-dimensional coordinates of the three-dimensional distance-measuring sensor;
converting the three-dimensional coordinates of the imaging apparatus into two-dimensional coordinates of the imaging apparatus, which are capable of specifying a position in a captured image which is captured through imaging performed by the imaging apparatus;
converting the three-dimensional coordinates of the imaging apparatus into two-dimensional coordinates of a display, which are capable of specifying a position on a screen;
assigning a pixel constituting the captured image to an interpolation position specified through an interpolating method based on the two-dimensional coordinates of the imaging apparatus and the two-dimensional coordinates of the display, on the screen;
acquiring the three-dimensional coordinates of the imaging apparatus by converting the three-dimensional coordinates of the three-dimensional distance-measuring sensor into the three-dimensional coordinates of the imaging apparatus; and
generating a polygonal patch on the basis of the three-dimensional coordinates of the imaging apparatus,
wherein the three-dimensional coordinates of the imaging apparatus define a position of an intersection of the polygonal patch,
the interpolation position is a position corresponding to a position other than the intersection of the polygonal patch, on the screen, and
the image processing method further comprises
creating a three-dimensional image by assigning a pixel at a position corresponding to the position of the intersection of the polygonal patch, among a plurality of pixels included in the captured image, to an on-screen intersection position corresponding to the position of the intersection of the polygonal patch, on the screen.
8 . A non-transitory computer-readable storage medium storing a program executable by a computer to perform a process comprising:
acquiring three-dimensional coordinates of a three-dimensional distance- measuring sensor defined in a three-dimensional coordinate applied to a three-dimensional distance-measuring sensor, on the basis of distance measurement results of the three-dimensional distance-measuring sensor for a plurality of measurement points, the three-dimensional coordinates of the three-dimensional distance-measuring sensor being capable of specifying positions of the plurality of measurement points, on condition that a positional relationship between the three-dimensional distance-measuring sensor and an imaging apparatus having a higher sampling resolution than the three-dimensional distance-measuring sensor is given;
acquiring three-dimensional coordinates of an imaging apparatus defined in a three-dimensional coordinate system applied to the imaging apparatus, on the basis of the three-dimensional coordinates of the three-dimensional distance-measuring sensor;
converting the three-dimensional coordinates of the imaging apparatus into two-dimensional coordinates of the imaging apparatus, which are capable of specifying a position in a captured image which is captured through imaging performed by the imaging apparatus;
converting the three-dimensional coordinates of the imaging apparatus into two-dimensional coordinates of a display, which are capable of specifying a position on a screen;
assigning a pixel constituting the captured image to an interpolation position specified through an interpolating method using the two-dimensional coordinates of the imaging apparatus and the two-dimensional coordinates of the display, on the screen;
acquiring the three-dimensional coordinates of the imaging apparatus by converting the three-dimensional coordinates of the three-dimensional distance-measuring sensor into the three-dimensional coordinates of the imaging apparatus; and
generating a polygonal patch on the basis of the three-dimensional coordinates of the imaging apparatus,
wherein the three-dimensional coordinates of the imaging apparatus define a position of an intersection of the polygonal patch,
the interpolation position is a position corresponding to a position other than the intersection of the polygonal patch, on the screen, and
the process further comprises
creating a three-dimensional image by assigning a pixel at a position corresponding to the position of the intersection of the polygonal patch, among a plurality of pixels included in the captured image, to an on-screen intersection position corresponding to the position of the intersection of the polygonal patch, on the screen.