Image processing apparatus, image processing system, and program therefor

1. An image processing system comprising:

a light projector projecting pattern light having a coordinate component in at least one direction that is determinable;

a camera imaging an object upon which the pattern light is projected to acquire a pattern image; and

a control apparatus comprising a processor configured with a program for calculating three-dimensional coordinates of at least one point on the object based on the pattern light and the pattern image;

wherein the processor is configured with the program to perform operations comprising:

determining a first point on the pattern image;

calculating a second point in a projection plane of the pattern light based on a location of the first point on the pattern image, the second point corresponding to the first point;

calculating a third point based on the second point in the projection plane, a nonlinear line in the projection plane calculated by analyzing the pattern image, and an epipolar line in the projection plane calculated based on the first point; and

calculating the three-dimensional coordinates of the point on the object based on the first point and the third point.

2. The image processing system according to claim 1 , wherein the processor is configured with the program to perform operations further comprising:

setting the second point as an initial value point A;

calculating a point B by performing a first operation relating to a coordinate transformation on the point A;

calculating a point C by performing a second operation based on the nonlinear line on the point B;

calculating a point D by performing a third operation relating to a coordinate transformation on the point C; and

calculating a point E by performing a fourth operation based on the epipolar line on the point D.

3. The image processing system according to claim 2 , wherein the processor is configured with the program to perform operations further comprising:

determining whether a predetermined termination condition is satisfied after the fourth operation is executed;

in response to determining that the termination condition is satisfied, setting the point E as the third point; and

in response to determining that the termination condition is not satisfied, setting the point E as the initial value point A and repeating the first operation, the second operation, the third operation and the fourth operation.

4. The image processing system according to claim 3 , wherein the termination condition comprises coordinates of the point A and the point E being within a predetermined range.

5. The image processing system according to claim 3 , wherein the termination condition comprises a number of times the first to fourth operations have been repeated.

6. The image processing system according to claim 1 , wherein the processor is configured with the program to perform operations comprising calculating the nonlinear line by analyzing a straight line having a coordinate component in one direction on the pattern image that is the same as that of the first point.

7. The image processing system according to claim 2 , wherein the processor is configured with the program to perform operations further comprising calculating the nonlinear line by analyzing a straight line having a coordinate component in one direction on the pattern image that is the same as that of the first point.

8. The image processing system according to claim 3 , wherein the processor is configured with the program to perform operations comprising calculating the nonlinear line by analyzing a straight line having a coordinate component in one direction on the pattern image that is the same as that of the first point.

9. The image processing system according to claim 4 , wherein the processor is configured with the program to perform operations comprising calculating the nonlinear line by analyzing a straight line having a coordinate component in one direction on the pattern image that is the same as that of the first point.

10. The image processing system according to claim 5 , wherein the processor is configured with the program to perform operations comprising calculating the nonlinear line by analyzing a straight line having a coordinate component in one direction on the pattern image that is the same as that of the first point.

11. A control apparatus connected to a light projector for projecting pattern light having a coordinate component in at least one direction that is determinable and a camera imaging an object upon which the pattern light is projected to acquire a pattern image, the control apparatus comprising a processor configured with a program for calculating three-dimensional coordinates of at least one point on the object based on the pattern light and the pattern image;

wherein the processor is configured with the program to perform operations comprising:

operation as a determination unit configured to determine a first point on the pattern image;

operation as a calculation unit configured to calculate a second point in a projection plane of the pattern light based on a location of the first point on the pattern image, the second point corresponding to the first point;

operation as a correction unit configured to calculate a third point based on the second point in the projection plane, a nonlinear line in the projection plane calculated by analyzing the pattern image, and an epipolar line in the projection plane calculated based on the first point; and

operation as a calculation unit configured to calculate the three-dimensional coordinates of the point on the object based on the first point and the third point.

12. A method performed by a control apparatus connected to a light projection apparatus for projecting pattern light having a coordinate component in at least one direction that is determinable and an imaging apparatus for imaging an object upon which the pattern light is projected to acquire a pattern image, the method calculating three-dimensional coordinates of at least one point on the object based on the pattern light and the pattern image, the method comprising:

determining a first point on the pattern image;

calculating a second point in a projection plane of the pattern light based on a location of the first point in the pattern image, the second point corresponding to the first point;

calculating a third point based on the second point in the projection plane, a nonlinear line in the projection plane calculated by analyzing the pattern image, and an epipolar line in the projection plane calculated based on the first point; and

calculating the three-dimensional coordinates of the point on the object based on the first point and the third point.

13. A non-transitory computer-readable recording medium storing a program for use in a control apparatus comprising a processor connected to a light projection apparatus for projecting pattern light having a coordinate component in at least one direction that is determinable and an imaging apparatus for imaging an object to which the pattern light is projected to acquire a pattern image, the program, when read and executed, causing the processor to perform operations comprising:

determining a first point on the pattern image;

calculating a second point in a projection plane of the pattern light based on a pattern imaged on a location of the first point, the second point corresponding to the first point;

calculating a third point based on the second point in the projection plane, a nonlinear line in the projection plane calculated by analyzing the pattern image, and an epipolar line in the projection plane calculated based on the first point; and

calculating three-dimensional coordinates of the point on the object based on the first point and the third point.

14. The method according to claim 12 , wherein calculating the third point further comprises:

setting the second point as an initial value point A;

calculating a point B by performing a first operation relating to a coordinate transformation on the point A;

calculating a point C by performing a second operation based on the nonlinear line on the point B;

calculating a point D by performing a third operation relating to a coordinate transformation on the point C; and

calculating a point E by performing a fourth operation based on the epipolar line on the point D.

15. The method according to claim 14 , further comprising:

determining whether a predetermined termination condition is satisfied after the fourth operation is executed;

in response to determining that the termination condition is satisfied, setting the point E as the third point; and

in response to determining that the termination condition is not satisfied, setting the point E as the initial value point A and repeating the first operation, the second operation, the third operation, and the fourth operation.

16. The method according to claim 15 , wherein the termination condition comprises coordinates of the point A and the point E being within a predetermined range.

17. The method according to claim 15 , wherein the termination condition comprises a number of times the first to fourth operations have been repeated.

18. The method according to claim 12 , further comprising calculating the nonlinear line by analyzing a straight line having a coordinate component in one direction on the pattern image that is the same as that of the first point.