Trajectory calculation device, trajectory calculating method, and trajectory calculating program
In the case where a three-dimensional position of a moving body is calculated based on image data taken by a plurality of cameras that are synchronized, high performance equipment and system such as a system to make the synchronization among the plurality of cameras and a camera having a built-in function to make the synchronization are required. It is also required to fix the camera position with high accuracy beforehand. It is made possible to calculate a trajectory of a moving body as a target in the three-dimensional space using image data taken by a plurality of cameras that are non-synchronized mutually, thereby solving the above issue. And positions of respective cameras are calculated in the three-dimensional space from a plurality of reference points having fixed position coordinates in the three-dimensional space that are commonly shown in the image data of the respective cameras, thereby solving the above issue.
1. A trajectory calculation device for calculating a trajectory of a target moving body from image data taken by a first camera and a second camera being mutually non-synchronized and installed in different positions, comprising:
a central processing unit,
an input device,
an output device, and
a memory,
wherein the central processing unit is operable to:
detect the target moving body from image data in a first image frame and a second image frame taken by the first camera and calculate two-dimensional positions of the target moving body in the first and the second image frames, which are consecutive over time, and in a third image frame taken by the second camera and calculate a two-dimensional position of the target moving body in the third image frame;
calculate a three-dimensional plane constituted by connecting three-dimensional positions of the target moving body in the first and the second image frames and an optical center point of a lens of the first camera as an existence plane from the two-dimensional positions of the target moving body and position coordinates and orientation of the first camera in a three-dimensional space; and
calculate the trajectory of the target moving body in the three-dimensional space from an intersection point of the existence plane and a straight line connecting an optical center point of a lens of the second camera and a three-dimensional position of the target moving body calculated from a two-dimensional position thereof in the third image frame and position coordinates and orientation of the second camera in the three-dimensional space, wherein the straight line intersects the existence plane at the intersection point.
2. The trajectory calculation device according to claim 1 , wherein the central processing unit is operable to calculate the position coordinates and orientation of the first and the second cameras from a plurality of reference points having fixed coordinates in the three-dimensional space and existing in the image data of the image frame.
3. The trajectory calculation device according to claim 2 , wherein optical axes of the first camera and the second camera are arranged substantially perpendicular to each other.
4. The trajectory calculation device according to claim 2 , wherein an optical axis of the second camera is arranged substantially perpendicular to a moving direction of the target moving body.
5. The trajectory calculation device according to claim 3 , wherein an optical axis of the second camera is arranged substantially perpendicular to a moving direction of the target moving body.
6. A trajectory calculation method for calculating a trajectory of a target moving body from image data taken by a first camera and a second camera being mutually non-synchronized and installed in different positions, the method comprising:
a two-dimensional position calculation step of detecting the target moving body from image data in a first image frame and a second image frame taken by the first camera and calculating two-dimensional positions of the target moving body in the first and the second image frames, which are consecutive over time, and in a third image frame taken by the second camera, calculating a two-dimensional position of the target moving body in the third image frame;
an existence plane calculation step of calculating a three-dimensional plane constituted by connecting three-dimensional positions of the target moving body in the first and the second image frames and an optical center point of a lens of the first camera as an existence plane from the two-dimensional positions of the target moving body and position coordinates and orientation of the first camera in a three-dimensional space; and
a three-dimensional trajectory calculation step of calculating the trajectory of the target moving body in the three-dimensional space from an intersection point of the existence plane and a straight line connecting an optical center point of a lens of the second camera and a three-dimensional position of the target moving body calculated from a two-dimensional position thereof in the third image frame and position coordinates and orientation of the second camera in the three-dimensional space, wherein the straight line intersects the existence plane at the intersection point.
7. The trajectory calculation method according to claim 6 , comprising: a camera information calculation step of calculating position coordinates and orientation of the first and the second cameras from a plurality of reference points having fixed coordinates in the three-dimensional space and existing in the image data of the image frame.
8. The trajectory calculation method according to claim 7 , wherein an optical axis of the second camera is arranged substantially perpendicular to a moving direction of the target moving body.
9. A trajectory calculation device capable of calculating a trajectory of a target moving body in a three-dimensional space using first video data and second video data obtained during a same time period by first and second non-synchronized cameras installed in different positions, comprising:
a central processing unit,
an input device,
an output device, and
a memory,
wherein the central processing unit is operable to:
detect the target moving body from image data in a first image frame and a second image frame constituting the first video data taken by the first camera and calculate two-dimensional positions of the target moving body in the first and the second image frames, which are consecutive over time, and detect the target moving body from image data in a third image from constituting the second video data taken by the second camera and calculate a two-dimensional position of the target moving body in the third image frame;
calculate a plane as an existence plane configured by a first straight line connecting an optical center point of a lens of the first camera and a first two-dimensional position of the target moving body in the first image frame and a second straight line connecting the optical center point of the lens of the first camera and a second two-dimensional position of the target moving body in the second image frame in the three-dimensional space from position coordinates of the first and the second two-dimensional positions and orientation and position coordinates of the optical center point of the lens of the first camera;
calculate three-dimensional position coordinates of an intersection at which a straight line connecting an optical center point of a lens of the second camera and the two-dimensional position of the target moving body in the third image frame intersects the existence plane, the third image frame having been taken at a time between the first and the second image frames, from three-dimensional position coordinates obtained from the two-dimensional position of the target moving body in the third image frame and orientation and position coordinates of the optical center point of the lens of the second camera, and calculate the trajectory of the target moving body in the three-dimensional space.
10. The trajectory calculation device according to claim 9 , wherein optical axes of the first and the second cameras are substantially perpendicular.
11. The trajectory calculation device according to claim 9 , wherein the central processing unit is operable to calculate the orientation and the position coordinates of the optical center points of the lenses of the first and second cameras from a plurality of reference points having predetermined position coordinates in the three-dimensional space and existing in the respective image data of the respective image frames.