Package measuring apparatus, package accepting system, package measuring method, and non-transitory computer readable medium
A package measuring apparatus includes a depth sensor, the package having a rectangular parallelepiped shape and placed on a mounting table, and at least one processor. The processor obtains spatial coordinates of four vertices within a space in which the center of the depth sensor is set as the point of origin based on data of a distance from the depth sensor to each of the four vertices and data of a position of each sensor element of the depth sensor corresponding to each of the four vertices. The processor calculates, based on the spatial coordinates of the four vertices, a length of each of three sides defined between a first vertex and three other vertices.
1. A package measuring apparatus comprising:
a depth sensor disposed at a position at which the depth sensor is able to pick-up an image of a package for three faces thereof simultaneously and configured to acquire a distance to the package by picking-up the image of the package, the package having a rectangular parallelepiped shape and placed on a mounting table; and
at least one processor for calculating a size of the package based on data acquired from the depth sensor,
wherein the processor performs the processing of:
picking-up an image of the mounting table on which nothing is placed using the depth sensor beforehand and storing the picked-up image as background depth image data;
controlling the depth sensor, acquiring depth image data of the package by picking-up the image of the package placed on the mounting table for the three faces thereof simultaneously using the depth sensor, and specifying data of the package in the depth image data of the package by comparing the depth image data of the package with the background depth image data;
specifying positions of a first vertex, a second vertex, a third vertex, and a fourth vertex, the first vertex being a vertex which is common among the three faces of the package and is at a position closest in distance from the depth sensor within the depth image data of the package, the second vertex being a vertex which is at a position on the leftmost side within the depth image data of the package, the third vertex being a vertex which is at a position on the rightmost side within the depth image data of the package, and the fourth vertex being a vertex which is at a position on the lowermost side within the depth image data of the package;
obtaining spatial coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex within a space in which the center of the depth sensor is set as the point of origin based on a view angle of the depth sensor; data of a distance from the depth sensor to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively; number of sensor elements of the depth sensor; and data of a position of each sensor element of the depth sensor corresponding to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively; and
calculating, based on the spatial coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex, a length of a first side defined between the first vertex and the second vertex, a length of a second side defined between the first vertex and the third vertex, and a length of a third side defined between the first vertex and the fourth vertex,
wherein, when a direction in which the depth sensor picks-up an image is Z-axis, a direction coplanar with the Z-axis and orthogonal to the Z-axis is X-axis, and a direction that is orthogonal to the X-axis and the Z-axis is Y-axis, the processor performs the processing of:
obtaining actual distances from the Z-axis to the image end in the X-axis direction at the first vertex, the second vertex, the third vertex, and the fourth vertex based on a view angle of the depth sensor in the X-axis direction and data of a distance from the depth sensor in the Z-axis direction to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively;
obtaining actual distances from the Z-axis to the image end in the Y-axis direction at the first vertex, the second vertex, the third vertex, and the fourth vertex based on a view angle of the depth sensor in the Y-axis direction and the data of the distance from the depth sensor in the Z-axis direction to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively;
obtaining a change rate of the actual distance in the X-axis direction for each sensor element in the first vertex, the second vertex, the third vertex, and the fourth vertex based on the actual distances from the Z-axis to the image end in the X-axis direction at the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively and number of sensor elements in the X-axis direction of the depth sensor;
obtaining a change rate of the actual distance in the Y-axis direction for each sensor element in the first vertex, the second vertex, the third vertex, and the fourth vertex based on the actual distances from the Z-axis to the image end in the Y-axis direction at the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively and number of sensor elements in the Y-axis direction of the depth sensor; and
determining the spatial coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex by
obtaining X coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex based on the position of each sensor element in the X-axis direction corresponding to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively and the change rate of the actual distance in the X-axis direction for each sensor element in the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively;
obtaining Y coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex based on the position of each sensor element in the Y-axis direction corresponding to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively and the change rate of the actual distance in the Y-axis direction for each sensor element in the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively; and
using data of distances in the Z-axis direction, as Z coordinates,
wherein the processor performs the processing of:
based on the data of distance from the depth sensor to the first vertex,
obtaining the change rate ΔXa of the actual distance in the X-axis direction for each sensor element in the first vertex by equation as follows:
ΔXa=Za×tan (θx/2)/(Xnmr/2), where Za is the Z coordinate of the first vertex, θx is the view angle of the depth sensor in the X-axis direction, and Xnmr is the number of sensor elements in the X-axis direction,
obtaining the change rate ΔYa of the actual distance in the Y-axis direction for each sensor element in the first vertex by equation as follows:
ΔYa=Za×tan (θy/2)/(Ynmr/2), where θy is the view angle of the depth sensor in the Y-axis direction and Ynmr is the number of sensor elements in the Y-axis direction,
obtaining the X coordinate of the first vertex by equation as follows:
Xa=ΔXa×((Xa−pixel)−Xnmr/2), where Xa is the X coordinate of the first vertex and Xa−pixel is the position of the sensor element in the X-axis direction corresponding to the first vertex,
obtaining the Y coordinate of the first vertex by equation as follows:
Ya=ΔYa×((Ya−pixel)−Ynmr/2), where Ya is the Y coordinate of the first vertex and Ya−pixel is the position of the sensor element in the Y-axis direction corresponding to the first vertex; and
obtaining the X and Y coordinates of the second vertex, the third vertex, and the fourth vertex, similarly.
2. The package measuring apparatus according to claim 1 , wherein the processor specifies the first vertex, the second vertex, the third vertex, and the fourth vertex based on the depth image data of the package.
3. The package measuring apparatus according to claim 1 , further comprising an image capturing apparatus,
wherein the processor specifies the first vertex, the second vertex, the third vertex, and the fourth vertex by performing image recognition of an image acquired by the image capturing apparatus.
4. The package measuring apparatus according to claim 3 , further comprising display moans, a display device,
wherein the processor causes the image capturing apparatus to capture an image of the package placed on the mounting table and causes the display device to display the captured image, and when at least one of the first to the fourth vertices of the package are not able to be specified, causes the display device to display a message prompting adjustment of a mounting state of the package.
5. The package measuring apparatus according to claim 1 , further comprising the mounting table,
wherein the depth sensor is attached to the mounting table.
6. The package measuring apparatus according to claim 1 , further comprising a weighing scale within the mounting table,
wherein the weighing scale is configured to measure a weight of the package and to output a result of the measurement.
7. A package accepting system comprising:
the package measuring apparatus according to claim 1 ;
an information acquisition apparatus; and
a payment processing apparatus, wherein
the information acquisition apparatus is configured to acquire information related to shipment of the package,
the package measuring apparatus is configured to calculate a shipping charge of the package based on the size of the package and the information related to the shipment of the package, and
the payment processing apparatus is configured to perform payment processing for the shipping charge.
8. A package measuring method for executing calculation processing for calculating a size of a package using a depth sensor disposed at a position at which the depth sensor is able to pick-up an image of a package for three faces thereof simultaneously and configured to acquire a distance to the package by picking-up the image of the package, the package having a rectangular parallelepiped shape and placed on a mounting table, the calculation processing comprising:
picking-up an image of the mounting table on which nothing is placed using the depth sensor beforehand and storing the picked-up image as background depth image data;
controlling the depth sensor, acquiring depth image data of the package by picking-up the image of the package placed on the mounting table for the three faces thereof simultaneously using the depth sensor, and specifying data of the package in the depth image data of the package by comparing the depth image data of the package with the background depth image data;
specifying positions of a first vertex, a second vertex, a third vertex, and a fourth vertex, the first vertex being a vertex which is common among the three faces of the package and is at a position closest in distance from the depth sensor within the depth image data of the package, the second vertex being a vertex which is at a position on the leftmost side within the depth image data of the package, the third vertex being a vertex which is at a position on the rightmost side within the depth image data of the package, and the fourth vertex being a vertex which is at a position on the lowermost side within the depth image data of the package;
obtaining spatial coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex within a space in which the center of the depth sensor is set as the point of origin based on a view angle of the depth sensor; data of a distance from the depth sensor to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively; number of sensor elements of the depth sensor; and data of a position of each sensor element of the depth sensor corresponding to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively; and
calculating, based on the spatial coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex, a length of a first side defined between the first vertex and the second vertex, a length of a second side defined between the first vertex and the third vertex, and a length of a third side defined between the first vertex and the fourth vertex,
wherein, when a direction in which the depth sensor picks-up an image is Z-axis, a direction coplanar with the Z-axis and orthogonal to the Z-axis is X-axis, and a direction that is orthogonal to the X-axis and the Z-axis is Y-axis, the calculation processing comprising:
obtaining actual distances from the Z-axis to the image end in the X-axis direction at the first vertex, the second vertex, the third vertex, and the fourth vertex based on a view angle of the depth sensor in the X-axis direction and data of a distance from the depth sensor in the Z-axis direction to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively;
obtaining actual distances from the Z-axis to the image end in the Y-axis direction at the first vertex, the second vertex, the third vertex, and the fourth vertex based on a view angle of the depth sensor in the Y-axis direction and the data of the distance from the depth sensor in the Z-axis direction to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively;
obtaining a change rate of the actual distance in the X-axis direction for each sensor element in the first vertex, the second vertex, the third vertex, and the fourth vertex based on the actual distances from the Z-axis to the image end in the X-axis direction at the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively and number of sensor elements in the X-axis direction of the depth sensor;
obtaining a change rate of the actual distance in the Y-axis direction for each sensor element in the first vertex, the second vertex, the third vertex, and the fourth vertex based on the actual distances from the Z-axis to the image end in the Y-axis direction at the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively and number of sensor elements in the Y-axis direction of the depth sensor; and
determining the spatial coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex by
obtaining X coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex based on the position of each sensor element in the X-axis direction corresponding to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively and the change rate of the actual distance in the X-axis direction for each sensor element in the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively;
obtaining Y coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex based on the position of each sensor element in the Y-axis direction corresponding to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively and the change rate of the actual distance in the Y-axis direction for each sensor element in the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively; and
using data of distances in the Z-axis direction, as Z coordinates,
wherein the calculation processing comprising:
based on the data of distance from the depth sensor to the first vertex,
obtaining the change rate ΔXa of the actual distance in the X-axis direction for each sensor element in the first vertex by equation as follows:
ΔXa=Za×tan (θx/2)/(Xnmr/2), where Za is the Z coordinate of the first vertex, θx is the view angle of the depth sensor in the X-axis direction, and Xnmr is the number of sensor elements in the X-axis direction,
obtaining the change rate ΔYa of the actual distance in the Y-axis direction for each sensor element in the first vertex by equation as follows:
ΔYa=Za×tan (θy/2)/(Ynmr/2), where θy is the view angle of the depth sensor in the Y-axis direction and Ynmr is the number of sensor elements in the Y-axis direction,
obtaining the X coordinate of the first vertex by equation as follows:
Xa=ΔXa×((Xa−pixel)−Xnmr/2), where Xa is the X coordinate of the first vertex and Xa−pixel is the position of the sensor element in the X-axis direction corresponding to the first vertex,
obtaining the Y coordinate of the first vertex by equation as follows:
Ya=ΔYa×((Ya−pixel)−Ynmr/2), where Ya is the Y coordinate of the first vertex and Ya−pixel is the position of the sensor element in the Y-axis direction corresponding to the first vertex; and
obtaining the X and Y coordinates of the second vertex, the third vertex, and the fourth vertex, similarly.
9. A non-transitory computer readable medium storing a program for causing a computer to perform calculation processing of calculating a size of a package using a depth sensor disposed at a position at which the depth sensor is able to pick-up an image of a package for three faces thereof simultaneously and configured to acquire a distance to the package by picking-up the image of the package, the package having a rectangular parallelepiped shape and placed on a mounting table, the calculation processing comprising:
picking-up an image of the mounting table on which nothing is placed using the depth sensor beforehand and storing the picked-up image as background depth image data;
controlling the depth sensor, acquiring depth image data of the package by picking-up the image of the package placed on the mounting table for the three faces thereof simultaneously using the depth sensor, and specifying data of the package in the depth image data of the package by comparing the depth image data of the package with the background depth image data;
specifying positions of a first vertex, a second vertex, a third vertex, and a fourth vertex, the first vertex being a vertex which is common among the three faces of the package and is at a position closest in distance from the depth sensor within the depth image data of the package, the second vertex being a vertex which is at a position on the leftmost side within the depth image data of the package, the third vertex being a vertex which is at a position on the rightmost side within the depth image data of the package, and the fourth vertex being a vertex which is at a position on the lowermost side within the depth image data of the package;
obtaining spatial coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex within a space in which the center of the depth sensor is set as the point of origin based on a view angle of the depth sensor; data of a distance from the depth sensor to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively; number of sensor elements of the depth sensor; and data of a position of each sensor element of the depth sensor corresponding to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively; and
calculating, based on the spatial coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex, a length of a first side defined between the first vertex and the second vertex, a length of a second side defined between the first vertex and the third vertex, and a length of a third side defined between the first vertex and the fourth vertex,
wherein, when a direction in which the depth sensor picks-up an image is Z-axis, a direction coplanar with the Z-axis and orthogonal to the Z-axis is X-axis, and a direction that is orthogonal to the X-axis and the Z-axis is Y-axis, the calculation processing comprising:
obtaining actual distances from the Z-axis to the image end in the X-axis direction at the first vertex, the second vertex, the third vertex, and the fourth vertex based on a view angle of the depth sensor in the X-axis direction and data of a distance from the depth sensor in the Z-axis direction to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively;
obtaining actual distances from the Z-axis to the image end in the Y-axis direction at the first vertex, the second vertex, the third vertex, and the fourth vertex based on a view angle of the depth sensor in the Y-axis direction and the data of the distance from the depth sensor in the Z-axis direction to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively;
obtaining a change rate of the actual distance in the X-axis direction for each sensor element in the first vertex, the second vertex, the third vertex, and the fourth vertex based on the actual distances from the Z-axis to the image end in the X-axis direction at the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively and number of sensor elements in the X-axis direction of the depth sensor;
obtaining a change rate of the actual distance in the Y-axis direction for each sensor element in the first vertex, the second vertex, the third vertex, and the fourth vertex based on the actual distances from the Z-axis to the image end in the Y-axis direction at the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively and number of sensor elements in the Y-axis direction of the depth sensor; and
determining the spatial coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex by
obtaining X coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex based on the position of each sensor element in the X-axis direction corresponding to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively and the change rate of the actual distance in the X-axis direction for each sensor element in the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively;
obtaining Y coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex based on the position of each sensor element in the Y-axis direction corresponding to each of the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively and the change rate of the actual distance in the Y-axis direction for each sensor element in the first vertex, the second vertex, the third vertex, and the fourth vertex, respectively; and
using data of distances in the Z-axis direction, as Z coordinates,
wherein the calculation processing comprising:
based on the data of distance from the depth sensor to the first vertex,
obtaining the change rate ΔXa of the actual distance in the X-axis direction for each sensor element in the first vertex by equation as follows:
ΔXa=Za×tan (θx/2)/(Xnmr/2), where Za is the Z coordinate of the first vertex, θx is the view angle of the depth sensor in the X-axis direction, and Xnmr is the number of sensor elements in the X-axis direction,
obtaining the change rate ΔYa of the actual distance in the Y-axis direction for each sensor element in the first vertex by equation as follows:
ΔYa=Za×tan (θy/2)/(Ynmr/2), where θy is the view angle of the depth sensor in the Y-axis direction and Ynmr is the number of sensor elements in the Y-axis direction,
obtaining the X coordinate of the first vertex by equation as follows:
Xa=ΔXa×((Xa−pixel)−Xnmr/2), where Xa is the X coordinate of the first vertex and Xa−pixel is the position of the sensor element in the X-axis direction corresponding to the first vertex,
obtaining the Y coordinate of the first vertex by equation as follows:
Ya=ΔYa×((Ya−pixel)−Ynmr/2), where Ya is the Y coordinate of the first vertex and Ya−pixel is the position of the sensor element in the Y-axis direction corresponding to the first vertex; and
obtaining the X and Y coordinates of the second vertex, the third vertex, and the fourth vertex, similarly.