Granted Patent B2

US 12,272,086 · App. 17/775,389 · Granted Apr 8, 2025

Inventor: Youhei Nishi (Kanagawa, JP)

Assignee: NEC Platforms, Ltd.

G06T7/60G06T7/73G06T2207/10028

Abstract

Claims (57)

1. A package measuring apparatus comprising:

a depth sensor disposed at a position at which the depth sensor is able to pick-up an image of a package having a rectangular parallelepiped shape and placed on a mounting table from substantially directly above the package, and configured to acquire a distance to the package by picking-up the image of the package;

at least one memory; and

at least one processor coupled to the at least one memory and configured to calculate a size of the package based on data acquired from the depth sensor,

wherein the at least one processor performs processes of:

picking-up an image of the mounting table on which nothing is placed using the depth sensor in advance and storing the picked-up image as background depth image data;

controlling the depth sensor, acquiring depth image data of the package by picking-up the image of the package placed on the mounting table from substantially directly above the package using the depth sensor, and specifying data of the package in the depth image data of the package by comparing the depth image data of the package with the background depth image data;

calculating a height of the package based on data of a distance from the depth sensor to an upper surface of the package in the depth image data of the package and data of a distance from the depth sensor to an upper surface of the mounting table in the background depth image data of the package;

specifying a position of a first vertex positioned on a leftmost side in the depth image data of the package, a position of a second vertex positioned on a rightmost side in the depth image data of the package, and a position of a third vertex positioned on an uppermost side or a lowermost side in the depth image data of the package;

obtaining two-dimensional coordinates of the first vertex, the second vertex, and the third vertex in a space in which a center of the depth sensor is set as a point of origin based on a view angle of the depth sensor, data of a distance from the depth sensor to each of the first vertex, the second vertex, and the third vertex, respectively, the number of sensor elements of the depth sensor, and data of a position of each sensor element of the depth sensor corresponding to each of the first vertex, the second vertex, and the third vertex, respectively; and

calculating, based on the two-dimensional coordinates of the first vertex, the second vertex, and the third vertex, a length of a first side defined between the first vertex and the third vertex, and a length of a second side defined between the second vertex and the third vertex, to output a calculated result.

2. The package measuring apparatus according to claim 1 ,

wherein the at least one processor performs processes of:

assuming that an image pick-up direction of the depth sensor is represented as a Z-axis, a direction coplanar with the Z-axis and orthogonal to the Z-axis is represented as an X-axis, and a direction orthogonal to the X-axis and the Z-axis is represented as a Y-axis, based on a view angle of the depth sensor in an X-axis direction and data of a distance of the first vertex, the second vertex, and the third vertex in a Z-axis direction, obtaining respective actual distances from the Z-axis to an image edge in the X-axis direction at the first vertex, the second vertex, and the third vertex, and based on a view angle of the depth sensor in a Y-axis direction and data of a distance of the first vertex, the second vertex, and the third vertex in the Z-axis direction, obtaining respective actual distances from the Z-axis to an image edge in the Y-axis direction at the first vertex, the second vertex, and the third vertex;

based on the actual distances from the Z-axis to the image edge in the X-axis direction at the first vertex, the second vertex, and the third vertex and the number of sensor elements of the depth sensor in the X-axis direction, obtaining respective rates of change of the actual distance of each of the sensor elements in the X-axis direction at the first vertex, the second vertex, and the third vertex, and based on the actual distances from the Z-axis to the image edge in the Y-axis direction at the first vertex, the second vertex, and the third vertex and the number of sensor elements of the depth sensor in the Y-axis direction, obtaining respective rates of change of the actual distance of each of the sensor elements in the Y-axis direction at the first vertex, the second vertex, and the third vertex; and

based on positions of the sensor elements in the X-axis direction at the first vertex, the second vertex, and the third vertex and the rates of change of the actual distance of each of the sensor elements in the X-axis direction, obtaining respective X-coordinates at the first vertex, the second vertex, and the third vertex, and based on positions of the sensor elements in the Y-axis direction at the first vertex, the second vertex, and the third vertex and the rates of change of the actual distance of each of the sensor elements in the Y-axis direction, obtaining respective Y-coordinates at the first vertex, the second vertex, and the third vertex, thereby forming two-dimensional coordinates of the first vertex, the second vertex, and the third vertex.

3. The package measuring apparatus according to claim 2 ,

wherein the at least one processor performs processes of:

assuming that a Z-coordinate of the first vertex is Za, the view angle of the depth sensor in the X-axis direction is θx, and the number of sensor elements in the X-axis direction is Xnmr based on the data of the distance from the depth sensor at the first vertex, obtaining the rate of change ΔXa of the actual distance of each of the sensor elements in the X-axis direction at the first vertex as represented by an equation below;

Δ Xa=Za ×tan (θ x/ 2)/( Xnmr/ 2)

assuming that the view angle of the depth sensor in the Y-axis direction is θy and the number of sensor elements in the Y-axis direction is Ynmr, obtaining the rate of change ΔYa of the actual distance of each of the sensor elements in the Y-axis direction at the first vertex as represented by an equation below;

Δ Ya=Za ×tan (θ y/ 2)/( Ynmr/ 2)

assuming that an X-coordinate of the first vertex is Xa and the position of the sensor element in the X-axis direction at the first vertex is Xa-pixel, obtaining the X-coordinate of the first vertex as represented by an equation below;

Xa=ΔXa ×(( Xa −pixel)— Xnmr/ 2)

assuming that a Y-coordinate of the first vertex is Ya and the position of the sensor element in the Y-axis direction at the first vertex is Ya-pixel, obtaining the Y-coordinate of the first vertex as represented by an equation below; and

Ya=ΔYa ×(( Ya −pixel)− Ynmr/ 2)

obtaining X-coordinates and Y-coordinates of the second vertex and the third vertex in the same manner.

4. The package measuring apparatus according to claim 1 , wherein the at least one processor performs a process of specifying the first vertex, the second vertex, and the third vertex, based on the depth image data of the package.

5. The package measuring apparatus according to claim 1 , further comprising an image pick-up apparatus,

wherein the at least one processor performs a process of specifying the first vertex, the second vertex, and the third vertex by performing image recognition on an image acquired by the image pick-up apparatus.

6. The package measuring apparatus according to claim 5 , further comprising a display unit,

wherein the at least one processor causes the image pick-up apparatus to pick up an image of the package on the mounting table, causes the display unit to display the image of the package, and causes the display unit to display a message prompting adjustment of a mounting state of the package when at least one of the first vertex, the second vertex, and the third vertex of the package is not able to be specified.

7. The package measuring apparatus according to claim 1 , further comprising the mounting table,

wherein the depth sensor is attached to the mounting table.

8. The package measuring apparatus according to claim 1 , further comprising a weighing scale inside the mounting table to measure a weight of the package and output a result of measurement.

9. A package accepting system comprising:

the package measuring apparatus according to claim 1 ;

an information acquisition apparatus; and

a payment processing apparatus, wherein

the information acquisition apparatus acquires information related to shipment of the package,

the package measuring apparatus calculates a shipping charge of the package based on a size of the package and the information related to the shipment of the package, and

the payment processing apparatus performs payment processing for the shipping charge of the package.

10. A package measuring method for calculating a size of a package using a depth sensor disposed at a position at which the depth sensor is able to pick-up an image of a package having a rectangular parallelepiped shape and placed on a mounting table from substantially directly above the package, and configured to acquire a distance to the package by picking-up the image of the package, the method comprising:

picking-up an image of the mounting table on which nothing is placed using the depth sensor in advance and storing the picked-up image as background depth image data;

controlling the depth sensor, acquiring depth image data of the package by picking-up the image of the package placed on the mounting table from substantially directly above the package using the depth sensor, and specifying data of the package in the depth image data of the package by comparing the depth image data of the package with the background depth image data;

calculating a height of the package based on data of a distance from the depth sensor to an upper surface of the package in the depth image data of the package and data of a distance from the depth sensor to an upper surface of the mounting table in the background depth image data of the package;

specifying a position of a first vertex positioned on a leftmost side in the depth image data of the package, a position of a second vertex positioned on a rightmost side in the depth image data of the package, and a position of a third vertex positioned on an uppermost side or a lowermost side in the depth image data of the package;

obtaining two-dimensional coordinates of the first vertex, the second vertex, and the third vertex in a space in which a center of the depth sensor is set as a point of origin based on a view angle of the depth sensor, data of a distance from the depth sensor to each of the first vertex, the second vertex, and the third vertex, respectively, the number of sensor elements of the depth sensor, and data of a position of each sensor element of the depth sensor corresponding to each of the first vertex, the second vertex, and the third vertex, respectively; and

calculating, based on the two-dimensional coordinates of the first vertex, the second vertex, and the third vertex, a length of a first side defined between the first vertex and the third vertex, and a length of a second side defined between the second vertex and the third vertex, to output a calculated result.

11. A non-transitory computer readable medium storing a program, the program being for causing a computer to perform calculation processing of calculating a size of a package using a depth sensor disposed at a position at which the depth sensor is able to pick-up an image of a package having a rectangular parallelepiped shape and placed on a mounting table from substantially directly above the package, and configured to acquire a distance to the package by picking-up the image of the package,

the calculation processing including:

picking-up an image of the mounting table on which nothing is placed using the depth sensor in advance and storing the picked-up image as background depth image data;

controlling the depth sensor, acquiring depth image data of the package by picking-up the image of the package placed on the mounting table from substantially directly above the package using the depth sensor, and specifying data of the package in the depth image data of the package by comparing the depth image data of the package with the background depth image data;

calculating a height of the package based on data of a distance from the depth sensor to an upper surface of the package in the depth image data of the package and data of a distance from the depth sensor to an upper surface of the mounting table in the background depth image data of the package;

specifying a position of a first vertex positioned on a leftmost side in the depth image data of the package, a position of a second vertex positioned on a rightmost side in the depth image data of the package, and a position of a third vertex positioned on an uppermost side or a lowermost side in the depth image data of the package;

obtaining two-dimensional coordinates of the first vertex, the second vertex, and the third vertex in a space in which a center of the depth sensor is set as a point of origin based on a view angle of the depth sensor, data of a distance from the depth sensor to each of the first vertex, the second vertex, and the third vertex, respectively, the number of sensor elements of the depth sensor, and data of a position of each sensor element of the depth sensor corresponding to each of the first vertex, the second vertex, and the third vertex, respectively; and

calculating, based on the two-dimensional coordinates of the first vertex, the second vertex, and the third vertex, a length of a first side defined between the first vertex and the third vertex, and a length of a second side defined between the second vertex and the third vertex, to output a calculated result.

Assignments (2)

Priority Claims (1)

Continuity (1)

References Cited (18)