Item detection device, item detection method, and industrial vehicle
There is provided an item detection device that detects an item to be loaded and unloaded and includes an image acquisition unit acquiring a surrounding image obtained by capturing surroundings of the item detection device, an information image creation unit creating an information image, in which information related to a part to be loaded and unloaded in the item has been converted into an easily recognizable state, on the basis of the surrounding image, and a computing unit computing at least one of a position and a posture of the part to be loaded and unloaded on the basis of the information image.
1. An item detection device that detects an item to be loaded and unloaded, the device comprising:
an image acquisition unit acquiring a surrounding image obtained by capturing surroundings of the item detection device;
an information image creation unit creating an information image, in which information related to a part to be loaded and unloaded in the item has been converted into an easily recognizable state, on the basis of the surrounding image; and
a computing unit computing at least one of a position and a posture of the part to be loaded and unloaded on the basis of the information image,
wherein the information image is an image obtained by projecting information acquired at a position where the surrounding image is acquired onto a plane that is set for a part of a surrounding structure that can approximate a front surface of the item
wherein a position and a posture of the plane is known during an operation in which the plane is set, and
wherein the plane is set during an operation of approaching an area in front of a front face of the shelf, and before entering the area, wherein the area is defined by two imaginary planes extending from opposing side faces of the shelf.
2. The item detection device according to claim 1 , further comprising:
an adjustment unit adjusting conditions for creating the information image.
3. The item detection device according to claim 1 ,
wherein the information image creation unit associates dimensions corresponding to one pixel with the information image, and
the computing unit performs computation on the basis of a relationship between the pixel of the information image and dimensions of the part to be loaded and unloaded.
4. The item detection device according to claim 1 ,
wherein the computing unit performs template matching between information related to an edge portion of the part to be loaded and unloaded detected from the information image and actual dimension information of the part to be loaded and unloaded stored in advance in a storage unit.
5. The item detection device according to claim 1 , further comprising:
a feature plane setting unit setting a feature plane onto which features of the part to be loaded and unloaded in the item are projected,
wherein the feature plane setting unit generates a three-dimensional restored shape related to the item and surroundings of the item on the basis of a plurality of the surrounding images captured at different positions and sets the feature plane on the basis of the restored shape, and
the information image creation unit creates the information image using the feature plane.
6. The item detection device according to claim 5 ,
wherein the feature plane setting unit sets the feature plane using a moving plane that moves in synchronization with movement of a place where the surrounding image is captured.
7. The item detection device according to claim 1 ,
wherein the surrounding image is an image acquired by a fisheye camera or a wide-angle camera.
8. The item detection device according to claim 3 ,
wherein the item is a pallet, the information image has a pallet candidate portion indicating a region in which the pallet is likely to be present, and
the computing unit has a shape pattern having a first region and a second region that imitate a shape of the pallet, applies the shape pattern to each of the pallet candidate portions, and calculates a degree of uniformity indicating a degree to which the first region and the second region are uniform regions from pixel value histograms in the first region and the second region.
9. The item detection device according to claim 8 ,
wherein the computing unit calculates the degree of uniformity on the basis of a sum of the number of pixels having pixel values in peak neighborhood regions in the pixel value histograms with respect to the total number of pixels in the first region and the second region.
10. The item detection device according to claim 8 ,
wherein the computing unit further has a similar shape pattern that imitates an object similar to the pallet and identifies the pallet from the pallet candidate portion using the shape pattern and the similar shape pattern.
11. The item detection device according to claim 8 ,
wherein the computing unit sets the pallet candidate portion, the first region, and the second region in a rectangular shape and uses an integral image to calculate the pixel value histograms.
12. The item detection device according to claim 2 ,
wherein, from a geometric relationship between a position where the surrounding image is acquired and a projection surface onto which information acquired at the position is projected, the adjustment unit estimates an inclination of a ridge line of a ridge-type peak in a variation in a degree of matching with respect to a variation in a position and a posture of the projection surface, and searches for a local maximum value of the degree of matching on the basis of a direction of the inclination.
13. The item detection device according to claim 4 ,
wherein the computing unit corrects a template used in the template matching on the basis of an angle formed between a viewing direction from an imaging unit that acquires the surrounding image to the item and the part to be loaded and unloaded.
14. An item detection method that detects an item to be loaded and unloaded, the method comprising:
an image acquisition step of acquiring a surrounding image obtained by capturing surroundings;
an information image creation step of creating an information image, in which information related to a part to be loaded and unloaded in the item has been converted into an easily recognizable state, on the basis of the surrounding image; and
a computing step of computing at least one of a position and a posture of the part to be loaded and unloaded on the basis of the information image,
wherein the information image is an image obtained by projecting information acquired at a position where the surrounding image is acquired onto a plane that is set for a part of a surrounding structure that can approximate a front surface of the item,
wherein a position and a posture of the plane is known during an operation in which the plane is set, and
wherein the plane is set during an operation of approaching an area in front of a front face of the shelf, and before entering the area, wherein the area is defined by two imaginary planes extending from opposing side faces of the shelf.
15. An industrial vehicle comprising:
a vehicle body;
an imaging unit capturing an image of surroundings of the vehicle body; and
a control unit performing control to detect an item to be loaded and unloaded on the basis of the image acquired by the imaging unit,
wherein the control unit includes:
an image acquisition unit acquiring a surrounding image obtained by capturing the surroundings of the vehicle body from the imaging unit;
an information image creation unit creating an information image, in which information related to a part to be loaded and unloaded in the item has been converted into an easily recognizable state, on the basis of the surrounding image; and
a computing unit computing at least one of a position and a posture of the part to be loaded and unloaded on the basis of the information image,
wherein the information image is an image obtained by projecting information acquired at a position where the surrounding image is acquired onto a plane that is set for a part of a surrounding structure that can approximate a front surface of the item,
wherein a position and a posture of the plane is known during an operation in which the plane is set, and
wherein the plane is set during an operation of approaching an area in front of a front face of the shelf, and before entering the area, wherein the area is defined by two imaginary planes extending from opposing side faces of the shelf.
16. The industrial vehicle according to claim 15 ,
wherein the control unit controls the position or the posture of the vehicle body on the basis of information related to at least one of a position and a posture of the item.