Information processing device, information processing method, and storage medium

1. An information processing device comprising:

one or more processors, wherein the one or more processors function as a plurality of units comprising:

(1) a branch determining unit configured to determine a branch of a line component which is detected based on a local feature extracted from an input image obtained by image capturing of a structure;

(2) a specifying unit configured to specify a range of a defect based on a result of the determination by the branch determining unit, wherein the defect is a crack formed on the structure;

(3) an extracting unit configured to extract a feature pertaining to a partial image of the defect from the input image, on the basis of a result of the specifying the range of the defect; and

(4) an attribute determining unit configured to determine an attribute of the defect using the feature pertaining to the partial image of the defect.

2. The information processing device according to claim 1 , wherein the extracting unit is further configured to extract a feature from the partial image of the defect as the feature pertaining to the partial image of the defect.

3. The information processing device according to claim 1 , wherein the extracting unit is further configured to extract a histogram feature from the partial image of the defect as the feature pertaining to the partial image of the defect, and

wherein the histogram feature is represented by numbers of pixels in the partial image for respective pixel value ranges.

4. The information processing device according to claim 1 , wherein the attribute determining unit is further configured to determine to which of a plurality of classes the attribute of the defect belongs, using multi-class classification.

5. The information processing device according to claim 4 , wherein the plurality of classes include a class indicating that the defect has been false positive.

6. The information processing device according to claim 1 , wherein the attribute determining unit is further configured to determine the attribute of the defect using a trained classifier.

7. The information processing device according to claim 1 , wherein the extracting unit is further configured to extract a feature from the partial image of the defect in accordance with a method selected in accordance with the size of the defect.

8. The information processing device according to claim 1 , wherein the specifying unit is further configured to specify a position of the defect, and

wherein the one or more processors further function as a setting unit configured to set a region of the partial image of the defect in accordance with the position of the defect.

9. The information processing device according to claim 8 , wherein the setting unit is further configured to set a plurality of regions in the partial image for a single defect, and

wherein the attribute determining unit is further configured to determine an attribute of the defect a plurality of times using each of the plurality of regions set in the partial image.

10. The information processing device according to claim 1 , wherein the specifying unit is further configured to individually specify a plurality of ranges of respective defects appearing in the input image, and

wherein the attribute determining unit is further configured to determine the attribute for each of the defects.

11. The information processing device according to claim 1 , wherein the branch determining unit is further configured to (a) detect a crack region using local features extracted from each of positions in the input image and (b) derive the line component indicating the position of the crack by thinning the crack region, and

wherein the specifying unit is further configured to (a) divide the line component in accordance with changes in the direction of the line component and (b) derive line components indicating the positions of individual cracks.

12. The information processing device according to claim 1 , wherein the one or more processors further function as a display control unit configured to cause a display device to display the position of the defect.

13. The information processing device according to claim 12 , wherein the display control unit is further configured to cause the display device to display the attribute of the defect along with the position of the defect.

14. The information processing device according to claim 13 , wherein the one or more processors further function as a calculating unit configured to calculate a reliability of the attribute, and

wherein the display control unit is further configured to cause the display device to display the reliability of the attribute along with the attribute of the defect.

15. The information processing device according to claim 14 , wherein the display control unit is further configured to change the display format of the defect on the basis of the reliability of the attribute of the defect.

16. The information processing device according to claim 1 , wherein the attribute determining unit is further configured to determine, as the attribute of the defect, (a) a width of the defect, (b) a level of danger posed by the defect, or (c) a cause of the defect.

17. The information processing device according to claim 1 , wherein the specifying unit is further configured to (a) separate the line component into a plurality of cracks, wherein at least one crack of the plurality of cracks corresponds to a series of connected polylines extending through the branch determined by the branch determining unit and (b) specify a range of a single crack of the plurality of cracks resulting from the separation, the single crack corresponding to the series of connected polylines extending through the branch,

wherein the extracting unit is further configured to extract a feature pertaining to a partial image of the single crack corresponding to the series of connected polylines extending through the branch, and

wherein the attribute determining unit is further configured to determine an attribute of the single crack corresponding to the series of connected polylines extending through the branch, using the feature pertaining to the partial image of the single crack.

18. The information processing device according to claim 1 , wherein the one or more processors further function as a defect detecting unit configured to detect a defect of an object in an input image,

wherein the defect detecting unit is further configured to detect the defect in accordance with a first feature of each pixel in the input image, obtained using an output from an intermediate layer of a CNN (Convolutional Neural Network) into which the input image has been input, and

wherein the extracting unit is further configured to extract a feature pertaining to the partial image of the defect in accordance with a second feature of each pixels in the partial image of the defect obtained using the output from the intermediate layer of the CNN.

19. An information processing device comprising:

one or more processors, wherein the one or more processors function as a plurality of units comprising:

(1) a defect detecting unit configured to detect a defect of an object in an input image;

(2) an extracting unit configured to extract a feature pertaining to a partial image of the defect from the input image, on the basis of a result of detecting the defect; and

(3) an attribute determining unit configured to determine an attribute of the defect using the feature pertaining to the partial image of the defect,

wherein the defect detecting unit is further configured to detect the defect in accordance with a first feature of each pixel in the input image, obtained using an output from an intermediate layer of a CNN (Convolutional Neural Network) into which the input image has been input, and

wherein the extracting unit is further configured to extract a feature pertaining to the partial image of the defect in accordance with a second feature of each pixel in the partial image of the defect obtained using the output from the intermediate layer of the CNN.

20. The information processing device according to claim 19 , wherein the extracting unit is further configured to extract a feature from the partial image of the defect in accordance with a method selected in accordance with the size of the defect.

21. The information processing device according to claim 19 , wherein the specifying unit is further configured to specify a position of the defect, and

wherein the one or more processors further function as a setting unit configured to set a region of the partial image of the defect in accordance with the position of the defect.

22. The information processing device according to claim 21 , wherein the setting unit is further configured to set a plurality of regions in the partial image for a single defect, and

wherein the attribute determining unit is further configured to determine an attribute of the defect a plurality of times using each of the plurality of regions set in the partial image.

23. The information processing device according to claim 19 , wherein the specifying unit is further configured to individually specify a plurality of ranges of respective defects appearing in the input image, and

wherein the attribute determining unit is further configured to determine the attribute for each of the defects.

24. The information processing device according to claim 19 , wherein the one or more processors further function as a display control unit configured to cause a display device to display the position of the defect.

25. The information processing device according to claim 24 , wherein the display control unit is further configured to cause the display device to display the attribute of the defect along with the position of the defect.

26. The information processing device according to claim 25 , wherein the one or more processors further function as a calculating unit configured to calculate a reliability of the attribute, and

wherein the display control unit is further configured to cause the display device to display the reliability of the attribute along with the attribute of the defect.

27. The information processing device according to claim 26 , wherein the display control unit is further configured to change the display format of the defect on the basis of the reliability of the attribute of the defect.

28. The information processing device according to claim 19 , wherein the object is a structure, and

wherein the defect is a crack in the structure.

29. The information processing device according to claim 19 , wherein the attribute determining unit is further configured to determine, as the attribute of the defect, (a) a width of the defect, (b) a level of danger posed by the defect, or (c) a cause of the defect.

30. An information processing method comprising:

determining a branch of a line component which is detected based on a local feature extracted from an input image obtained by image capturing of a structure;

specifying a range of a defect based on a result of the determining, wherein the defect is a crack formed on the structure;

extracting a feature pertaining to a partial image of the defect from the input image, on the basis of a result of the specifying the range of the defect; and

determining an attribute of the defect using the feature pertaining to the partial image of the defect.

31. A non-transitory computer-readable medium storing a program which, when executed by a computer comprising a processor and a memory, causes the computer to perform a method comprising:

determining a branch of a line component which is detected based on a local feature extracted from an input image obtained by image capturing of a structure;

specifying a range of a defect based on a result of the determining, wherein the defect is a crack formed on the structure;

extracting a feature pertaining to a partial image of the defect from the input image, on the basis of a result of the specifying the range of the defect; and

determining an attribute of the defect using the feature pertaining to the partial image of the defect.

32. An information processing method comprising:

detecting a defect of an object in an input image;

extracting a feature pertaining to a partial image of the defect from the input image, on the basis of a result of detecting the defect; and

determining an attribute of the defect using the feature pertaining to the partial image of the defect,

wherein the detecting comprises detecting the defect in accordance with a first feature of each pixel in the input image, obtained using an output from an intermediate layer of a CNN (Convolutional Neural Network) into which the input image has been input, and

wherein the extracting comprises extracting a feature pertaining to the partial image of the defect in accordance with a second feature of each pixel in the partial image of the defect obtained using the output from the intermediate layer of the CNN.

33. A non-transitory computer-readable medium storing a program which, when executed by a computer comprising a processor and a memory, causes the computer to perform a method comprising:

detecting a defect of an object in an input image;

extracting a feature pertaining to a partial image of the defect from the input image, on the basis of a result of detecting the defect; and

determining an attribute of the defect using the feature pertaining to the partial image of the defect,

wherein the detecting comprises detecting the defect in accordance with a first feature of each pixel in the input image, obtained using an output from an intermediate layer of a CNN (Convolutional Neural Network) into which the input image has been input, and

wherein the extracting comprises extracting a feature pertaining to the partial image of the defect in accordance with a second feature of each pixel in the partial image of the defect obtained using the output from the intermediate layer of the CNN.