Information processing device, method and medium

1 . An information processing device comprising:

at least one memory configured to store program code; and

at least one processor coupled to the memory, the at least one processor being configured to access the at least one memory and operate as instructed by the program code, the program code including:

acquiring code configured to cause at least one of the at least one processor to acquire, by inputting a target image showing a fastening part that is an inspected target to a machine learning model, a feature area including a feature point relating to reference indication added to the fastening part, the feature area being based on an estimated coordinate relating to a position of the feature point and a distribution of the estimated coordinate; and

determining code configured to:

cause at least one of the at least one processor to determine whether the feature area satisfies a prescribed reference, and

determine a fastening state of the fastening part based on the estimated coordinate when the feature area satisfies the prescribed reference.

2 . The information processing device according to claim 1 , wherein

the fastening part has a fastening material and a fastened target that are screw-fastened,

the reference indication is one that is added to the fastening material and the fastened target after primary fastening, and that shows a change in a positional relationship between the reference indication on the fastening material and the reference indication on the fastened target when the secondary fastening is performed, and

the program code is configured to cause at least one of the at least one processor to determine the fastening state of the fastening part based on the estimated coordinate of the feature point relating to each of the reference indication on the fastening material and the reference indication on the fastened target that show the change in positional relationship when the secondary fastening is performed.

3 . The information processing device according to claim 2 , wherein

the feature point includes a feature point relating to each of a fastening shaft in the fastening part, the reference indication on the fastening material, and the reference indication on the fastened target.

4 . The information processing device according to claim 2 , wherein

the fastening material has a bolt and a nut, and

the feature point includes a feature point showing each of a fastening shaft in the fastening part, a position of the reference indication at an outer edge of a tip end of the bolt, a position of the reference indication at an inner edge of a surface of the nut, a position of the reference indication at an outer edge of the surface of the nut, and a position of the reference indication on the fastened target.

5 . The information processing device according to claim 4 , wherein

the program code is further configured to cause at least one of the at least one processor to determine that the fastening part is properly fastened when an angle formed by a first approximate straight line passing through the fastening shaft in the fastening part, the position of the reference indication at the outer edge of the tip end of the bolt, and the position of the reference indication on the fastened target and a second approximate straight line passing through the fastening shaft in the fastening part, the position of the reference indication at the inner edge of the surface of the nut, and the position of the reference indication at the outer edge of the surface of the nut falls within a prescribed range.

6 . The information processing device according to claim 1 , wherein

the estimated coordinate is a two-dimensional coordinate, and

the distribution of the estimated coordinate is a standard deviation for each of two axes constituting the two-dimensional coordinate.

7 . The information processing device according to claim 1 , wherein

the program code is further configured to cause at least one of the at least one processor to determine the fastening state of the fastening part as being unsure when the feature area does not satisfy the prescribed reference.

8 . The information processing device according to claim 1 , wherein the program code is further configured to cause at least one of the at least one processor to:

generate the machine learning model using teacher data in which a teacher image obtained by capturing an image of a screw-fastened fastening part is an input value and a coordinate showing a position of a feature point in the teacher image relating to reference indication added to the fastening part is an output value.

9 . The information processing device according to claim 1 , wherein the program code is further configured to cause at least one of the at least one processor to:

notify a user of a result determined by the at least one processor and/or a result determined by the at least one processor.

10 . The information processing device according to claim 1 , wherein the program code is further configured to cause at least one of the at least one processor to:

acquire a captured image as an inspected target; and

acquire the target image by specifying a portion in which an image of the fastening part in the captured image is captured and cutting out the portion in which the image of the fastening part is captured from the captured image.

11 . The information processing device according to claim 1 , wherein the program code is further configured to cause at least one of the at least one processor to:

perform, before processing by the at least one processor, image transformation to cancel projection applied when the target image is captured.

12 . The information processing device according to claim 1 , wherein the program code is further configured to cause at least one of the at least one processor to:

transform, before processing by the at least one processor, a coordinate of the feature point detected by the at least one processor so that projection applied when the target image is captured is canceled.

13 . The information processing device according to claim 1 , wherein a size of the feature area is specified by the distribution of the estimated coordinate,

wherein the prescribed reference is a prescribed reference with respect to the size of the feature area, and

wherein the program code is further configured to cause at least one of the at least one processor to determine the fastening state of the fastening part using the estimated coordinate only when it is determined that the feature area satisfies the prescribed reference.

14 . An information processing method in which a computer performs:

acquiring, by inputting a target image showing a fastening part that is an inspected target to a machine learning model, a feature area including a feature point relating to reference indication added to the fastening part, the feature area being based on an estimated coordinate relating to a position of the feature point and a distribution of the estimated coordinate;

determining whether the feature area satisfies a prescribed reference; and

determining a fastening state of the fastening part based on the estimated coordinate when the feature area satisfies the prescribed reference.

15 . A non-transitory computer-readable recording medium having recorded thereon an information processing program configured to cause a computer to execute:

acquiring, by inputting a target image showing a fastening part that is an inspected target to a machine learning model, a feature area including a feature point relating to reference indication added to the fastening part, the feature area being based on an estimated coordinate relating to a position of the feature point and a distribution of the estimated coordinate;

determining whether the feature area satisfies a prescribed reference; and

determining a fastening state of the fastening part based on the estimated coordinate when the feature area satisfies the prescribed reference.