Image-based search and prediction system for physical defect investigations
Techniques are provided for image-based search and prediction for physical defect investigations. One method comprises obtaining, for multiple investigations of physical defects associated with corresponding devices, (i) images of the physical defects, (ii) descriptions of the physical defects, and (iii) descriptions of a manual classification of the physical defects; storing the images for each investigation in an image repository, wherein each stored image comprises a set of tags based on the corresponding description of the physical defect and the corresponding description of the manual classification of the physical defect; and training an image classification model using the stored images, each with the corresponding description of the physical defect and the corresponding manual classification of the physical defect. A given investigation of a physical defect can be resolved using the image repository and/or the trained image classification model.
1. A method, comprising:
obtaining, for each of a plurality of investigations of a physical defect associated with a corresponding device, (i) one or more images of the physical defect, (ii) a textual description of the physical defect, and (iii) a description of a classification of the physical defect; and
storing the one or more images for each investigation in a searchable image repository, wherein each stored image comprises a set of tags based at least in part on the corresponding textual description of the physical defect, the corresponding description of the classification of the physical defect and a description of the corresponding device, wherein the set of tags is stored with the respective stored image as metadata, and wherein the searchable image repository comprises an index based at least in part on the set of tags for a plurality of stored images;
wherein a given investigation of a physical defect is resolved at least in part by applying a textual description of the device associated with the given investigation and at least one symptom of the physical defect associated with the given investigation to an image-based search engine that identifies one or more images in the searchable image repository related to the applied textual description using the index of the searchable image repository;
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.
2. The method of claim 1 , wherein each textual description of the physical defect is obtained from a respective customer.
3. The method of claim 1 , wherein each description of the classification of a resolution of a respective physical defect investigation is obtained from a corresponding support agent that investigated the respective physical defect.
4. The method of claim 1 , further comprising training one or more image classification models using each of at least a plurality of the stored images with the corresponding textual description of the physical defect and the corresponding description of the classification of the physical defect by applying each stored image with the corresponding textual description of the physical defect to a training process, wherein the corresponding description of the classification of the physical defect is provided as a classification label.
5. The method of claim 4 , further comprising generating a workflow recommendation of a next action by a support agent using one or more of the searchable image repository and the one or more trained image classification models.
6. The method of claim 5 , wherein the one or more trained image classification models provide a predicted cause of the physical defect and a policy-based recommendation for a treatment of the given investigation under an applicable warranty.
7. The method of claim 1 , further comprising identifying one or more images in the searchable image repository relevant to the given investigation by applying a textual description of the physical defect associated with the given investigation to the image-based search engine.
8. An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
the at least one processing device being configured to implement the following steps:
obtaining, for each of a plurality of investigations of a physical defect associated with a corresponding device, (i) one or more images of the physical defect, (ii) a textual description of the physical defect, and (iii) a description of a classification of the physical defect; and
storing the one or more images for each investigation in a searchable image repository, wherein each stored image comprises a set of tags based at least in part on the corresponding textual description of the physical defect, the corresponding description of the classification of the physical defect and a description of the corresponding device, wherein the set of tags is stored with the respective stored image as metadata, and wherein the searchable image repository comprises an index based at least in part on the set of tags for a plurality of stored images;
training one or more image classification models using each of at least a plurality of the stored images with the corresponding textual description of the physical defect and the corresponding description of the classification of the physical defect,
wherein a given investigation of a physical defect is resolved at least in part by applying a textual description of the device associated with the given investigation and at least one symptom of the physical defect associated with the given investigation to an image-based search engine that identifies one or more images in the searchable image repository related to the applied textual description using the index of the searchable image repository.
9. The apparatus of claim 8 , wherein each textual description of the physical defect is obtained from a respective customer.
10. The apparatus of claim 8 , wherein each description of the classification of a resolution of a respective physical defect investigation is obtained from a corresponding support agent that investigated the respective physical defect.
11. The apparatus of claim 8 , further comprising training one or more image classification models using each of at least a plurality of the stored images with the corresponding textual description of the physical defect and the corresponding description of the classification of the physical defect by applying each stored image with the corresponding textual description of the physical defect to a training process, wherein the corresponding description of the classification of the physical defect is provided as a classification label.
12. The apparatus of claim 11 , further comprising generating a workflow recommendation of a next action by a support agent using one or more of the searchable image repository and the one or more trained image classification models.
13. The apparatus of claim 12 , wherein the one or more trained image classification models provide a predicted cause of the physical defect and a policy-based recommendation for a treatment of the given investigation under an applicable warranty.
14. The apparatus of claim 8 , further comprising identifying one or more images in the searchable image repository relevant to the given investigation by applying a textual description of the physical defect associated with the given investigation to the image-based search engine.
15. A non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device to perform the following steps:
obtaining, for each of a plurality of investigations of a physical defect associated with a corresponding device, (i) one or more images of the physical defect, (ii) a textual description of the physical defect, and (iii) a description of a classification of the physical defect; and
storing the one or more images for each investigation in a searchable image repository, wherein each stored image comprises a set of tags based at least in part on the corresponding textual description of the physical defect, the corresponding description of the classification of the physical defect and a description of the corresponding device, wherein the set of tags is stored with the respective stored image as metadata, and wherein the searchable image repository comprises an index based at least in part on the set of tags for a plurality of stored images;
training one or more image classification models using each of at least a plurality of the stored images with the corresponding textual description of the physical defect and the corresponding description of the classification of the physical defect,
wherein a given investigation of a physical defect is resolved at least in part by applying a textual description of the device associated with the given investigation and at least one symptom of the physical defect associated with the given investigation to an image-based search engine that identifies one or more images in the searchable image repository related to the applied textual description using the index of the searchable image repository.
16. The non-transitory processor-readable storage medium of claim 15 , wherein each description of the classification of a resolution of a respective physical defect investigation is obtained from a corresponding support agent that investigated the respective physical defect.
17. The non-transitory processor-readable storage medium of claim 15 , further comprising training one or more image classification models using each of at least a plurality of the stored images with the corresponding textual description of the physical defect and the corresponding description of the classification of the physical defect by applying each stored image with the corresponding textual description of the physical defect to a training process, wherein the corresponding description of the classification of the physical defect is provided as a classification label.
18. The non-transitory processor-readable storage medium of claim 17 , further comprising generating a workflow recommendation of a next action by a support agent using one or more of the searchable image repository and the one or more trained image classification models.
19. The non-transitory processor-readable storage medium of claim 18 , wherein the one or more trained image classification models provide a predicted cause of the physical defect and a policy-based recommendation for a treatment of the given investigation under an applicable warranty.
20. The non-transitory processor-readable storage medium of claim 15 , further comprising identifying one or more images in the searchable image repository relevant to the given investigation by applying a textual description of the physical defect associated with the given investigation to the image-based search engine.