SCALABLE VECTOR CAGES: VECTOR-TO-PIXEL METADATA TRANSFER FOR OBJECT PART CLASSIFICATION
Improved and alternative processes for object segmentation segment from captured images are presented. According to an aspect there is provided, systems and methods for classifying segments of an object. The systems and methods include processing captured images using a plurality of cages to identify a cage for image alignment, the cage defining segments of the object, aligning the captured images onto the cage to identify segments of the object in the captured images, and detecting one or more defects in the segments of the object in the captured images.
1 . A system for classifying segments of a vehicle, the system comprising:
a server having non-transitory computer readable storage medium with executable instructions for causing one or more processors to:
process captured images using a plurality of cages to identify a cage for image alignment, the cage defining segments of the vehicle, wherein the captured images are of the vehicle or at least a portion thereof;
align the captured images onto the cage to identify segments of the vehicle in the captured images;
detect one or more physical conditions of the segments of the vehicle in the captured images.
2 . (canceled)
3 . The system of claim 1 , wherein the server is further configured to cause one or more processors to configure an interface application with an object capture module to capture the images of the vehicle, wherein the object capture module captures metadata for the captured images and the server is further configured to use the metadata to process the captured images.
4 . (canceled)
5 . The system of claim 1 , wherein the cage is for an object type, wherein the server retrieves the object type based on a serial or identification number of the object, wherein the server is further configured to identify the object type based on the captured images by extracting a serial or identification number of the vehicle from an image.
6 . (canceled)
7 . The system of claim 1 , wherein the cage comprises one or more different cage views.
8 . The system of claim 1 , wherein the cage can be rendered with segment outlines of varying widths, wherein the cage comprises a graphic script in a domain specific language.
9 . The system of claim 1 , wherein the cage comprises a 3D model.
10 . The system of claim 1 , the server is further configured to generate a virtual representation of the vehicle using the captured images and the cage.
11 . (canceled)
12 . The system of claim 1 , wherein the server is further configured to compute cost data for repair of the one or more defects.
13 . The system of claim 1 , wherein the processor is configured to align the captured images onto the cage by:
generating a semantic mask from the captured images using semantic segmentation models; and
optimizing a homography matrix between the cage and the semantic mask, wherein the homography matrix defines a composition of rotations, translations, scaling, distortion correction and/or to align the cage with the semantic mask, wherein the cage and the semantic mask are compared as two binary arrays of a same shape.
14 . (canceled)
15 . The system of claim 1 wherein the server is further configured for training at least one machine learning model, the machine learning model using the cages to identify and assess conditions of components of the vehicle.
16 . A system for classifying segments of a vehicle the system comprising:
a server having non-transitory computer readable storage medium with executable instructions for causing one or more processors to configure:
an image alignment process for captured images to align the captured images onto a cage, the cage defining segments of the vehicle;
a recognition engine to process the captured images to classify one or more of the segments of the vehicle.
17 . A method for classifying segments of an object, the method comprising:
processing captured images using a plurality of cages to identify a cage for image alignment, the cage defining segments of the vehicle;
aligning the captured images onto the cage to identify segments of the vehicle in the captured images; and
detecting one or more physical conditions of the segments of the vehicle in the captured images.
18 . (canceled)
19 . (canceled)
20 . The method of claim 17 , the method further comprising:
capturing metadata for the captured images; and
using the metadata to at least one of align the captured images and detect the one or more defects.
21 . The method of claim 17 , wherein the cage is for an object type, the method comprising retrieving the object type based on a serial or identification number of the object.
22 . (canceled)
23 . The method of claim 17 , wherein the cage comprises one or more different cage views.
24 . The method of claim 17 , wherein the cage can be rendered with segment outlines of varying widths.
25 . The method of claim 17 , wherein the cage comprises a 3D model.
26 . The method of claim 17 , the method further comprising generating a virtual representation of the object using the captured images and the cage.
27 . The method of claim 21 , the method further comprising identifying the object type based on the captured images.
28 . (canceled)
29 . The method of claim 17 , wherein the aligning the captured images onto the cage comprises:
generating a semantic mask from the captured images using semantic segmentation models; and
optimizing a homography matrix between the cage and the semantic mask, wherein the homography matrix defines a composition of rotations, translations, scaling and/or distortion correction to align the cage with the semantic mask, wherein the cage and the semantic mask are compared as two binary arrays of a same shape.
30 - 45 . (canceled)