Machine learning based cycle time tracking and reporting for vehicles
Systems and methods for machine-learning based cycle time tracking and reporting for vehicles are provided. A system includes a processor coupled with memory. The system identifies one or more models trained with machine learning relating to physical characteristics of vehicles and location designations associated with vehicle areas. The system receives, from one or more cameras, a video stream that captures a vehicle disposed in a vehicle area comprising a location designation. The system determines, based on an analysis of a plurality of frames of the video stream and via the one or more models, a type of the vehicle disposed in the vehicle area and a duration the vehicle is disposed in the vehicle area. The system performs, based on the type of the vehicle and a comparison of the duration of the vehicle with a threshold, an action to cause delivery of the vehicle from the vehicle area.
1 . A system, comprising:
one or more processors, coupled with memory, to:
receive, from one or more cameras, a video stream that captures a component for a vehicle, the component disposed in a predetermined area;
determine, via one or more models trained with machine learning and based on the video stream, a type of the component;
detect, responsive to the type of the component matching a predetermined type and an analysis of a plurality of frames of the video stream with the one or more models, a welding arc and a duration the component is disposed in the predetermined area subsequent to detection of the welding arc; and
perform, based on the duration, an action to control a welding process for the component of the vehicle.
2 . The system of claim 1 , comprising the one or more processors to:
identify, via the one or more models, a location of the welding arc on the component; and
provide, for display via a graphical user interface, an indication of the location of the welding arc on the component and the type of the component.
3 . The system of claim 1 , comprising the one or more processors to:
identify, via the video stream input into the one or more models, a plurality of locations of welding arcs on the component;
determine an amount of welding performed at each of the plurality of locations;
construct a heatmap that indicates the amount of welding performed at each of the plurality of locations; and
provide the heatmap for display via a graphical user interface.
4 . The system of claim 1 , comprising the one or more processors to:
identify, via the one or more models, a welder located within a distance threshold of a location of the welding arc;
determine a second duration the welder is located within the distance threshold of the location of the welding arc; and
select the action to control the welding process based on the second duration.
5 . The system of claim 1 , comprising the one or more processors to:
identify, via the one or more models, a count of welding arcs located within a distance threshold of the component; and
provide, for display via a graphical user interface, an indication of the count of welding arcs overlaid on a display of the video stream.
6 . The system of claim 1 , comprising the one or more processors to:
identify, via the one or more models, a count of welders located within a distance threshold of the component; and
provide, for display via a graphical user interface, an indication of the count of welders overlaid on a display of the video stream.
7 . The system of claim 1 , comprising the one or more processors to:
identify, via the one or more models, an inspector located within a distance threshold of a location of the welding arc;
determine a second duration the inspector is located within the distance threshold of the location of the welding arc;
determine a quality of the welding process based on the second duration; and
select the action to control the welding process based on the quality.
8 . The system of claim 1 , comprising the one or more processors to:
train the one or more models trained with machine learning using training data comprising historical image data comprising welding arcs emitted during historic welding processes under varied lighting conditions and manufacturing environments.
9 . The system of claim 1 , comprising the one or more processors to:
determine, via input of the video stream into the one or more models, a second duration of the welding process that lacks any welding arcs, inspectors, or welders within a distance threshold of the component; and
perform, based on the second duration greater than an idle time threshold, an action to control the welding process for the component of the vehicle.
10 . The system of claim 1 , comprising the one or more processors to:
generate a value of a performance metric for the welding process based on the duration; and
select the action to perform to improve the value of the performance metric.
11 . A method, comprising:
receiving, by one or more processors coupled with memory, from one or more cameras, a video stream that captures a component for a vehicle, the component disposed in a predetermined area;
determining, by the one or more processors via one or more models trained with machine learning and based on the video stream, a type of the component;
detecting, by the one or more processors, responsive to the type of the component matching a predetermined type and an analysis of a plurality of frames of the video stream with the one or more models, a welding arc and a duration the component is disposed in the predetermined area subsequent to detection of the welding arc; and
performing, by the one or more processors based on the duration, an action to control a welding process for the component of the vehicle.
12 . The method of claim 11 , comprising:
identifying, by the one or more processors via the one or more models, a location of the welding arc on the component; and
providing, by the one or more processors for display via a graphical user interface, an indication of the location of the welding arc on the component and the type of the component.
13 . The method of claim 11 , comprising:
identifying, by the one or more processors via the video stream input into the one or more models, a plurality of locations of welding arcs on the component;
determining, by the one or more processors, an amount of welding performed at each of the plurality of locations;
constructing, by the one or more processors, a heatmap that indicates the amount of welding performed at each of the plurality of locations; and
providing, by the one or more processors, the heatmap for display via a graphical user interface.
14 . The method of claim 11 , comprising:
identifying, by the one or more processors via the one or more models, a welder located within a distance threshold of a location of the welding arc;
determining, by the one or more processors, a second duration the welder is located within the distance threshold of the location of the welding arc; and
selecting, by the one or more processors, the action to control the welding process based on the second duration.
15 . The method of claim 11 , comprising:
identifying, by the one or more processors via the one or more models, a count of welding arcs located within a distance threshold of the component; and
providing, by the one or more processors for display via a graphical user interface, an indication of the count of welding arcs overlaid on a display of the video stream.
16 . The method of claim 11 , comprising:
identifying, by the one or more processors via the one or more models, a count of welders located within a distance threshold of the component; and
providing, by the one or more processors for display via a graphical user interface, an indication of the count of welders overlaid on a display of the video stream.
17 . The method of claim 11 , comprising:
identifying, by the one or more processors via the one or more models, an inspector located within a distance threshold of a location of the welding arc;
determining, by the one or more processors, a second duration the inspector is located within the distance threshold of the location of the welding arc;
determining, by the one or more processors, a quality of the welding process based on the second duration; and
selecting, by the one or more processors, the action to control the welding process based on the quality.
18 . The method of claim 11 , comprising:
training, by the one or more processors, the one or more models trained with machine learning using training data comprising historical image data comprising welding arcs emitted during historic welding processes under varied lighting conditions and manufacturing environments.
19 . The method of claim 11 , comprising:
determining, by the one or more processors via input of the video stream into the one or more models, a second duration of the welding process that lacks any welding arcs, inspectors, or welders within a distance threshold of the component; and
performing, by the one or more processors, based on the second duration greater than an idle time threshold, an action to control the welding process for the component of the vehicle.
20 . A vehicle manufacturing system, comprising:
markings surrounding a welding area in a manufacturing facility;
a camera oriented to capture image data from the welding area; and
one or more processors, coupled with memory, to:
receive, from the camera, a video stream that captures a component for a vehicle, the component disposed in the welding area;
determine, via one or more models trained with machine learning and based on the video stream, a type of the component;
detect, responsive to the type of the component matching a predetermined type and an analysis of a plurality of frames of the video stream with the one or more models, a welding arc and a duration the component is disposed in the predetermined area subsequent to detection of the welding arc; and
perform, based on the duration, an action to control a welding process for the component of the vehicle.