Vehicle classification and vehicle pose estimation
A method of classifying a work machine within a worksite is provided. The method may include receiving one or more captured images of the work machine from one or more image capture devices, identifying at least two classifiers within the captured images of the work machine, determining spatial relationships between the classifiers within the captured images, determining a classification and a pose of the work machine based on the spatial relationships, and displaying analytics based on the classification and the pose of the work machine.
1. A method of classifying a work machine within a worksite, the method comprising:
receiving one or more captured images of the work machine from one or more image capture devices positioned around the worksite and remote from the work machine;
identifying at least two classifiers within the captured images of the work machine, wherein each classifier is indicative of a known component that is common to all work machines of the work machine type of the work machine;
determining spatial relationships between the classifiers within the captured images based on the relative positions of the classifiers as measured within the captured images;
determining a classification and a pose of the work machine based on the spatial relationships;
tracking the determined poses of the work machine over time;
determining analytics related to the work machine and the worksite based on the tracked poses, the analytics including actual data derived from the poses of the work machine and representing a volume of work actually performed by the work machine within a given timeframe, and model data representing a predetermined volume of work performed by the work machine within the given timeframe; and
displaying the analytics based on the classifications and the poses of the work machine.
2. The method of claim 1 , wherein the captured images include at least one video and the image capture devices includes at least one video camera installed in relation to one of the work machine and the worksite.
3. The method of claim 1 , wherein each of the classifiers, the classification and the pose is visually recognized based on reference data retrieved from a database, the reference data visually associating each of the classifiers to one of a plurality of known machine components and visually associating different spatial relationships to known classifications and poses.
4. The method of claim 3 , wherein the known machine components include two or more of wheels, tracks, linkages, machine implements, operator cabs, and work machines.
5. The method of claim 1 , wherein the analytics include one or more of work productivity of the work machine, work progress, a state of the work machine, and a state of a terrain within the worksite.
6. A system for classifying a work machine within a worksite, the system comprising:
one or more image capture devices configured to generate one or more captured images of the work machine; and
a visual recognition device in communication with the one or more image capture devices positioned around the worksite and remote from the work machine, the visual recognition device configured to
receive the captured images from the image capture devices,
identify at least two classifiers within the captured images wherein each classifier is indicative of a known component that is common to all work machines of the work machine type of the work machine,
determine spatial relationships between the classifiers within the captured images based on the relative positions of the classifiers as measured within the captured images,
determine a classification and a pose of the work machine based on the spatial relationships,
track the determined poses of the work machine over time,
determine analytics related to the work machine and the worksite based on the tracked poses, the analytics including actual data derived from the poses of the work machine and representing a volume of work actually performed by the work machine within a given timeframe, and model data representing a predetermined volume of work performed by the work machine within the given timeframe, and
cause the display of the analytics based on the classifications and the poses of the work machine.
7. The system of claim 6 , wherein the image capture devices include at least one video camera installed in relation to one of the work machine and the worksite, and the captured images include at least one video.
8. The system of claim 6 , further comprising one or more databases in communication with the visual recognition device and configured to retrievably store reference data, the reference data visually associating each of the classifiers to one of a plurality of known machine components and visually associating different spatial relationships to known classifications and poses.
9. The system of claim 8 , wherein the visual recognition device is configured to visually recognize each of the classifiers based on the reference data.
10. The system of claim 8 , wherein the visual recognition device is configured to visually recognize the classification and the pose of the work machine based on the reference data.
11. The system of claim 8 , wherein the database retrievably stores reference data visually associating the classifiers to known machine components including two or more of wheels, tracks, linkages, machine implements, operator cabs, and work machines.
12. The system of claim 6 , further comprising one or more tracking devices configured to acquire terrain measurements from the worksite, the visual recognition device being configured to track the pose of the work machine and the terrain measurements over time, and determine the analytics related to the work machine and the worksite based on the tracked pose and terrain measurements, the analytics including one or more of work productivity of the work machine, work progress, a state of the work machine, and a state of a terrain within the worksite.
13. A device for classifying a work machine within a worksite, the device comprising:
a communication device in communication with one or more image capture devices positioned around the worksite and remote from the work machine;
an output device;
a memory configured to retrievably store one or more algorithms; and
a controller in communication with each of the communication device, the output device and the memory, the controller being configured to
receive one or more captured images of the work machine from the image capture devices,
identify at least two classifiers within the captured images, wherein each classifier is indicative of a known component that is common to all work machines of the work machine type of the work machine,
determine spatial relationships between the classifiers within the captured images based on the relative positions of the classifiers as measured within the captured images,
determine a classification and a pose of the work machine based on the spatial relationships,
track the determined poses of the work machine over time and store the poses in the memory,
determine analytics related to the work machine and the worksite based on the tracked poses, the analytics including actual data derived from the poses of the work machine and representing a volume of work actually performed by the work machine within a given timeframe, and model data representing a desired volume of work performed by the work machine within the given timeframe, and
cause the output device to display the analytics based on the classifications and the poses of the work machine.
14. The device of claim 13 , wherein the communication device is in communication with one or more databases configured to retrievably store reference data, the reference data visually associating each of the classifiers to one of a plurality of known machine components and visually associating different spatial relationships to known classifications and poses.
15. The device of claim 14 , wherein the controller is configured to visually recognize each of the classifiers, the classification and the pose of the work machine based on the reference data.
16. The device of claim 14 , wherein the database retrievably stores reference data visually associating the classifiers to known machine components including two or more of wheels, tracks, linkages, machine implements, operator cabs, and work machines.
17. The device of claim 13 , wherein the communication device is in communication with one or more tracking devices configured to acquire terrain measurements from the worksite, the controller being configured to track the pose of the work machine and the terrain measurements over time, and determine the analytics related to the work machine and the worksite based on the tracked pose and terrain measurements, the analytics including one or more of work productivity of the work machine, work progress, a state of the work machine, and a state of a terrain within the worksite.