Searchable integration of geospatial based information with captured images during worksite operations
A computer-implemented system and method provide for searchable integration of operating images and geospatial data, particularly for agricultural field operations or construction worksites in which visual images may be captured by work vehicles during normal operation rather than requiring specific trips by employees or agronomists. Captured images are uploaded from sensors associated with a work vehicle traversing a work area, along with associated geospatial data, to a remote data storage device via a communications network. The uploaded images and geospatial data are selectively correlated with respect to a geospatial map. Responsive to user input corresponding to a selected one or more nodes in the geospatial map, at least one of the uploaded images is identified as corresponding to the selected one or more nodes. A display is generated representing the identified at least one of the uploaded images on a display unit associated with the received user input.
1 . A computer-implemented method for searchable integration of operating images and geospatial data, the method comprising:
uploading captured images from one or more perception sensors associated with a work vehicle traversing a work area and in the course of operations on a terrain thereof to a remote data storage device via a communications network;
determining geospatial data associated with each captured image, wherein the geospatial data for each captured image comprises first information representing a location of the work machine, second information representing a time when the image was captured, and third information representing one or more material and/or terrain characteristics of the work area at the time when the image was captured;
developing one or more computer-based models over time for selectively correlating a first data set comprising the uploaded images and a second data set comprising the geospatial data with respect to a geospatial map, for each of a plurality of user-selectable priorities for optimizing image selection, and further establishing a searchable data structure that links the captured images and associated geospatial data to corresponding geographic coordinates;
generating a first display on a display unit representing an overhead view of at least a portion of the work area;
responsive to user input via the first display corresponding to a selection of one or more nodes in the geospatial map, identifying at least one location within the work area corresponding to the selected one or more nodes in the geospatial map, and responsive to further user input specifying one or more priorities for optimized image selection from among the first, second, and third information of the geospatial data, identifying at least one of the uploaded images by reference to the one or more computer-based models and based on the specified one or more priorities; and
generating a second display representing the identified at least one of the uploaded images on the display unit associated with the received user input.
2 . The method of claim 1 , comprising automatically controlling one or more of the operations of the work vehicle based at least in part on captured images from at least one of the one or more sensors.
3 . The method of claim 2 , wherein the work vehicle comprises a first work vehicle, the method further comprising automatically controlling one or more operations of a second work vehicle traversing the work area based at least in part on the captured images from the at least one of the one or more sensors associated with the first work vehicle.
4 . The method of claim 1 , wherein the plurality of user-selectable priorities for optimizing image selection comprise a requirement of a most recent uploaded image corresponding to the identified at least one location.
5 . The method of claim 1 , wherein the plurality of user-selectable priorities for optimizing image selection comprise a requirement of a most similar time with respect to seasonal and/or time of day attributes.
6 . The method of claim 1 , wherein the generated first display comprises indicia representing respective material and/or terrain characteristics of one or more portions of the work area.
7 . The method of claim 6 , further comprising:
identifying the at least one of the uploaded images as being nearest in location to the selected one or more nodes and further sharing at least one of the material and/or terrain characteristics of portions of the work area corresponding to the selected one or more nodes; and
generating the second display on the display unit corresponding to the identified at least one of the uploaded images.
8 . A computer-implemented system for searchable integration of operating images and geospatial data, the system comprising:
one or more servers functionally linked via a communications network to one or more work vehicles, and configured to:
upload captured images from one or more perception sensors associated with respective work vehicles while traversing a work area and in the course of operations on a terrain thereof to a remote data storage device;
determining geospatial data associated with each captured image, wherein the geospatial data for each captured image comprises first information representing a location of the work machine, second information representing a time when the image was captured, and third information representing one or more material and/or terrain characteristics of the work area at the time when the image was captured;
developing one or more computer-based models over time to selectively correlate a first data set comprising the uploaded images and a second data set comprising the geospatial data with respect to a geospatial map, for each of a plurality of user-selectable priorities for optimizing image selection, and further by establishing a searchable data structure that links the captured images and associated geospatial data to corresponding geographic coordinates;
generate a first display on a display unit representing an overhead view of at least a portion of the work area;
responsive to user input via the first display corresponding to a selection of one or more nodes in the geospatial map, identify at least one location within the work area corresponding to the selected one or more nodes in the geospatial map, and responsive to further user input specifying one or more priorities for optimized image selection from among the first, second, and third information of the geospatial data, identify at least one of the uploaded images by reference to the one or more computer-based models and based on the specified one or more priorities; and
generate a second display representing the identified at least one of the uploaded images on the display unit associated with the received user input.
9 . The system of claim 8 , further comprising respective controllers associated with the work vehicles and configured to automatically control one or more of the operations thereof while traversing the work area based at least in part on captured images from at least one of the associated one or more sensors.
10 . The system of claim 9 , wherein the one or more servers are configured to provide the captured images or information derived from the captured images from the one or more sensors associated with a first work vehicle to the controller of a second work vehicle, and wherein the controller of the second work vehicle is configured to control one or more operations thereof while traversing the work area based at least in part on the captured images or information derived from the captured images from the one or more sensors associated with the first work vehicle.
11 . The system of claim 8 , wherein the plurality of user-selectable priorities for optimizing image selection comprise a requirement of a most recent uploaded image corresponding to the identified at least one location.
12 . The system of claim 8 , wherein the plurality of user-selectable priorities for optimizing image selection comprise a requirement of a most similar time with respect to seasonal and/or time of day attributes.
13 . The system of claim 8 , wherein the generated first display comprises indicia representing respective material and/or terrain characteristics of one or more portions of the work area.
14 . The system of claim 13 , wherein the one or more servers are configured to:
identify the at least one of the uploaded images as being nearest in location to the selected one or more nodes and further sharing at least one of the material and/or terrain characteristics of portions of the work area corresponding to the selected one or more nodes; and
generate the second display on the display unit corresponding to the identified at least one of the uploaded images.
15 . The method of claim 1 , wherein captured images are selectively uploaded to the remote data storage device based on predetermined criteria for image transmission.
16 . The method of claim 15 , wherein the predetermined criteria comprise determining that a respective image satisfies specified image resolution and/or content requirements.
17 . The method of claim 2 , comprising controlling a respective orientation of the one or more perception sensors between first and second fields of view, wherein images collected with the first field of view are used for the automatically controlling one or more of the operations of the work vehicle, and wherein images collected with the second field of view are uploaded to the remote device.
18 . The system of claim 8 , wherein captured images are selectively uploaded to the remote data storage device based on predetermined criteria for image transmission.
19 . The system of claim 18 , wherein the predetermined criteria comprise determining that a respective image satisfies specified image resolution and/or content requirements.
20 . The system of claim 9 , wherein a respective orientation of the one or more perception sensors is controlled between first and second fields of view, wherein images collected with the first field of view are used for the automatically controlling one or more of the operations of the work vehicle, and wherein images collected with the second field of view are uploaded to the remote device.