Using human intelligence tasks for precise image analysis
Described are systems, methods, computer programs, and user interfaces for image location, acquisition, analysis, and data correlation that uses human-in-the-loop processing, Human Intelligence Tasks (HIT), and/or or automated image processing. Results obtained using image analysis are correlated to non-spatial information useful for commerce and trade. For example, images of regions of interest of the earth are used to count items (e.g., cars in a store parking lot to predict store revenues), detect events (e.g., unloading of a container ship, or evaluating the completion of a construction project), or quantify items (e.g., the water level in a reservoir, the area of a farming plot).
1. A network-based system for obtaining information from imagery, the system comprising:
an image analysis subsystem configured to:
present imagery, via a network communication channel, to a plurality of worker interfaces configured to be executed on a respective plurality of network devices, wherein each of the plurality of worker interfaces comprises a user interface configured to:
display the imagery;
allow a user to click using a pointer device moving an indicator on a display;
receive a user click at a click location;
place a mark on the imagery at the click location;
remove the mark responsive to a second click at the click location; and
submit the mark as the user response;
receive, via the network communication channel, a plurality of user responses regarding the imagery; and
automatically determine a count of one or more object of interests present in the imagery based on the user responses; and
a non-spatial correlation subsystem configured to:
automatically correlate the count of the objects with a non-spatial data; and
automatically predict a future non-spatial data based on a future count and the correlation between the non-spatial data and the count.
2. The system of claim 1 , wherein the user interface is configured to allow a user to place a plurality of marks and requires a minimum distance between marks.
3. The system of claim 1 , wherein the user interface is configured to measure a time between clicks.
4. The system of claim 1 wherein the user responses are aggregated into at least one two-dimensional click cloud indicating potential locations of objects of interest located in the imagery.
5. The system of claim 4 wherein the image analysis subsystem is configured to
fit each two dimensional click cloud to a two-dimensional probability distribution function to create at least one peak comprising a center; and
determine a count of the objects of interest based on a count of the peaks.
6. The system of claim 5 wherein the probability density function is a Gaussian type exponential function, and wherein a width of the function is proportional to an expected size of the object of interest.
7. The system of claim 5 wherein the at least one peak comprises a minimum percentage of clicks within a maximum distance from the center.
8. The system of claim 7 wherein the maximum distance is at least one of: half of an expected width of the object of interest, half of a median width of the object of interest, and half of an average width of an object of interest.
9. The system of claim 5 wherein image analysis subsystem is configured to use the click cloud, the at least one peak, and the count as training classifiers for a computer learning program.
10. The system of claim 5 wherein the image analysis subsystem is further configured to determine a confidence score for the count.
11. The system of claim 10 wherein the confidence score is determined based on a measurement of a time between clicks from the user responses that make up the two-dimensional click cloud.
12. The system of claim 10 wherein each user response comprises a user count, and the confidence score is determined based on a variance between the user counts across all user responses.
13. The system of claim 10 wherein the confidence score is determined based on an amount of time taken to create the user responses.
14. The system of claim 10 wherein the image analysis subsystem is further configured to alter the imagery if the confidence score is below a threshold.
15. The system of claim 5 wherein the image analysis subsystem is further configured to use a density of clicks in the at least one peak to alter the imagery presented to the worker interfaces.
16. The system of claim 5 wherein the image analysis subsystem is further configured to correlate a time between clicks to a density of clicks in the at least one peak.
17. The system of claim 5 wherein image analysis subsystem is configured to use the user responses, and either the count or the two-dimensional click cloud to determine an overcount or an undercount of a user providing one of the user responses.
18. The system of claim 1 wherein the non-spatial correlation subsystem is further configured to predict a future count based on a future non-spatial data and the correlation between the non-spatial data and the count.
19. A network-based system for estimating non-spatial data from overhead imagery, the system comprising:
a location search subsystem configured to convert a location query received via a network communication channel into one or more derived coordinates;
an image acquisition subsystem configured to obtain imagery of a polygon of interest based on the derived coordinates;
an image analysis subsystem configured to:
process the obtained imagery to distinguish one or more objects of interest from a background of the imagery, wherein to distinguish the one or more objects the image analysis subsystem is configured to automatically determine at least one two-dimensional click cloud indicating potential locations of objects of interest located in the imagery;
receive results determined from the objects of interest; and
a non-spatial correlation subsystem configured to automatically determine a correlation between a non-spatial data and the results.
20. A network-based system for obtaining information from imagery, the system comprising:
an image analysis subsystem configured to:
present imagery, via a network communication channel, to a plurality of worker interfaces configured to be executed on a respective plurality of network devices;
receive, via the network communication channel, a plurality of user responses regarding the imagery;
automatically determine a count of one or more object of interests present in the imagery based on the user responses; and
automatically aggregate the plurality of user responses into at least one two-dimensional click cloud indicating potential locations of objects of interest located in the imagery; and
a non-spatial correlation subsystem configured to:
automatically correlate the count of the objects with a non-spatial data; and
automatically predict a future non-spatial data based on a future count and the correlation between the non-spatial data and the count.
21. The system of claim 20 , wherein the image analysis subsystem is configured to
fit each two dimensional click cloud to a two-dimensional probability distribution function to create at least one peak comprising a center; and
determine a count of the objects of interest based on a count of the peaks.
22. The system of claim 21 , wherein the probability density function is a Gaussian type exponential function, and wherein a width of the function is proportional to an expected size of the object of interest.
23. The system of claim 21 , wherein the at least one peak comprises a minimum percentage of clicks within a maximum distance from the center.
24. The system of claim 23 , wherein the maximum distance is at least one of: half of an expected width of the object of interest, half of a median width of the object of interest, and half of an average width of an object of interest.
25. The system of claim 21 , wherein image analysis subsystem is configured to use the click cloud, peak, and count as training classifiers for a computer learning program.
26. The system of claim 21 , wherein the image analysis subsystem is further configured to use a density of clicks in the at least one peak to alter the imagery presented to the worker interfaces.
27. The system of claim 21 , wherein the image analysis subsystem is further configured to correlate a time between clicks to a density of clicks in the at least one peak.
28. The system of claim 20 , wherein the image analysis subsystem is further configured to determine a confidence score for the count.
29. The system of claim 28 , wherein the confidence score is determined based on a measurement of a time between clicks from the user responses that make up the two-dimensional click cloud.
30. The system of claim 28 , wherein each user response comprises a user count, and the confidence score is determined based on a variance between the user counts across all user responses.
31. The system of claim 28 , wherein the confidence score is determined based on an amount of time taken to create the user responses.
32. The system of claim 28 , wherein the image analysis subsystem is further configured to alter the imagery if the confidence score is below a threshold.
33. The system of claim 20 , wherein image analysis subsystem is configured to use the user responses, and either the count or the two-dimensional click cloud to determine an overcount or an undercount of a user providing one of the user responses.
34. The system of claim 20 , wherein the non-spatial correlation subsystem is further configured to predict a future count based on a future non-spatial data and the correlation between the non-spatial data and the count.