Counting crowds by augmenting convolutional neural network estimates with fifth generation signal processing data
An example method for estimating a number of individuals present in a crowd includes generating a density map based on an image of a crowd, using a convolutional neural network, augmenting the density map with cellular signal processing data to produce an augmented density map, and estimating a number of individuals present in the crowd, based on the augmented density map.
1. A method comprising:
generating, by a processing system including at least one processor, a density map based on an image of a crowd, using a convolutional neural network;
augmenting, by the processing system, the density map with cellular signal processing data to produce an augmented density map, wherein the augmenting comprises incorporating the cellular signal processing data as a transformation layer on the density map that is output by the convolutional neural network, wherein the transformation layer is separate from the convolutional neural network, wherein a parameter of the transformation layer is adjusted simultaneously with training of the convolutional neural network, and wherein the parameter comprises a threshold value for determining when an estimate of a number of individuals present in the crowd is to be increased in response to a detection of an individual in the cellular signal processing data; and
estimating, by the processing system, a number of individuals present in the crowd, based on the augmented density map.
2. The method of claim 1 , wherein the convolutional neural network comprises an image-based convolutional neural network that takes the image of the crowd as an input and produces the density map as an output.
3. The method of claim 1 , wherein the density map comprises a plurality of points, and wherein each point of the plurality of points represents an individual of the number of individuals who is depicted in the image of the crowd.
4. The method of claim 3 , wherein the augmented density map adds new points to the plurality of points of the density map by including one or more false negatives.
5. The method of claim 1 , wherein the cellular signal processing data is captured from user endpoint devices carried by some of the individuals of the number of individuals.
6. The method of claim 1 , wherein the transformation layer aligns the cellular signal processing data with the density map that is output by the convolutional neural network.
7. The method of claim 1 , wherein the augmenting relies on knowledge of locations of fixed-location transmitters which are located in a vicinity of the crowd.
8. The method of claim 1 , wherein the estimating is based on a plurality of augmented density maps including the augmented density map.
9. The method of claim 8 , wherein the plurality of augmented density maps depicts the crowd from a plurality of different perspectives.
10. The method of claim 1 , wherein the image of the crowd is obtained from a camera, and the cellular signal processing data is obtained from a fixed location transmitter that is separate from the camera.
11. The method of claim 1 , wherein the image of the crowd and the cellular signal processing data are obtained from a same device.
12. The method of claim 11 , wherein the same device is an unmanned aerial vehicle.
13. The method of claim 1 , wherein the cellular processing data is produced by a plurality of mobile user endpoint devices carried by some individuals of the number of individuals.
14. The method of claim 1 , wherein the cellular signal processing data comprises fifth generation signal processing data.
15. The method of claim 1 , wherein the estimate of the number of individuals present in the crowd is to be increased in response to the detection of the individual in the cellular signal processing data when the threshold value indicates that the individual was not previously accounted for in the density map that is output by the convolutional neural network, and wherein the estimate of the number of individuals present in the crowd is to remain at a current number in response to the detection of the individual in the cellular signal processing data when the threshold value indicates that the individual was previously accounted for in the density map that is output by the convolutional neural network.
16. A system comprising:
a processing system including at least one processor; and
a computer-readable medium storing instructions which, when executed by the processing system, cause the processing system to perform operations, the operations comprising:
generating a density map based on an image of a crowd, using a convolutional neural network;
augmenting the density map with cellular signal processing data to produce an augmented density map, wherein the augmenting comprises incorporating the cellular signal processing data as a transformation layer on the density map that is output by the convolutional neural network, wherein the transformation layer is separate from the convolutional neural network, wherein a parameter of the transformation layer is adjusted simultaneously with training of the convolutional neural network, and wherein the parameter comprises a threshold value for determining when an estimate of a number of individuals present in the crowd is to be increased in response to a detection of an individual in the cellular signal processing data; and
estimating a number of individuals present in the crowd, based on the augmented density map.
17. A non-transitory computer-readable medium storing instructions which, when executed by a processing system including at least one processor, cause the processing system to perform operations, the operations comprising:
generating a density map based on an image of a crowd, using a convolutional neural network;
augmenting the density map with cellular signal processing data to produce an augmented density map, wherein the augmenting comprises incorporating the cellular signal processing data as a transformation layer on the density map that is output by the convolutional neural network, wherein the transformation layer is separate from the convolutional neural network, wherein a parameter of the transformation layer is adjusted simultaneously with training of the convolutional neural network, and wherein the parameter comprises a threshold value for determining when an estimate of a number of individuals present in the crowd is to be increased in response to a detection of an individual in the cellular signal processing data; and
estimating a number of individuals present in the crowd, based on the augmented density map.
18. The non-transitory computer-readable medium of claim 17 , wherein the convolutional neural network comprises an image-based convolutional neural network that takes the image of the crowd as an input and produces the density map as an output.
19. The non-transitory computer-readable medium of claim 17 , wherein the density map comprises a plurality of points, wherein each point of the plurality of points represents an individual of the number of individuals who is depicted in the image of the crowd, and wherein the augmented density map adds new points to the plurality of points of the density map by including false negatives.
20. The non-transitory computer-readable medium of claim 17 , wherein the transformation layer aligns the cellular signal processing data with the density map that is output by the convolutional neural network.