Acoustic sensor processing
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for monitoring vehicles traversing a roadway using acoustic sensors. In some implementations, a server obtains data from an acoustic sensor monitoring road actors traversing a roadway at a first location. The server obtains data from an imaging sensor monitoring the road actors traversing the roadway at a second location. The server generates correlation data using the data from the acoustic sensor and the data from the imaging sensor. The server determines observations of the road actors traversing the roadway using the data from the acoustic sensor and the data from the imaging sensor. The server trains a machine-learning model to estimate characteristics of the road actors using the correlation data and the determined observations of the road actors from the imaging sensor and the acoustic sensor.
1. A computer-implemented method comprising:
obtaining, by one or more processors, data from an acoustic sensor monitoring road actors traversing a roadway at a first location;
obtaining, by the one or more processors, data from an imaging sensor monitoring the road actors traversing the roadway at a second location;
generating, by the one or more processors, correlation data using the data from the acoustic sensor and the data from the imaging sensor;
determining, by the one or more processors, observations of the road actors traversing the roadway using the data from the acoustic sensor and the data from the imaging sensor; and
training, by the one or more processors, a machine-learning model to estimate characteristics of the road actors using the correlation data and the determined observations of the road actors from the imaging sensor and the acoustic sensor,
wherein generating the correlation data using the data from the acoustic sensor and the data from the imager sensor further comprises:
generating, by the one or more processors, joint correlation data for modeling an environment of the road actors traversing the roadway using (i) the data from the acoustic sensor at the first location, (ii) the data from the imaging sensor at the second location, and (iii) the observations of the road actors traversing the roadway, the joint correlation data indicating (i) first characteristics of the road actors at the first location not in a field of view of the imaging sensor, (ii) second characteristics of the road actors at the second location not in a field of view of the acoustic sensor, and (iii) third characteristics of the road actors at a third location in a field of view of both the acoustic sensor and the imaging sensor.
2. The computer-implemented method of claim 1 , wherein the first location is different from the second location.
3. The computer-implemented method of claim 1 , wherein the first location is similar to the second location.
4. The computer-implemented method of claim 1 , wherein obtaining data from the acoustic sensor monitoring the road actors further comprises:
receiving, by the one or more processors, first acoustic data from a first acoustic sensor at a first time; and
receiving, by the one or more processors, second acoustic data from a second acoustic sensor at a second time.
5. The computer-implemented method of claim 4 , wherein a difference between the first time and the second time represents (i) a distance between the first acoustic sensor and the second acoustic sensor on the roadway and (ii) a speed at which the road actor moves on the roadway between the first acoustic sensor and the second acoustic sensor.
6. The computer-implemented method of claim 4 , wherein the first acoustic data represents a sound profile of the road actor at the first time and the second acoustic data represents the sound profile of the road actor at the second time.
7. The computer-implemented method of claim 1 , wherein the imaging sensor comprises at least one of a LIDAR system, a RADAR system, and a camera.
8. The computer-implemented method of claim 1 , wherein determining the observations of the road actors using the data from the imaging sensor and the acoustic sensor further comprises:
determining, by the one or more processors, a sound profile for each of the road actors traversing the roadway;
determining, by the one or more processors, a location for each of the road actors in the data from the imaging sensor;
determining, by the one or more processors, a color for each of the road actors in the data from the imaging sensor; and
determining, by the one or more processors, a size for each of the road actors in the data from the imaging sensor.
9. The computer-implemented method of claim 1 , wherein training the machine-learning model to estimate the characteristics of the road actors using the correlation data and the determined characteristics of the road actors from the imaging sensor and the acoustic sensor further comprises:
training, by the one or more processors, the machine-learning model to estimate characteristics of the road actors in a location where the imaging sensor cannot view the roadway.
10. The computer-implemented method of claim 1 , further comprising:
estimating, by the one or more processors, the characteristics of the road actors on the roadway by providing data from the acoustic sensor to the trained machine-learning model.
11. The computer-implemented method of claim 1 , wherein the characteristics of the road actors comprise at least one of a number of axles in a road actor, a speed of the road actor, an acceleration of the road actor, a congestion of the roadway, and a number of road actors at the first location and the second location.
12. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
obtaining, by one or more processors, data from an acoustic sensor monitoring road actors traversing a roadway at a first location;
obtaining, by the one or more processors, data from an imaging sensor monitoring the road actors traversing the roadway at a second location;
generating, by the one or more processors, correlation data using the data from the acoustic sensor and the data from the imaging sensor;
determining, by the one or more processors, observations of the road actors traversing the roadway using the data from the acoustic sensor and the data from the imaging sensor; and training, by the one or more processors, a machine-learning model to estimate characteristics of the road actors using the correlation data and the determined observations of the road actors from the imaging sensor and the acoustic sensor,
wherein generating the correlation data using the data from the acoustic sensor and the data from the imager sensor further comprises:
generating, by the one or more processors, joint correlation data for modeling an environment of the road actors traversing the roadway using (i) the data from the acoustic sensor at the first location, (ii) the data from the imaging sensor at the second location, and (iii) the observations of the road actors traversing the roadway, the joint correlation data indicating (i) first characteristics of the road actors at the first location not in a field of view of the imaging sensor, (ii) second characteristics of the road actors at the second location not in a field of view of the acoustic sensor, and (iii) third characteristics of the road actors at a third location in a field of view of both the acoustic sensor and the imaging sensor.
13. The system of claim 12 , wherein the first location is different from the second location.
14. The system of claim 12 , wherein the first location is similar to the second location.
15. The system of claim 12 , wherein obtaining data from the acoustic sensor monitoring the road actors further comprises:
receiving, by the one or more processors, first acoustic data from a first acoustic sensor at a first time; and
receiving, by the one or more processors, second acoustic data from a second acoustic sensor at a second time.
16. The system of claim 15 , wherein a difference between the first time and the second time represents (i) a distance between the first acoustic sensor and the second acoustic sensor on the roadway and (ii) a speed at which the road actor moves on the roadway between the first acoustic sensor and the second acoustic sensor.
17. The system of claim 15 , wherein the first acoustic data represents a sound profile of the road actor at the first time and the second acoustic data represents the sound profile of the road actor at the second time.
18. The system of claim 12 , wherein the imaging sensor comprises at least one of a LIDAR system, a RADAR system, and a camera.
19. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
obtaining, by one or more processors, data from an acoustic sensor monitoring road actors traversing a roadway at a first location;
obtaining, by the one or more processors, data from an imaging sensor monitoring the road actors traversing the roadway at a second location;
generating, by the one or more processors, correlation data using the data from the acoustic sensor and the data from the imaging sensor;
determining, by the one or more processors, observations of the road actors traversing the roadway using the data from the acoustic sensor and the data from the imaging sensor; and training, by the one or more processors, a machine-learning model to estimate characteristics of the road actors using the correlation data and the determined observations of the road actors from the imaging sensor and the acoustic sensor,
wherein generating the correlation data using the data from the acoustic sensor and the data from the imager sensor further comprises:
generating, by the one or more processors, joint correlation data for modeling an environment of the road actors traversing the roadway using (i) the data from the acoustic sensor at the first location, (ii) the data from the imaging sensor at the second location, and (iii) the observations of the road actors traversing the roadway, the joint correlation data indicating (i) first characteristics of the road actors at the first location not in a field of view of the imaging sensor, (ii) second characteristics of the road actors at the second location not in a field of view of the acoustic sensor, and (iii) third characteristics of the road actors at a third location in a field of view of both the acoustic sensor and the imaging sensor.