Zero shot localization for autonomous driving
A method for determining a location of a vehicle includes obtaining, by a processor, a plurality of aerial image segment signatures of segments of a region including the vehicle, and a plurality of sensed image signatures associated with the region including the vehicle. The method matches a selected aerial image segment signature segment signatures to a selected sensed image signature. Based on the matching step, the processor generates probabilistic location information regarding the location of the vehicle. The method also includes generating a movement estimate of the vehicles. The movement estimate is generated based on a vehicle location comparison across a plurality of vehicle sensed images, and the plurality of vehicle sensed images are captured at a plurality of time intervals. The method also includes determining the location of the vehicle by combining the movement estimate of the vehicle and the probabilistic location information.
1 . A method that is computer implemented and is for determining a location of a vehicle comprising:
obtaining, by a processor, a plurality of aerial image segment signatures of segments of a region including the vehicle;
obtaining, by the processor, a plurality of vehicle sensed image signatures associated with the region including the vehicle while the vehicle is in motion;
matching, by the processor, a selected aerial image segment signature of the plurality of aerial image segment signatures to a selected sensed image signature of the plurality of sensed image signatures;
based on the matching step, generating, by the processor, probabilistic location information regarding the location of the vehicle; the probabilistic location information provides, per each position of multiple positions within the region, a probability that the vehicle is located at the position;
generating, by the processor, a movement estimate of the vehicle, wherein the movement estimate is generated based on a vehicle location comparison across a plurality of vehicle sensed images, wherein the plurality of vehicle sensed images are captured at a plurality of time intervals; and
determining, by the processor, the location of the vehicle by combining the movement estimate of the vehicle and the probabilistic location information.
2 . The method according to claim 1 , wherein obtaining the plurality of aerial image segment signatures comprises generating the plurality of aerial image segment signatures by the processor.
3 . The method according to claim 1 , wherein obtaining the plurality of sensed image signatures comprises generating the plurality of sensed image signatures by the processor.
4 . The method according to claim 1 , wherein the vehicle sensed images are 360-degree vehicle sensed images.
5 . The method according to claim 1 , wherein the generating of the probabilistic location information comprises applying a random sample consensus process.
6 . The method according to claim 1 , wherein the determining of the location of the vehicle is also responsive to motion information gained from non-image sensors of the vehicle.
7 . The method according to claim 1 , wherein the determining of the location of the vehicle is based on a combination of the movement estimate of the vehicle, the probabilistic location information, and coarse vehicle location information.
8 . The method according to claim 1 , wherein the probabilistic location information is a heatmap.
9 . The method according to claim 8 , wherein the plurality of vehicle sensed images are associated with a plurality of heatmaps.
10 . The method according to claim 1 , wherein the obtaining of the plurality of the aerial image segment signatures and the obtaining of the plurality of sensed image signatures associated with the region including the vehicle comprises feeding to a machine learning process, the segments of the region including the vehicle and the sensed image signatures associated with the region including the vehicle; wherein the machine learning process being trained to (i) provide a similar signature to a ground vehicle sensed image of a region and an aerial image segment signature of that region, and (ii) provide dissimilar signatures to a ground vehicle sensed image and an aerial image segment of different regions.
11 . The method according to claim 1 , wherein the probabilistic location information provides the probability that the vehicle is located at the position per each of the plurality vehicle sensed images.
12 . A non-transitory computer readable medium for determining a location of a vehicle, the non-transitory computer readable medium stores instructions that once executed by a location system of the vehicle cause the location system to:
obtain a plurality of aerial image segment signatures of segments of a region including the vehicle;
obtain a plurality of vehicle sensed image signatures associated with the region including the vehicle, while the vehicle is in motion;
match a selected aerial image segment signature of the plurality of aerial image segment signatures to a selected sensed image signature of the plurality of sensed image signatures;
based on the matching step, generate probabilistic location information regarding the location of the vehicle; the probabilistic location information provides, per each position of multiple positions within the region, a probability that the vehicle is located at the position;
generate a movement estimate of the vehicles, wherein the movement estimate is generated based on a vehicle location comparison across a plurality of vehicle sensed images, wherein the plurality of vehicle sensed images are captured at a plurality of time intervals; and
determine the location of the vehicle by combining the movement estimate of the vehicle and the probabilistic location information.
13 . The non-transitory computer readable medium according to claim 12 , wherein obtaining the plurality of aerial image segment signatures comprises generating the plurality of aerial image segment signatures.
14 . The non-transitory computer readable medium according to claim 12 , wherein obtaining the plurality of sensed image signatures comprises generating the plurality of sensed image signatures by the processor.
15 . The non-transitory computer readable medium according to claim 12 , wherein each of the plurality of sensed images is a 360-degree vehicle sensed image.
16 . The non-transitory computer readable medium according to claim 12 , wherein the generating of the probabilistic location information comprises applying a random sample consensus process.
17 . The non-transitory computer readable medium according to claim 12 , wherein the determining the location of the vehicle further is responsive to motion information gained from non-image sensors of the vehicle.
18 . The non-transitory computer readable medium according to claim 12 , wherein the determining of the location of the vehicle is based only on a combination of the movement estimate of the vehicle, the probabilistic location information, and coarse vehicle location information.
19 . The non-transitory computer readable medium according to claim 12 , wherein the determining of the location of the vehicle is based only on a combination of the movement estimate of the vehicle and the probabilistic location information.
20 . The non-transitory computer readable medium according to claim 12 , wherein the probabilistic location information is a heatmap.
21 . A location system of a vehicle, the location system comprising:
one or more processing circuits that comprise at least a part of an integrated circuit, the one or more processing circuits are configured to:
obtain a plurality of aerial image segment signatures of segments of a region including the vehicle;
obtain a plurality of vehicle sensed image signatures associated with the region including the vehicle, while the vehicle is in motion;
match a selected aerial image segment signature of the plurality of aerial image segment signatures to a selected sensed image signature of the plurality of sensed image signatures;
based on the matching step, generate probabilistic location information regarding the location of the vehicle; the probabilistic location information provides, per each position of multiple positions within the region, a probability that the vehicle is located at the position;
generate a movement estimate of the vehicles, wherein the movement estimate is generated based on a vehicle location comparison across a plurality of vehicle sensed images, wherein the plurality of vehicle sensed images are captured at a plurality of time intervals; and
determine the location of the vehicle by combining the movement estimate of the vehicle and the probabilistic location information.
22 . The location system according to claim 21 , wherein obtaining the plurality of aerial image segment signatures comprises generating the plurality of aerial image segment signatures.
23 . The location system according to claim 21 , wherein obtaining the plurality of sensed image signatures comprises generating the plurality of sensed image signatures by the processor.