Systems, methods and devices for map-based object's localization deep learning and object's motion trajectories on geospatial maps using neural network
An object of initial unknown position on a map may be determined by traversing through moving and turning to establish motion trajectory to reduce its spatial uncertainty to a single location that would fit only to a certain map trajectory. An artificial neural network model learns from object motion on different map topologies may establish the object's end-to-end positioning from embedding map topologies and object motion. The proposed method includes learning potential motion patterns from the map and perform trajectory classification in the map's edge-space. Two different trajectory representations, namely angle representation and augmented angle representation (incorporates distance traversed) are considered and both a Graph Neural Network and an RNN are trained from the map for each representation to compare their performances. The results from the actual visual-inertial odometry have shown that the proposed approach is able to learn the map and localize the object based on its motion trajectories.
1. A method for generating object absolute positioning, comprising:
in response to receiving motion based relative position signals {s 1 , s 2 . . . s n } generated by the at least one device of an object that traverses within a map M, wherein the map M is represented as a graph G, comprising a plurality of nodes V={v 1 , v 2 . . . v n } and a plurality of edges E={e 1 , e 2 . . . e n }, wherein: each of the nodes {v 1 , v 2 . . . v n } is assigned with a unique identification which represents a place and other features attributed to that place at a certain time sequence, and each of the edges {e 1 , e 2 . . . e n } is assigned with a unique identification which represents a traversable path between the nodes,
executing by a processor, an algorithm stored in a memory of at least one device of the object to perform steps, comprising:
extracting, by the processor of the at least one device of the object, the motion based relative position signals {s 1 , s 2 . . . s n } generated from the at least one device of the object to obtain time sequenced edges { . . . e i−1 , e i , e i+1 , e i+2 . . . } and nodes { . . . v i−1 , v i , v i+1 , v i+2 . . . } information traversed by the object at time t at node v i , such that the object at the at node v i either makes a turn or continues to move in one direction, wherein the node v i is associated with a distance l i traversed and a turning angle (which represents directional information formed at the node v i between a previous edge e i−1 and a next edge e i+1 of the object has traversed;
generating, by the processor, a relative motion trajectory T r of the object over n nodes {v 1 , v 2 . . . v n } from one or both of n−1 distances {l 1 , l 2 . . . l n−1 } and n−2 turning angles {φ1 . . . φn−2} computed from the motion based relative position signals {s 1 , s 2 . . . s n };
quantizing the n−2 turning angles {φ1 . . . φn−2} to identify a given discrete bin for each relative motion trajectory T r at the time t from a plurality of discrete bins;
training a neural network model that maps the graph G to an embedding space Z based on object traversable edges {e 1 , e 2 . . . e n } and nodes {v 1 , v 2 . . . v n } to learn object's motion trajectories j on the map M; and
generating a geolocation A x of the object according to the trained neural network within the map M and the embedding space Z.
2. The method according to claim 1 , wherein the trained neural network comprising a trained Graph Neural Network (GNN) model utilizing a message passing mechanism that fires neurons by processing the time sequenced edges { . . . e i−1 , e i , e i+1 , e i+2 . . . } and nodes { . . . v i−1 , v i , v i+1 , v i+2 . . . } at the time t i to learn or estimate the geolocation A x of the object on the map M.
3. The method according to claim 2 , wherein the trained Graph Neural Network that learns or estimates the geolocation A x of the object on the map M is defined by a non-linear function ƒ:
ƒ: X→Y based on the message passing mechanism,
wherein X is an input feature sequence: X=(φ1, φ2, . . . φn) or X={l 1 , l 2 . . . l n−1 }, where φl∈R, (i=1, 2 . . . n), n is a length of sequence, and φl is ith angle, wherein l 1 , l 2 . . . l n−1 are distances of the sequence;
wherein Y is an output position label: Y=e i , e i ∈E, where E={e 1 , e 2 . . . e n } is the output label space, k is the number of edges in the at least one defined map.
4. The method according to claim 2 , comprising using a softmax function:
P
(
Y
=
i
l
z
)
=
softmax
(
z
)
=
e
z
∑
j
=
0
k
e
z
to calculate a probability of each edge with a final output which corresponds to unique edge ID in a given map graph, where, z is the final linear output, Y=i represents the edge ID and is equal to i.
5. The method according to claim 1 , wherein the trained neural network comprising a trained Recurrent Neural Network (RNN) model comprising layers of neurons having feedback loops for processing one-time sequenced edge e i and node v i at the time t to learn or estimate the location A x of the object on the map M.
6. The method according to claim 5 , wherein the trained Recurrent Neural Network model processes the input sequence x t at an instance of time t, and an output sequence generated at the time instant t depends on all previous hidden states h t−1 inputs, the non-linear function ƒ, and produces a new state through a non-linear activation function ƒ and g:
h t =ƒ( W·x t +U·h t−1 +b h );
y t =g ( V·h t +b y )
where, U; V; W are weighted matrices, h t and y t are hidden output and final output, and b h and b y are bias terms.
7. The method according to claim 5 , wherein the trained RNN further comprising a Long Short-Term Memory (LSTM) network or a transformer network which uses internal cyclic mechanisms and attention mechanisms, wherein each angle φ1 in the sequence generated by a preprocessing phase is fed into the LSTM network one by one.
8. The method according to claim 1 , wherein the at least one device of the object that generates the motion based relative position signals {s 1 , s 2 . . . s n } comprises one or a combination of: a fixed image camera, a video camera, a mobile smart phone, an inertial measurement unit (IMU), a Light Detection and Ranging (LIDAR) structured light sensors (depth sensors) or a sound navigating ranges (SONAR) (under water depth sensors) that measure two dimensional (2-D) or three dimensional (3D) accelerations and gyroscope, a visual inertial odometer.
9. The method according to claim 1 , further comprising using one or both of distances {l 1 , l 2 . . . l n−1 } between nodes or augmented angle representation in the turn angle φi to improve accuracy, wherein the motion based relative position signals {s 1 , s 2 . . . s n } are generated in two dimensional (2-D) and three dimensional (3-D) space . . . .
10. The method according to claim 1 , wherein the map M is generated from inputs from one or a combination of: a blue print, geographical information system, a fixed image picture, motion images, an OpenStreetMap (OSM), Simultaneous Localization and Mapping (SLAM), Global Positioning System (GPS), Indoor Positioning System (IPS), under water topographical map and visual Odometry (VO).