Autonomous vehicle using heterogeneous redundancy checks
The invention is notably directed to an autonomous vehicle, e.g., an autonomous or semi-autonomous vehicle such as a self-driving car. The autonomous vehicle comprises a drive-by-wire (DbW) system ( 300 ), a set of perception sensors 21 - 24 , such as lidars and cameras, and two processing systems, i.e., a first processing system ( 100 ) and a second processing system ( 200 ). The first processing system is configured to form a main perception based on signals from each of the perception sensors of the set, estimate states of the vehicle based on feedback signals from the DbW system, and compute trajectories for the autonomous vehicle based on the perception formed and the estimated states. The second processing system is configured to form an auxiliary perception based on signals from only a subset of the perception sensors, validate the computed trajectories based on the auxiliary perception formed, and cause to forward the validated trajectories to the DbW system. In other words, distinct perceptions are formed from overlapping sets of sensors, whereby one of the perceptions formed is used to validate trajectories obtained from the other. This requires less computational efforts, inasmuch as less signals (and therefore less information) are required to form the auxiliary perception. However, doing so is more likely to allow inconsistencies to be detected, thanks to the heterogeneity of sensor signals considered in input to the main and auxiliary perceptions. The invention is further directed to related methods and computer program products.
1 . An autonomous vehicle comprising:
a drive-by-wire (DbW) system;
a set of perception sensors; and
two processing systems including:
a first processing system, which is configured to
form a main perception of surroundings of the autonomous vehicle based on signals from each of the perception sensors, whereby semantics are assigned to sensory data captured by the signals from each of the perception sensors, in operation,
estimate states of the vehicle based on feedback signals from the DbW system, and
compute trajectories for the autonomous vehicle based on the perception formed and the estimated states, and
a second processing system, which is configured to
form an auxiliary perception of surroundings of the autonomous vehicle based on signals from only a subset of the perception sensors, whereby semantics are assigned to sensory data captured by the signals from only the subset of the perception sensors, in operation,
validate the trajectories computed by the first processing system based on the auxiliary perception formed by performing redundancy checks, wherein said redundancy checks are heterogeneous redundancy checks, due to heterogeneity of signals used to form the main perception and the auxiliary perception, and
cause to forward the validated trajectories to the DbW system.
2 . The autonomous vehicle according to claim 1 , wherein the second processing system is further configured to
form said auxiliary perception as a global representation, which includes a world representation and embeds a representation of the autonomous vehicle,
validate, at each time point of a sequence of time points, the estimated states based on the auxiliary perception as formed at one or more previous one of the time points, whereby the computed trajectories are further validated based on the validated states, in operation, and
update, at said each time point, both the world representation, based on said signals from the subset of sensors, and the representation of the autonomous vehicle, based on states of the vehicle as previously validated at one or more previous ones of the time points.
3 . The autonomous vehicle according to claim 2 , wherein
the first processing system includes:
a main perception unit, which is connected to each of the sensors and is configured to form the main perception;
a state estimation unit, to which the DbW system is connected, and which is configured to estimate the states of the vehicle; and
a motion planning unit, which is configured to compute said trajectories, and
the second processing system includes:
an auxiliary perception unit, which is configured to form said auxiliary perception; and
a validation unit, which is configured to validate the computed trajectories and cause to forward the validated trajectories to the DbW system.
4 . The autonomous vehicle according to claim 3 , wherein
the validation unit is configured to validate the computed trajectories by verifying that the computed trajectories are collision-free, based on said world representation, under the condition that the estimated states are validated.
5 . The autonomous vehicle according to claim 3 , wherein
the autonomous vehicle further comprises sets of processors, each of the sets comprising one or more processors, and
the main perception unit, the state estimation unit, the motion planning unit, the auxiliary perception unit, and the validation unit, are mapped onto respective ones of the sets of processors.
6 . The autonomous vehicle according to claim 5 , wherein
the first processing system and the second processing system are implemented as distinct computers of the autonomous vehicle.
7 . The autonomous vehicle according to claim 3 , wherein the auxiliary perception unit is configured to run:
an occupancy grid map generator designed to generate occupancy grids for successive ones of said time points based on signals obtained from said subset of perception sensors, the occupancy grids capturing said global representation; and
a vehicle pose checker, which is designed to validate the estimated states of the vehicle by comparing a first pose of the vehicle corresponding to the estimated states with a second pose of the vehicle as captured in said occupancy grids by the representation of the autonomous vehicle.
8 . The autonomous vehicle according to claim 7 , wherein
the vehicle pose checker is designed to validate the estimated states of the vehicle by comparing first speeds of the vehicle as captured by the estimated states with second speeds of the vehicle as captured in said occupancy grids by at least two successive representations of the autonomous vehicle at two or more successive ones of the time points.
9 . The autonomous vehicle according to claim 7 , wherein
the occupancy grid map generator is designed to update, at said each time point, a current grid of the occupancy grids based on the first pose as validated by the vehicle pose checker at one or more previous ones of the time points, so as to update the representation of the autonomous vehicle in the current grid.
10 . The autonomous vehicle according to claim 9 , wherein
the occupancy grid map generator is designed to update the current grid of the occupancy grids based on the first pose as validated by the vehicle pose checker at one or more immediately preceding ones of the time points.
11 . The autonomous vehicle according to claim 9 , wherein
the validation unit is configured to validate the computed trajectories by verifying that such trajectories are collision-free according to said occupancy grids, provided that the poses of the vehicle, are validated by the vehicle pose checker.
12 . The autonomous vehicle according to claim 11 , wherein
the set of perception sensors include one or more lidars and one or more cameras, while said subset of perception sensors include the one or more lidars but does not include any of the one or more cameras.
13 . The autonomous vehicle according to claim 12 , wherein
the one or more lidars involve a plurality of lidars, and
the occupancy grid map generator is designed to obtain each occupancy grid of said occupancy grids by independently obtaining concurrent occupancy grids based on signals obtained from distinct ones of the lidars and then merging the concurrent occupancy grids obtained into said each occupancy grid.
14 . The autonomous vehicle according to claim 13 , wherein
the occupancy grid map generator is configured to obtain said concurrent occupancy grids in polar coordinates and then merge the concurrent occupancy grids obtained into said each occupancy grid, the latter defined in cartesian coordinates.
15 . The autonomous vehicle according to claim 13 , wherein
said each occupancy grid comprises cells that can have different cell states, the latter including an occupied state and a free state, and
the occupancy grid map generator is further designed to update cell states of cells of the occupancy grids based on time-redundant information obtained for the cells, whereby a change to any cell state is taken into account by the occupancy grid map generator only if information characterizing this change is observed twice in a row for two successive ones of said time points.
16 . The autonomous vehicle according to claim 15 , wherein
the cell states further include an unknown state, in addition to said occupied state and said free state, and
the occupancy grid map generator is configured to implement a reset mechanism to reset the state of any cell, for which no information can be obtained for a given time period or a given number of successive ones of the grids, to the unknown state.
17 . The autonomous vehicle according to claim 15 , wherein
the occupancy grids are updated at a frequency that is between 6 Hz and 18 Hz.
18 . A method of driving an autonomous vehicle, the autonomous vehicle comprising a drive-by-wire (DbW) system, a set of perception sensors, and two processing systems, the latter including a first processing system and a second processing system, wherein
the method comprises, at the first processing system,
forming a main perception of surroundings of the autonomous vehicle based on signals from each of the perception sensors, whereby semantics are assigned to sensory data captured by the signals from each of the perception sensors,
estimating states of the vehicle based on feedback signals from the DbW system, and
computing trajectories for the autonomous vehicle based on the formed perception and the estimated states, and
the method further comprises, at the second processing system,
forming an auxiliary perception of surroundings of the autonomous vehicle based on signals from only a subset of the perception sensors, whereby semantics are assigned to sensory data captured by the signals from only the subset of the perception sensors,
validating the trajectories computed by the first processing system based on the auxiliary perception formed by performing redundancy checks, wherein said redundancy checks are heterogeneous redundancy checks, due to heterogeneity of signals used to form the main perception and the auxiliary perception, and
causing to forward the validated trajectories to the DbW system.
19 . A non-transitory computer program product for driving an autonomous vehicle comprising a drive-by-wire (DbW) system, a set of perception sensors, and two processing systems, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by processing means of the two processing systems, to cause
a first processing system of the two processing systems to
form a main perception of surroundings of the autonomous vehicle based on signals from each of the perception sensors, whereby semantics are assigned to sensory data captured by the signals from each of the perception sensors, in operation,
estimate states of the vehicle based on feedback signals from the DbW system, and
compute trajectories for the autonomous vehicle based on the perception formed and the estimated states, and
a second processing system of the two processing systems to
form an auxiliary perception of surroundings of the autonomous vehicle based on signals from only a subset of the perception sensors, whereby semantics are assigned to sensory data captured by the signals from only the subset of the perception sensors, in operation,
validate the trajectories computed by the first processing system based on the auxiliary perception formed by performing redundancy checks, wherein said redundancy checks are heterogeneous redundancy checks, due to heterogeneity of signals used to form the main perception and the auxiliary perception, and
forward the validated trajectories to the DbW system.