System for autonomously or partially autonomously driving a vehicle with communication module for obtaining additional information from a vehicle driver and corresponding method
Control commands in a system and method for autonomously driving a vehicle or partially autonomously driving a vehicle are generated on the basis of an environment representation, which is generated from sensor signal of a sensing means. The environment representation where ambiguous objects are identified is such that information obtained from the driver of the vehicle are added. The system generates an information request. Additional information is extracted and accumulated in the environment representation map. Traffic is then determined and suitable control signals for the vehicles are generated. In case no ambiguous objects are included in the environment representation but the system is not capable of deciding on traffic, the driver is asked to disambiguate the situation or to instruct on dealing with the traffic.
1. A system for autonomously or partially autonomously driving a vehicle, the system comprising:
at least one sensor for physically sensing an environment of the vehicle;
a representation generation device for generating an initial representation of the environment, wherein the initial representation represents objects and other information on the environment of the vehicle by accumulating information on the objects in the environment derived from output of the sensor and the representation generation device is configured to classify the objects in the initial representation into either identifiable objects or ambiguous objects, for which no information or information with a confidence below a predetermined threshold is derivable from the sensor output and to determine which of the ambiguous objects have an influence on a traffic situation that currently needs to be evaluated;
a communication device for obtaining, from a vehicle's driver, additional information on at least one determined ambiguous object, wherein the representation generation device is configured to generate an enhanced environment representation by adding the additional information on the at least one object to the initial environment representation; and
an evaluation device for evaluating the current traffic situation based on the enhanced environment representation, and for generating control signals for controlling the vehicle like steering, throttle or brakes to achieve a control task based on the evaluated current traffic situation.
2. The system according to claim 1 , wherein
the evaluation means is configured to determine, whether or not the control signals to achieve the control task can be generated based on the determined current traffic situation; and
the communication device is configured to obtain, from the vehicle's driver, information how the control task shall be realized, if for the determined current traffic situation no control signals for controlling the vehicle that are suitable to achieve the control task can be generated.
3. The system according to claim 1 , wherein
the system comprises a driver's state estimation device.
4. The system according to claim 3 , wherein
the communication device is adapted to communicate with the driver on the basis of the driver's state.
5. A system for autonomously or partially autonomously driving a vehicle, the system comprising:
at least one sensor for physically sensing an environment of the vehicle;
a representation generation device for generating an initial representation of the environment, wherein the initial representation represents objects and other information on the environment of the vehicle, by accumulating information on the objects in the environment derived from output of the sensor, wherein the representation generation device is configured to classify the objects in the initial representation into either identifiable objects or ambiguous objects, for which no information or information with a confidence below a predetermined threshold is derivable from the sensor output and to determine which of the ambiguous objects have an influence on a traffic situation that currently needs to be evaluated;
a communication device for obtaining, from a driver of the vehicle, additional information on at least one determined ambiguous object, wherein the representation generation device is configured to generate an enhanced environment representation by adding the additional information on the at least one object to the initial environment representation, and
an evaluation device for evaluating the current traffic situation based on the enhanced environment representation, and for generating control signals for controlling the vehicle like steering, throttle or brakes to achieve a control task based on the evaluated current traffic situation, wherein
the communication device comprises a driver information request generator for generating and outputting a request to the driver to obtain the additional information,
the information request generator comprises at least one of the following: a driver intention estimation device configured to estimate intention of the driver with respect to an autonomous maneuver or a partially autonomous maneuver, a driver situation awareness estimation device configured to estimate current traffic situation awareness of the driver, and a driver system control model estimation device configured to estimate a model that the driver uses for understanding the ongoing autonomous vehicle control or partially autonomous vehicle control, and
the amount of information given in the request to the driver or a manner of presentation of such information is adapted based on an estimation result of at least one of the driver intention estimation device, the driver situation awareness estimation device, and the driver system control model estimation device.
6. The system according to claim 1 , wherein
the communication device is configured to request the driver to communicate with another traffic participant.
7. The system according to claim 1 , wherein
the communication device inputs and/or outputs speech for communication with the driver.
8. The system according to claim 1 , wherein
the communication device is a head up display used for communication with the driver.
9. A method for autonomously or partially autonomously driving a vehicle, comprising the steps of:
sensing, by at least one sensor, an environment of the vehicle;
generating, by a controlling device, an initial representation of the environment, wherein the initial representation represents objects and other information on the environment of the vehicle, by accumulating information on the objects in the environment derived from output of the sensor, and the objects in the initial representation are classified into either identifiable objects or ambiguous objects, for which no information or information with a confidence below a predetermined threshold is derivable from the sensor output and, from the ambiguous objects, objects which have an influence on a traffic situation that currently needs to be evaluated are determined;
obtaining, from a vehicle's driver, additional information on at least one determined ambiguous object;
generating, by the controlling device, an enhanced environment representation by adding the additional information on the at least one object to the initial environment representation;
evaluating, by the controlling device, the current traffic situation based on the enhanced environment representation; and
generating, by the controlling device, control signals for controlling the vehicle like steering, throttle or brakes to achieve a control task based on the evaluated current traffic situation.
10. A method for autonomously or partially autonomously driving a vehicle, the method comprising the steps of:
sensing, by at least one sensor, an environment of the vehicle;
generating, by a controlling device, an initial representation of the environment, which represents objects and other information on the environment of the vehicle, by accumulating information on objects in the environment derived from output of the sensor, wherein the objects in the initial representation are classified into either identifiable objects or ambiguous objects, for which no information or information with a confidence below a predetermined threshold is derivable from the sensor output and, from the ambiguous objects, objects which have an influence on a traffic situation that currently needs to be evaluated are determined;
generating, by the controlling device, a request to a vehicle's driver to obtain additional information on at least one determined ambiguous object by estimating at least one of the following: intention of the driver with respect to an autonomous maneuver or a partially autonomous maneuver, a current traffic situation awareness of the driver and a model that the driver uses for understanding the ongoing autonomous vehicle control or partially autonomous vehicle control, wherein the amount of information given in the request to the driver or a manner of presentation of such information is adapted based on a result of the estimation;
outputting the generate request to the driver;
obtaining, from the driver, the additional information on at least one determined ambiguous object;
generating, by the controlling device, an enhanced environment representation by adding the additional information on the at least one object to the initial environment representation;
evaluating, by the controlling device, the current traffic situation based on the enhanced environment representation; and
generating, by the controlling device, control signals for controlling the vehicle like steering, throttle or brakes to achieve a control task based on the evaluated current traffic situation.