Symbiotic warning methods and systems to warn nearby target actors while driving for collision avoidance

1 . A processor-implemented method, comprising the steps of:

receiving, via one or more hardware processors, one or more road contextual parameters to create a 360-degree scene perception of road surroundings of a host vehicle, wherein the 360-degree scene perception of road surroundings of the host vehicle comprises one or more actors present on a road as the host vehicle navigates on the road, wherein the 360-degree scene perception of the road surroundings of the host vehicle include one or more images and one or more sensor values;

estimating, via the one or more hardware processors, one or more 3-dimensional (3-D) scene semantics of each of the one or more actors present on the road as the host vehicle navigates, using the 360-degree scene perception of road surroundings, using a localizing technique;

detecting, via the one or more hardware processors, one or more priority actors those lead to probable collisions with the host vehicle, out of the one or more actors present on the road as the host vehicle navigates, based on the one or more 3-dimensional (3-D) scene semantics, using a path estimation and tracking technique;

deciding, via the one or more hardware processors, to generate a symbiotic warning signal, to one or more priority actors, based on the one or more 3-dimensional (3-D) scene semantics and the 360-degree scene perception of road surroundings, using a symbiotic warning trained model, wherein the symbiotic warning trained model is obtained by:

receiving a training dataset comprising a plurality of training samples, wherein the training dataset is associated with one or more driving environment scenarios and comprises one or more training sub-datasets, wherein each training sub-dataset is associated with each driving environment scenario and comprises one or more training samples of the plurality of training samples, and wherein each of the plurality of training samples comprises (i) a training 360-degree scene perception of road surroundings of a training host vehicle and (ii) one or more training 3-dimensional (3-D) scene semantics of the training host vehicle, associated with a plurality of training actors, wherein the one or more driving environment scenarios include (i) cut-in lane driving, (ii) driving through a hair pin bend road on a hill side, (iii) an abnormal/un-intentional lane change by the priority actors while the host vehicle in a blind spot, (iv) overtaking the host vehicle in a narrow lane, (v) a sudden close appearance of pedestrian/animal/other vehicle on road, (vi) merging lane and T-Junction situations, (vii) vehicle backing up, and (viii) target vehicle in an ego lane applied sudden brake, wherein the training dataset associated with (i) the cut-in lane driving includes the host vehicle speed and cut-in target vehicle speed (ii) the driving through the hair pin bend road on the hill side includes the host vehicle speed with no target vehicle present, the host vehicle speed and oncoming target vehicle speed, repeat the trained dataset with different road curvature (vi) the merging lane and T-Junction situations includes the host vehicle speed and the target vehicle is merging with the target vehicle speed, the host vehicle speed and the target vehicle is merging from T-junction with the target vehicle speed (viii) the target vehicle in the ego lane applied sudden brake includes the host vehicle speed and the target vehicle speed in the ego lane, the host vehicle speed and slow-moving target or stationary target in the ego lane;

processing each of the plurality of training samples present in each training sub-dataset to obtain a plurality of processed training samples from the plurality of training samples, wherein each processed training sample comprises a state and an action of each training priority actor and the training host vehicle, from (i) the training 360-degree scene perception of road surroundings and (ii) one or more training 3-dimensional (3-D) scene semantics;

assigning a training symbiotic warning signal of a plurality of symbiotic warning signals, for each processed training sample, based on the state and the action of each training priority actor and the training host vehicle; and

training a deep reinforcement learning neural network model, with each processed training sample at a time using an associated training symbiotic warning signal assigned, until the plurality of processed training samples is completed, to obtain the symbiotic warning trained model;

taking automatic control of host vehicle communication devices including a horn, a headlight flash or communicate with a dedicated short-range communication (DSRC) technology to generate the symbiotic warning signal, to one or more priority actors including nearby target vehicles, pedestrians and animals, based on the decision to generate and based on type of the priority actor and the driving environment scenario; and

controlling, via the one or more hardware processors, the host vehicle to avoid collisions with the one or more priority actors in response to the generated symbiotic warning signal, wherein the processor-implemented method is performed by a symbiotic warning system installed or equipped in the host vehicle.

2 . The processor-implemented method of claim 1 , wherein detecting one or more priority actors those lead to probable collisions, out of the one or more actors present on the road as the host vehicle navigates, based on the one or more 3-dimensional (3-D) scene semantics, using a path estimation and tracking technique, comprises:

detecting one or more actors of interest, from the one or more actors present on the road as the host vehicle navigates, using the path estimation and tracking technique;

determining one or more predicted moves of (i) each of the one or more actors of interest and (ii) the host vehicle, using the path estimation and tracking technique; and

mapping (i) the one or more predicted moves and (ii) a current move, of each of the one or more actors of interest and the host vehicle, to detect the one or more priority actors those may lead to probable collisions, wherein the current move is a present movement in real time,

wherein processing of output from Advanced driver-assistance systems (ADAS) system including Adaptive Cruise Control, Emergency braking system, Blind spot warning, Rear/Front cross traffic alert, Lateral collision warning is performed for reuse of existing algorithms and avoids repetition.

3 . The processor-implemented method of claim 1 , wherein deciding to generate the symbiotic warning signal of the one or more symbiotic warning signals, to one or more priority actors based on the one or more 3-dimensional (3-D) scene semantics and the 360-degree scene perception of road surroundings, using the symbiotic warning trained model, comprising:

determining a current state of the host vehicle and the one or more priority actors, using the 360-degree scene perception of road surroundings, wherein the state is associated with current dynamics of the each of the one or more priority actors on the road;

determining a current action for the host vehicle and the one or more priority actors, based on the current state of the host vehicle and the one or more priority actors, using the one or more 3-dimensional (3-D) scene semantics and the 360-degree scene perception of road surroundings, wherein the action describes one or more possible moves that each of the one or more priority actors and the host vehicle; and

passing (i) the current state of the host vehicle and the one or more priority actors, (ii) the current action for the host vehicle and the one or more priority actors, to the symbiotic warning trained model, to decide whether to generate the symbiotic warning signal, to one or more priority actors those lead to probable collisions.

4 . The processor-implemented method of claim 3 , wherein training the deep reinforcement learning neural network model, with each processed training sample at a time using the associated training symbiotic warning signal assigned, until the plurality of processed training samples is completed, to obtain the symbiotic warning trained model, comprises:

passing the state of a training host vehicle, of each processed training sample, to a reinforcement learning (RL) agent of the deep reinforcement learning neural network model;

obtaining (i) a predicted action of each training priority actor and the training host vehicle, for a given state of the training host vehicle, and (ii) a predicted symbiotic warning signal, from each processed training sample, from the RL agent;

comparing (i) the predicted action of each training priority actor and the training host vehicle with the from each processed training sample with the action of each training priority actor and the training host vehicle, and (ii) the predicted symbiotic warning signal with the corresponding training symbiotic warning signal assigned, for each processed training sample, to provide a reward for the RL agent based on the comparison; and

performing the training of the reinforcement learning (RL) agent with a successive processed training sample, until the plurality of processed training samples is completed, to maximize the reward for the RL agent.

5 . The processor-implemented method of claim 1 , wherein the one or more actors present on the road comprises one or more motorized and non-motorized vehicles, pedestrians, and animals present on the road surrounding the host vehicle.

6 . The processor-implemented method of claim 1 , wherein the symbiotic warning signal is generated through horn, headlights, a vehicle-vehicle to communication, and a combination thereof.

7 . The processor-implemented method of claim 1 , wherein the one or more road contextual parameters are associated with road modelling and the one or more road contextual parameters are received from one or more of: one or more 360-degree Lidars, one or more front corner radars, one or more rear corner radars, one or more cameras, one or more ultrasonic sensors, and one or more geographical road information devices, or a combination thereof, installed in the host vehicle.

8 . The processor implemented method of claim 1 , wherein the 360-degree scene perception of the road surroundings of the host vehicle provides a visual representation and allows the host vehicle to recognize and classify the one or more actors in the surrounding during the navigation of the host vehicle.

9 . The processor implemented method of claim 1 , wherein the one or more 3-D scene semantics includes a number of one or more motorized vehicles, one or more non-motorized vehicles, one or more pedestrians, a current distance between the host vehicle and each of the one or more motorized vehicles, the one or more non-motorized vehicles, the one or more pedestrians, an acceleration of each of the one or more motorized vehicles, the one or more non-motorized vehicles, dimensions and dynamics of each of the one or more motorized vehicles, the one or more non-motorized vehicles, a current moving direction of each of the one or more motorized vehicles, the one or more nonmotorized vehicles, the one or more pedestrians, wherein the localizing technique includes localization algorithms that to calculate a position and an orientation of the host vehicle with respect to the position of one or more actors in the road surroundings, wherein a visual odometry algorithm uses key points matching in consecutive video frames, wherein key points are the position and the orientation, wherein with each frame the key points are used as the an input to a mapping algorithm, wherein the mapping algorithm computes the position and orientation of each actor in a current frame with respect to previous frame and classifies all actors on the road.

10 . The processor implemented method of claim 1 , further comprising calculating a Time-to-collision (TTC), wherein the Time-To-collision is a distance between each actor and the host vehicle divided by their relative speed, wherein Time-to-collision is used for warning computation.

11 . A system comprising:

a memory storing instructions;

one or more input/output (I/O) interfaces; and

one or more hardware processors coupled to the memory via the one or more I/O interfaces, wherein the one or more hardware processors are configured by the instructions to:

receive one or more road contextual parameters to create a 360-degree scene perception of road surroundings of a host vehicle, wherein the 360-degree scene perception of road surroundings of the host vehicle comprises one or more actors present on a road as the host vehicle navigates on the road, wherein the 360-degree scene perception of the road surroundings of the host vehicle include one or more images and one or more sensor values;

estimate one or more 3-dimensional (3-D) scene semantics of each of the one or more actors present on the road as the host vehicle navigates, using the 360-degree scene perception of road surroundings, using a localizing technique;

detect one or more priority actors those lead to probable collisions with the host vehicle, out of the one or more actors present on the road as the host vehicle navigates, based on the one or more 3-dimensional (3-D) scene semantics, using a path estimation and tracking technique;

decide to generate a symbiotic warning signal, to one or more priority actors, based on the one or more 3-dimensional (3-D) scene semantics and the 360-degree scene perception of road surroundings, using a symbiotic warning trained model, wherein the symbiotic warning trained model is obtained by:

receiving a training dataset comprising a plurality of training samples, wherein the training dataset is associated with one or more driving environment scenarios and comprises one or more training sub-datasets, wherein each training sub-dataset is associated with each driving environment scenario and comprises one or more training samples of the plurality of training samples, and wherein each of the plurality of training samples comprises (i) a training 360-degree scene perception of road surroundings of a training host vehicle and (ii) one or more training 3-dimensional (3-D) scene semantics of the training host vehicle, associated with a plurality of training actors, wherein the one or more driving environment scenarios include (i) cut-in lane driving, (ii) driving through a hair pin bend road on a hill side, (iii) an abnormal/un-intentional lane change by the priority actors while the host vehicle in a blind spot, (iv) overtaking the host vehicle in a narrow lane, (v) a sudden close appearance of pedestrian/animal/other vehicle on road, (vi) merging lane and T-Junction situations, (vii) vehicle backing up, and (viii) target vehicle in an ego lane applied sudden brake, wherein the training dataset associated with (i) the cut-in lane driving includes the host vehicle speed and cut-in target vehicle speed (ii) the driving through the hair pin bend road on the hill side includes the host vehicle speed with no target vehicle present, the host vehicle speed and oncoming target vehicle speed, repeat the trained dataset with different road curvature (vi) the merging lane and T-Junction situations includes the host vehicle speed and the target vehicle is merging with the target vehicle speed, the host vehicle speed and the target vehicle is merging from T-junction with the target vehicle speed (viii) the target vehicle in the ego lane applied sudden brake includes the host vehicle speed and the target vehicle speed in the ego lane, the host vehicle speed and slow-moving target or stationary target in the ego lane;

processing each of the plurality of training samples present in each training sub-dataset to obtain a plurality of processed training samples from the plurality of training samples, wherein each processed training sample comprises a state and an action of each training priority actor and the training host vehicle, from (i) the training 360-degree scene perception of road surroundings and (ii) one or more training 3-dimensional (3-D) scene semantics;

assigning a training symbiotic warning signal of a plurality of symbiotic warning signals, for each processed training sample, based on the state and the action of each training priority actor and the training host vehicle; and

training a deep reinforcement learning neural network model, with each processed training sample at a time using an associated training symbiotic warning signal assigned, until the plurality of processed training samples is completed, to obtain the symbiotic warning trained model;

take automatic control of host vehicle communication devices including a horn, a headlight flash or communicate with a dedicated short-range communication (DSRC) technology to generate the symbiotic warning signal, to one or more priority actors including nearby target vehicles, pedestrians and animals, based on the decision to generate and based on type of the priority actor and the driving environment scenario; and

control the host vehicle to avoid collisions with the one or more priority actors in response to the generated symbiotic warning signal, wherein the processor-implemented method is performed by a symbiotic warning system installed or equipped in the host vehicle.

12 . The system as claimed in claim 11 , wherein the symbiotic warning and collision avoidance module is configured via the one or more hardware processors, to detect one or more priority actors those lead to probable collisions, out of the one or more actors present on the road as the host vehicle navigates, based on the one or more 3-dimensional (3-D) scene semantics, using a path estimation and tracking technique, by:

detecting one or more actors of interest, from the one or more actors present on the road as the host vehicle navigates, using the path estimation and tracking technique;

determining one or more predicted moves of (i) each of the one or more actors of interest and (ii) the host vehicle, using the path estimation and tracking technique; and

mapping (i) the one or more predicted moves and (ii) a current move, of each of the one or more actors of interest and the host vehicle, to detect the one or more priority actors those may lead to probable collisions, wherein the current move is a present movement in real time,

wherein processing of output from Advanced driver-assistance systems (ADAS) system including Adaptive Cruise Control, Emergency braking system, Blind spot warning, Rear/Front cross traffic alert, Lateral collision warning is performed for reuse of existing algorithms and avoids repetition.

13 . The system as claimed in claim 11 , wherein the symbiotic warning and collision avoidance module is configured via the one or more hardware processors, to decide to generate the symbiotic warning signal of the one or more symbiotic warning signals, to one or more priority actors based on the one or more 3-dimensional (3-D) scene semantics and the 360-degree scene perception of road surroundings, using the symbiotic warning trained model, by:

determining a current state of the host vehicle and the one or more priority actors, using the 360-degree scene perception of road surroundings, wherein the state is associated with current dynamics of the each of the one or more priority actors on the road;

determining a current action for the host vehicle and the one or more priority actors, based on the current state of the host vehicle and the one or more priority actors, using the one or more 3-dimensional (3-D) scene semantics and the 360-degree scene perception of road surroundings, wherein the action describes one or more possible moves that each of the one or more priority actors and the host vehicle; and

passing (i) the current state of the host vehicle and the one or more priority actors, (ii) the current action for the host vehicle and the one or more priority actors, to the symbiotic warning trained model, to decide whether to generate the symbiotic warning signal, to one or more priority actors those lead to probable collisions.

14 . The system as claimed in claim 11 , wherein the one or more hardware processors are configured to train the deep reinforcement learning neural network model, with each processed training sample at time using the associated training symbiotic warning signal assigned, until the plurality of processed training samples is completed, to obtain the symbiotic warning trained model, by:

passing the state of a training host vehicle, of each processed training sample, to a reinforcement learning (RL) agent of the deep reinforcement learning neural network model;

obtaining (i) a predicted action of each training priority actor and the training host vehicle, for a given state of the training host vehicle, and (ii) a predicted symbiotic warning signal, from each processed training sample, from the RL agent;

comparing (i) the predicted action of each training priority actor and the training host vehicle with the from each processed training sample with the action of each training priority actor and the training host vehicle, and (ii) the predicted symbiotic warning signal with the corresponding training symbiotic warning signal assigned, for each processed training sample, to provide a reward for the RL agent based on the comparison; and

performing the training of the reinforcement learning (RL) agent with a successive processed training sample, until the plurality of processed training samples is completed, to maximize the reward for the RL agent.

15 . The system as claimed in claim 11 , wherein the one or more actors present on the road comprises one or more motorized and non-motorized vehicles, pedestrians, and animals present on the road surrounding the host vehicle.

16 . The system as claimed in claim 11 , wherein the symbiotic warning signal is generated through horn, headlights, a vehicle-vehicle to communication, and a combination thereof.

17 . The system as claimed in claim 11 , wherein the one or more road contextual parameters are associated with road modelling and the one or more road contextual parameters are received from one or more of: one or more 360-degree Lidars, one or more front corner radars, one or more rear corner radars, one or more cameras, one or more ultrasonic sensors, and one or more geographical road information devices, or a combination thereof, installed in the host vehicle.

18 . One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:

receiving one or more road contextual parameters to create a 360-degree scene perception of road surroundings of a host vehicle, wherein the 360-degree scene perception of road surroundings of the host vehicle comprises one or more actors present on a road as the host vehicle navigates on the road, wherein the 360-degree scene perception of the road surroundings of the host vehicle include one or more images and one or more sensor values;

estimating one or more 3-dimensional scene semantics of each of the one or more actors present on the road as the host vehicle navigates, using the 360-degree scene perception of road surroundings, using a localizing technique;

detecting one or more priority actors those lead to probable collisions with the host vehicle, out of the one or more actors present on the road as the host vehicle navigates, based on the one or more 3-dimensional scene semantics, using a path estimation and tracking technique;

deciding to generate a symbiotic warning signal, to one or more priority actors, based on the one or more 3-dimensional scene semantics and the 360-degree scene perception of road surroundings, using a symbiotic warning trained model, wherein the symbiotic warning trained model is obtained by:

receiving a training dataset comprising a plurality of training samples, wherein the training dataset is associated with one or more driving environment scenarios and comprises one or more training sub-datasets, wherein each training sub-dataset is associated with each driving environment scenario and comprises one or more training samples of the plurality of training samples, and wherein each of the plurality of training samples comprises (i) a training 360-degree scene perception of road surroundings of a training host vehicle and (ii) one or more training 3-dimensional (3-D) scene semantics of the training host vehicle, associated with a plurality of training actors, wherein the one or more driving environment scenarios include (i) cut-in lane driving, (ii) driving through a hair pin bend road on a hill side, (iii) an abnormal/un-intentional lane change by the priority actors while the host vehicle in a blind spot, (iv) overtaking the host vehicle in a narrow lane, (v) a sudden close appearance of pedestrian/animal/other vehicle on road, (vi) merging lane and T-Junction situations, (vii) vehicle backing up, and (viii) target vehicle in an ego lane applied sudden brake, wherein the training dataset associated with (i) the cut-in lane driving includes the host vehicle speed and cut-in target vehicle speed (ii) the driving through the hair pin bend road on the hill side includes the host vehicle speed with no target vehicle present, the host vehicle speed and oncoming target vehicle speed, repeat the trained dataset with different road curvature (vi) the merging lane and T-Junction situations includes the host vehicle speed and the target vehicle is merging with the target vehicle speed, the host vehicle speed and the target vehicle is merging from T-junction with the target vehicle speed (viii) the target vehicle in the ego lane applied sudden brake includes the host vehicle speed and the target vehicle speed in the ego lane, the host vehicle speed and slow-moving target or stationary target in the ego lane;

processing each of the plurality of training samples present in each training sub-dataset to obtain a plurality of processed training samples from the plurality of training samples, wherein each processed training sample comprises a state and an action of each training priority actor and the training host vehicle, from (i) the training 360-degree scene perception of road surroundings and (ii) one or more training 3-dimensional (3-D) scene semantics;

assigning a training symbiotic warning signal of a plurality of symbiotic warning signals, for each processed training sample, based on the state and the action of each training priority actor and the training host vehicle; and

training a deep reinforcement learning neural network model, with each processed training sample at a time using an associated training symbiotic warning signal assigned, until the plurality of processed training samples is completed, to obtain the symbiotic warning trained model;

taking automatic control of host vehicle communication devices including a horn, a headlight flash or communicate with a dedicated short-range communication (DSRC) technology to generate the symbiotic warning signal, to one or more priority actors including nearby target vehicles, pedestrians and animals, based on the decision to generate and based on type of the priority actor and the driving environment scenario; and

controlling the host vehicle to avoid collisions with the one or more priority actors in response to the generated symbiotic warning signal, wherein the processor-implemented method is performed by a symbiotic warning system installed or equipped in the host vehicle.