Synchronised sensor fusion pipeline

1 . A method for monitoring traffic, the method comprising:

obtaining a first dataset from a radar sensor, wherein:

the first dataset includes a first set of spatial coordinates and a velocity of one or more traffic objects;

obtaining a second dataset from an image sensor, wherein:

the second dataset includes a second set of spatial coordinates of the one or more traffic objects, and wherein the first dataset and the second dataset each is dynamically adjusted based on environmental factors;

synchronizing the radar sensor with the image sensor to coordinate a radar data stream with an image data stream;

applying a first classifier on the first dataset to generate first classifications;

applying a second classifier on the second dataset to generate second classifications; and

integrating the first classifications and the second classifications to generate a composite classification.

2 . The method for monitoring traffic of claim 1 , wherein:

the first dataset is a radar point-cloud having the first set of spatial coordinates and Doppler speed information of the one or more traffic objects, the first dataset being captured according to a coverage area of the radar sensor that is fixedly mounted, and

the second dataset is an image data including an arrangement of pixels that create the second set of spatial coordinates of the one or more traffic objects, the second dataset being captured according to a field of view of the image sensor that is fixedly mounted.

3 . The method for monitoring traffic of claim 2 , wherein the method further comprises:

clustering the one or more traffic objects of the first dataset and the second dataset prior to applying the first classifier and the second classifier;

applying the first classifier on a first cluster of the first dataset to generate the first classifications;

applying the second classifier on a second cluster of the second dataset to generate the second classifications;

detecting the one or more traffic objects by associating detection points to:

the first cluster according to the first set of spatial coordinates, and

the second cluster according to the second set of spatial coordinates; and

monitoring the one or more traffic objects continuously to determine a first trajectory(ies) of the one or more traffic objects derived from the first dataset and a second trajectory(ies) of the one or more traffic objects derived from the second dataset.

4 . The method for monitoring traffic of claim 3 , wherein integrating the first classifications and the second classifications involves:

fusing the first classifications and the second classifications through a decision-level fusion process that leverages weighted averaging and/or voting mechanisms to create the composite classification; and

fusing the first trajectories and the second trajectories by transforming the first dataset and the second dataset according to spatial attributes of the one or more traffic objects to create a refined trajectory for the one or more traffic objects, wherein transforming the first dataset and the second dataset involves using a sensor fusion algorithm.

5 . The method for monitoring traffic of claim 4 , wherein fusing the first classifications and the second classifications through the decision-level fusion process involves adjusting weights based on a confidence level of the first classifications and the second classifications.

6 . The method for monitoring traffic of claim 4 , wherein the method further comprises tracking the one or more traffic objects by using the composite classification and the refined trajectory.

7 . The method for monitoring traffic of claim 1 , wherein the radar sensor and the image sensor are movably mounted to observe the traffic.

8 . The method for monitoring traffic of claim 1 , wherein the synchronizing step further comprises using Robot Operating System (ROS) messages to coordinate the radar data stream with the image data stream, and wherein a rate of synchronization is in a range from 25 milliseconds to 35 milliseconds.

9 . The method for monitoring traffic of claim 1 , wherein:

the first classifier employs neural networks or support vector machines to generate the first classifications from the first dataset, and

the second classifier uses convolutional neural networks or feature extraction algorithms to generate the second classifications from the second dataset.

10 . A system for monitoring traffic, the system comprising:

a radar sensor configured to obtain a first dataset, wherein:

the first dataset includes a first set of spatial coordinates and a velocity of one or more traffic objects;

an image sensor configured to obtain a second dataset, wherein:

the second dataset includes a second set of spatial coordinates of the one or more traffic objects, and wherein the first dataset and the second dataset each is dynamically adjusted based on environmental factors; and

a fusion processor communicably coupled to the radar sensor and the image sensor to:

synchronize the radar sensor with the image sensor to coordinate a radar data stream with an image data stream;

apply a first classifier on the first dataset to generate first classifications;

apply a second classifier on the second dataset to generate second classifications; and

integrate the first classifications and the second classifications to generate a composite classification.

11 . The system for monitoring traffic of claim 10 , wherein:

the first dataset is a radar point-cloud having the first set of spatial coordinates and Doppler speed information of the one or more traffic objects, the first dataset being captured according to a coverage area of the radar sensor that is fixedly mounted, and

the second dataset is an image data including an arrangement of pixels that create the second set of spatial coordinates of the one or more traffic objects, the second dataset being captured according to a field of view of the image sensor that is fixedly mounted.

12 . The system for monitoring traffic of claim 11 , wherein the fusion processor is configured to:

cluster the one or more traffic objects of the first dataset and the second dataset prior to applying the first classifier and the second classifier;

apply the first classifier on a first cluster of the first dataset to generate the first classifications;

apply the second classifier on a second cluster of the second dataset to generate the second classifications;

detect the one or more traffic objects by associating detection points to:

the first cluster according to the first set of spatial coordinates, and

the second cluster according to the second set of spatial coordinates; and

monitor the plurality of traffic objects continuously to determine a first trajectory(ies) of the one or more traffic objects derived from the first data set and a second trajectory(ies) of the one or more traffic objects derived from the second data set.

13 . The system for monitoring traffic of claim 12 , wherein the fusion processor is further configured to integrate the first classifications and the second classifications by:

fusing the first classifications and the second classifications through a decision-level fusion process that leverages weighted averaging and/or voting mechanisms to create the composite classification; and

fusing the first trajectories and the second trajectories by transforming the first dataset and the second dataset according to spatial attributes of the one or more traffic objects to create a refined trajectory for the one or more traffic objects, wherein transforming the first dataset and the second dataset involves using a sensor fusion algorithm.

14 . The system for monitoring traffic of claim 13 , wherein the fusion processor is further configured to fuse the first classifications and the second classifications through the decision-level fusion process that involves adjusting weights based on a confidence level of the first classifications and the second classifications.

15 . The system for monitoring traffic of claim 13 , wherein the fusion processor is further configured to track the one or more traffic objects by using the composite classification and the refined trajectory.

16 . The system for monitoring traffic of claim 10 , wherein the radar sensor and the image sensor are movably mounted to observe the traffic.

17 . The system for monitoring traffic of claim 10 , wherein the fusion processor uses Robot Operating System (ROS) messages to coordinate the radar data stream with the image data stream, and wherein a rate of synchronization is in a range from 25 milliseconds to 35 milliseconds.

18 . The system for monitoring traffic of claim 10 , wherein:

the first classifier employs neural networks or support vector machines to generate the first classifications from the first dataset, and

the second classifier uses convolutional neural networks or feature extraction algorithms to generate the second classifications from the second dataset.