Monitoring and responding to poor visibility conditions by analyzing images using deep neural network
An apparatus comprises an interface and a processor. The interface may be configured to receive pixel data corresponding to an exterior view from a vehicle. The processor may be configured to process the pixel data arranged as video frames and perform computer vision operations on the video frames. The computer vision operations may be configured to detect a reference object in the video frames, calculate a first distance to the reference object during a first time period, and calculate a second distance to the reference object during a second time period. The processor may be further configured to compare the second distance to the first distance, determine a presence of a visual obstruction based on the comparison, and generate a control signal to enable a lamp in response to determining the presence of the visual obstruction.
1. An apparatus comprising:
an interface configured to receive pixel data corresponding to an exterior view from a vehicle; and
a processor configured to (i) process the pixel data arranged as video frames, (ii) perform computer vision operations on the video frames to (a) detect a reference object in the video frames, (b) calculate a first distance to the reference object during a first time period, and (c) calculate a second distance to the reference object during a second time period, (iii) compare the second distance to the first distance, (iv) determine a presence of a visual obstruction based on the comparison, and (v) generate a control signal to enable a lamp in response to determining the presence of the visual obstruction, wherein the reference object comprises a road marking for a dashed center line.
2. The apparatus according to claim 1 , wherein the visual obstruction is at least one of fog, rain, or snow.
3. The apparatus according to claim 1 , wherein the processor, in determining the presence of the visual obstruction, is further configured to (i) perform computer vision operations on the video frames to calculate a third distance to the reference object during a third time period while the lamp is enabled, (ii) compare the third distance to the second distance, and (iii) determine a continuing status of the lamp based on the comparison of the third distance to the second distance.
4. The apparatus according to claim 1 , wherein the processor is further configured to (i) perform computer vision operations on the video frames during a third time period to detect a second vehicle behind the vehicle and (ii) generate the control signal to disable the lamp in response to detecting the second vehicle.
5. The apparatus according to claim 4 , wherein (i) the control signal is configured to enable and disable the lamp and a second fog lamp, (ii) the lamp comprises a rear fog lamp and the second fog lamp comprises a front fog lamp, and (iii) the computer vision operations on the video frames during the third time period to detect the second vehicle are performed on portions of the video frames comprising a representation of a rear view mirror.
6. The apparatus according to claim 1 , wherein (i) the interface is further configured to receive additional pixel data corresponding to a second exterior view from the vehicle and (ii) the processor is further configured to (a) process the additional pixel data arranged as additional video frames, (b) perform computer vision operations on the additional video frames during a third time period to detect a second vehicle behind the vehicle, and (c) generate the control signal to disable a second lamp in response to detecting the second vehicle.
7. The apparatus according to claim 6 , wherein (i) the lamp comprises a front fog lamp, (ii) the second lamp comprises a rear fog lamp, and (iii) the second exterior view is to a rear of the vehicle.
8. The apparatus according to claim 1 , wherein the computer vision operations are implemented by a convolutional neural network.
9. The apparatus according to claim 8 , wherein the convolutional neural network is trained using fleet learning.
10. The apparatus according to claim 9 , wherein (i) the fleet learning comprises capturing reference images, (ii) the reference images comprise a surface of a roadway, and (iii) the reference images are used as training data for the convolutional neural network.
11. The apparatus according to claim 1 , wherein (i) the computer vision operations detect the reference object by performing feature extraction based on weight values for each of a plurality of visual features that are associated with the reference object extracted from the video frames and (ii) the weight values are determined in response to an analysis of training data by the processor prior to the feature extraction.
12. The apparatus according to claim 1 , wherein the computer vision operations are further performed by (i) applying a feature detection window to each of a plurality of layers extracted from the video frames and (ii) a convolution operation using matrix multiplication of the plurality of layers defined by the feature detection window.
13. The apparatus according to claim 1 , wherein calculations for the first distance and the second distance are performed by calculating a furthest distance to a reoccurring reference object.
14. The apparatus according to claim 1 , wherein the processor is further configured to (i) perform computer vision operations on the video frames during a third time period to detect a second vehicle behind the vehicle, (ii) analyze characteristics of the second vehicle to detect a visual alert, (iii) determine whether the visual alert is related to the lamp, (iv) disable the lamp in response to the visual alert if the visual alert is related to the lamp and (v) do not disable the lamp in response to the visual alert if the visual alert is not related to the lamp.
15. The apparatus according to claim 14 , wherein the visual alert is related to the lamp if the visual alert comprises at least one of flashing head lights, flashing high beams and flashing hazards.
16. The apparatus according to claim 14 , wherein the processor is further configured to (i) perform the computer vision operations to determine whether the second vehicle has passed the vehicle if the lamp has been disabled in response to the visual alert and (ii) re-enable the lamp in response to the second vehicle passing the vehicle.
17. An apparatus comprising:
an interface configured to receive pixel data corresponding to an exterior view from a vehicle; and
a processor configured to (i) process the pixel data arranged as video frames, (ii) perform computer vision operations on the video frames to (a) detect a reference object in the video frames, (b) calculate a first distance to the reference object during a first time period, and (c) calculate a second distance to the reference object during a second time period, (iii) compare the second distance to the first distance, (iv) determine a presence of a visual obstruction based on the comparison, and (v) generate a control signal to enable a lamp in response to determining the presence of the visual obstruction, wherein calculations for the first distance and the second distance are performed by calculating a furthest distance to a reoccurring reference object.
18. An apparatus comprising:
an interface configured to receive pixel data corresponding to an exterior view from a vehicle; and
a processor configured to (i) process the pixel data arranged as video frames, (ii) perform computer vision operations on the video frames to (a) detect a reference object in the video frames, (b) calculate a first distance to the reference object during a first time period, and (c) calculate a second distance to the reference object during a second time period, (iii) compare the second distance to the first distance, (iv) determine a presence of a visual obstruction based on the comparison, and (v) generate a control signal to enable a lamp in response to determining the presence of the visual obstruction, wherein the processor, in determining the presence of the visual obstruction, is further configured to
(i) perform computer vision operations on the video frames to calculate a third distance to the reference object during a third time period while the lamp is enabled,
(ii) compare the third distance to the second distance, and
(iii) determine a continuing status of the lamp based on the comparison of the third distance to the second distance.
19. The apparatus according to claim 18 , wherein (i) the third distance is greater than the second distance and (ii) the continuing status of the lamp is enabled.
20. The apparatus according to claim 18 , wherein (i) the third distance is less than the second distance, (ii) the continuing status of the lamp is disabled, and (iii) the processor is further configured to generate the control signal to disable the lamp in response to determining the third distance is less than the second distance.