Methods and devices for triggering vehicular actions based on passenger actions

1. A device to trigger a vehicular action based on monitoring one or more occupants of a vehicle, the device comprising one or more processors configured to:

identify a plurality of features in a plurality of subsets of image data detailing the one or more occupants;

track changes over time of the plurality of features over the plurality of subsets of image data;

determine a state, from a plurality of states, of the one or more occupants based on the tracked changes;

trigger the vehicular action based on the determined state;

identify the plurality of features by providing inputs, based on the plurality of subsets of image data, to a two-dimensional convolutional neural network (2D CNN); and

determine each of the inputs, at least in part, based on a point location in each of the plurality of subsets of image data.

2. The device of claim 1 , wherein the plurality of subsets of image data are based on video frames obtained from a video taken of the one or more occupants of the vehicle.

3. The device of claim 1 , wherein the 2D CNN provides an output comprising the identified plurality of features in each of the plurality of subsets of image data.

4. The device of claim 3 , wherein the output comprises a plurality of segments, wherein each of the plurality of segments corresponds to a respective subset of the plurality of subsets of image data, and wherein each segment comprises a value corresponding to each of the plurality of features.

5. The device of claim 4 , wherein the value for each respective feature ranges from a minimum to a maximum, wherein the minimum indicates that the respective feature is non-existent in the segment and an increase in the value towards the maximum indicates that the respective feature is more prevalent in the segment.

6. The device of claim 1 , wherein the one or more processors are configured to track the changes by providing the plurality of features in the plurality of subsets of image data in a series of segments as inputs to a spatio-temporal model, wherein each segment corresponds to a subset of image data of the plurality of subsets of image data.

7. The device of claim 6 , wherein the spatio-temporal model comprises a plurality of kernels to track changes in the values corresponding to a respective feature across the series of segments.

8. The device of claim 7 , wherein the one or more processors are configured to track the changes by identifying an increase in the value of a respective feature across the series of segments, wherein the increase in value corresponds to an emergence of the feature.

9. The device of claim 7 , wherein the one or more processors are configured to track the changes by identifying a decrease in the value of a respective feature across the series of segments, wherein the decrease in value corresponds to the feature vanishing.

10. The device of claim 1 , wherein the one or more processors are configured to determine the state of the one or more occupants by selecting a state from the plurality of states with a highest probability of occurrence based on the tracked changes of each of the plurality of features.

11. The device of claim 1 , wherein each of the plurality of states has a time value associated with it, wherein the one or more processors are configured to trigger the vehicular action by calculating a theoretical safe distance based on the time value.

12. The device of claim 11 , wherein the data comprises a second set of data corresponding to one or more objects detected outside of the vehicle, wherein the one or more processors are configured to determine a real physical distance to the one or more detected objects.

13. The device of claim 11 , wherein the one or more processors are configured to trigger the vehicular action based on a comparison of the theoretical safe distance and the real physical distance.

14. The device of claim 13 , wherein the one or more processors are configured to trigger the vehicular action by modifying one or more driving parameters based on the comparison or by selecting a notification to send via a user interface of the vehicle.

15. A method to trigger a vehicular action based on monitoring one or more occupants of a vehicle, the method comprising:

identifying a plurality of features in a plurality of subsets of image data detailing the one or more occupants;

tracking changes over time of the plurality of features over the plurality of subsets of image data;

determining a state, from a plurality of states, of the one or more occupants based on the tracked changes;

triggering the vehicular action based on the determined state;

identifying the plurality of features by providing inputs, based on the plurality of subsets of image data, to a two-dimensional convolutional neural network (2D CNN); and

determining each of the inputs, at least in part, based on a point location in each of the plurality of subsets of image data.

16. The method of claim 15 , wherein each of the plurality of states has a time value associated with it, the method further comprising triggering the vehicular action by calculating a theoretical safe distance based on the time value.

17. One or more non-transitory computer readable media including instructions thereon that, when executed by one or more processors of a device, cause the device to:

identify a plurality of features in a plurality of subsets of image data detailing the one or more occupants;

track changes over time of the plurality of features over the plurality of subsets of image data;

determine a state, from a plurality of states, of the one or more occupants based on the tracked changes;

trigger the vehicular action based on the determined state;

identify the plurality of features by providing inputs, based on the plurality of subsets of image data, to a two-dimensional convolutional neural network (2D CNN); and

determine each of the inputs, at least in part, based on a point location in each of the plurality of subsets of image data.

18. The one or more non-transitory computer readable media or claim 17 , wherein each of the plurality of states has a time value associated with it, and further causing the device to trigger the vehicular action by calculating a theoretical safe distance based on the time value.