Methods and systems for determining a state indicating whether a seat belt of a vehicle is used

1 . A computer implemented method for determining a state indicating whether a seat belt of a vehicle is used, the method comprising the following steps carried out by a computer:

acquiring, using a camera, at least one image of a portion of an interior of the vehicle, the camera being located in a front row central high position of the vehicle that is one of at a rear-view mirror of the vehicle, in a central console of the vehicle, above a dashboard of the vehicle, or in a roof overhead console of the vehicle;

determining whether the at least one image comprises a buckle receiver;

responsive to determining that the at least one image comprises the buckle receiver, determining the state based on the at least one image; and

responsive to determining that the at least one image does not comprise the buckle receiver, performing:

extracting, from the at least one image, using a machine-learning image processing method that utilizes an artificial neural network, at least one observed cue from a plurality of observed cues related to a user of the seat belt or to a buckle of the seat belt, each observed cue of the plurality of observed cues being associated with a conditional probability of the state indicating, given the observed cue, (i) whether, when the state of the seat belt is in an unbuckled state in a previous time step, the state remains in the unbuckled state or changes to a buckled state, or (ii) whether, when the state of the seat belt is in the buckled state in the previous time step, the state remains in the buckled state or changes to the unbuckled state;

determining, using Bayes' theorem, a probability of a change of the state of the seat belt based on the conditional probability associated with each of the at least one observed cue extracted from the at least one image; and

updating the state of the seat belt based on the determined probability; and

outputting a seat belt reminder to the user based on the state of the seat belt.

2 . The computer implemented method of claim 1 , wherein the at least one observed cue extracted from the at least one image relates to at least one key point of at least one body part of the user.

3 . The computer implemented method of claim 1 , wherein the at least one observed cue extracted from the at least one relates to a position of the buckle.

4 . The computer implemented method of claim 1 , wherein the portion of the interior of the vehicle comprises at least one of a portion near the buckle of the seat belt or a portion near the buckle receiver of the seat belt.

5 . The computer implemented method of claim 1 , wherein if it is determined that the at least one image does not comprise the buckle receiver, the method further comprises:

determining a trajectory of a hand of the user of the seat belt relative to the buckle;

wherein the at least one observed cue extracted from the at least one image includes the determined trajectory of the hand of the user.

6 . The computer implemented method of claim 5 , further comprising:

determining the trajectory of the hand towards the buckle receiver; and

setting the associated probability for the observed cue associated with the determined trajectory of the hand of the user of a change of the state from the unbuckled state to the buckled state higher than a probability of a change of the state from the buckled state to the unbuckled state.

7 . The computer implemented method of claim 5 , further comprising:

determining the trajectory of the hand away from the buckle receiver; and

setting the associated probability for the observed cue associated with the determined trajectory of the hand of the user of a change of the state from the unbuckled state to the buckled state lower than a probability of a change of the state from the buckled state to the unbuckled state.

8 . The computer implemented method of claim 1 , wherein if it is determined that the at least one image does not comprise the buckle receiver, the method further comprises:

classifying a pose of a hand of the user of the seat belt near the buckle;

wherein the at least one observed cue extracted from the at least one image includes the determined pose.

9 . The computer implemented method of claim 8 , further comprising:

classifying the pose of the hand as a grabbing pose; and

setting the associated probability for the observed cue associated with the determined pose of a change of the state from the unbuckled state to the buckled state higher than a probability of a change of the state from the unbuckled state to the buckled state in response to the determined pose being an open hand pose.

10 . The computer implemented method of claim 1 , wherein if it is determined that the at least one image does not comprise the buckle receiver, the method further comprises:

determining whether the buckle is located in a hand of the user;

wherein the at least one observed cue extracted from the at least one image includes whether the buckle is located in the hand of the user.

11 . The computer implemented method of claim 1 , further comprising:

determining a gaze direction of the user of the seat belt relative to the buckle;

wherein the at least one observed cue extracted from the at least one image includes the gaze direction.

12 . A computer system comprising:

a plurality of computer hardware components including a processor; and

a non-transitory computer readable medium comprising instructions, which when executed by the processor, cause the processor to perform operations comprising:

acquire, using a camera, at least one image of a portion of an interior of a vehicle, the camera being located in a front row central high position of the vehicle that is one of at a rear-view mirror of the vehicle, in a central console of the vehicle, above a dashboard of the vehicle, or in a roof overhead console of the vehicle;

determine whether the at least one image comprises a buckle receiver;

responsive to a determination that the at least one image comprises the buckle receiver, determine a state indicating whether a seat belt of the vehicle is used based on the at least one image; and

responsive to a determination that the at least one image does not comprise the buckle receiver:

extract, from the at least one image, using a machine-learning image processing method that utilizes an artificial neural network, at least one observed cue from a plurality of observed cues related to a user of the seat belt or to a buckle of the seat belt, each observed cue of the plurality of observed cues being associated with a conditional probability of the state indicating, given the observed cue, (i) whether, when the state of the seat belt is in an unbuckled state in a previous time step, the state remains in the unbuckled state or changes to a buckled state, or (ii) whether, when the state of the seat belt is in the buckled state in the previous time step, the state remains in the buckled state or changes to the unbuckled state;

determine, using Bayes' theorem, a probability of a change of the state of the seat belt based on the conditional probability associated with each of the at least one observed cue extracted from the at least one image; and

update the state of the seat belt based on the determined probability; and

output a seat belt reminder to the user based on the state of the seat belt.

13 . The computer system according to claim 12 , wherein the at least one observed cue extracted from the at least one image relates to at least one key point of at least one body part of the user.

14 . The computer system according to claim 12 , wherein the at least one observed cue extracted from the at least one relates to a position of the buckle.

15 . The computer system according to claim 12 , wherein the portion of the interior of the vehicle comprises at least one of a portion near the buckle of the seat belt or a portion near the buckle receiver of the seat belt.

16 . The computer system according to claim 12 , wherein further responsive to the determination that the at least one image does not comprise the buckle receiver, the instructions further cause the processor to execute further operations to:

determine a trajectory of a hand of the user of the seat belt relative to the buckle;

wherein the at least one observed cue extracted from the at least one image includes the determined trajectory of the hand of the user.

17 . The computer system according to claim 12 , wherein further responsive to the determination that the at least one image does not comprise the buckle receiver, the instructions further cause the processor to execute further operations to:

classify a pose of a hand of the user of the seat belt near the buckle;

wherein the at least one observed cue extracted from the at least one image includes the determined pose.

18 . The computer system according to claim 12 , wherein further responsive to the determination that the at least one image does not comprise the buckle receiver, the instructions further cause the processor to execute further operations to:

determine whether the buckle is located in a hand of the user;

wherein the at least one observed cue extracted from the at least one image includes whether the buckle is located in the hand of the user.

19 . A vehicle comprising:

a camera located in a front row central high position of the vehicle that is one of at a rear-view mirror of the vehicle, in a central console of the vehicle, above a dashboard of the vehicle, or in a roof overhead console of the vehicle;

a processor;

a non-transitory computer readable medium comprising instructions, which when executed by the processor, cause the processor to:

acquire, from the camera, at least one image of a portion of an interior of the vehicle;

determine whether the at least one image comprises a buckle receiver;

responsive to a determination that the at least one image comprises the buckle receiver, determine a state indicating whether a seat belt of the vehicle is used based on the at least one image; and

responsive to a determination that the at least one image does not comprise the buckle receiver:

extract, from the at least one image, using a machine-learning image processing method that utilizes an artificial neural network, at least one observed cue from a plurality of observed cues related to a user of the seat belt or to a buckle of the seat belt, each observed cue of the plurality of observed cues being associated with a conditional probability of the state indicating, given the observed cue, (i) whether, when the state of the seat belt is in an unbuckled state in a previous time step, the state remains in the unbuckled state or changes to a buckled state, or (ii) whether, when the state of the seat belt is in the buckled state in the previous time step, the state remains in the buckled state or changes to the unbuckled state;

determine, using Bayes' theorem, a probability of a change of the state of the seat belt based on the conditional probability associated with each of the at least one observed cue extracted from the at least one image; and

update the state of the seat belt based on the determined probability; and

output a seat belt reminder to the user based on state of the seat belt.