Systems and methods for estimating grip intensity on a steering wheel
Systems, methods, and other embodiments described herein relate to implementing and calibrating a learning model for inferring operator intent by estimating grip intensity. In one embodiment, a method includes estimating, using a learning model during a driving scenario, first grip intensity on a steering device for a vehicle according to initial image data depicting a hand of an operator gripping outside the set areas that have pressure sensors. The method also includes calibrating the learning model for the operator and the steering device using grip measurements and additional image data acquired from gripping inside the set areas. The method also includes computing, using the learning model during the driving scenario, second grip intensity outside the set areas on the steering device according to hand images acquired about the operator. The method also includes adapting a vehicle parameter of the vehicle according to the second grip intensity.
1. A detection system comprising:
a camera;
a processor; and
a memory storing instructions that, when executed by the processor, cause the processor to:
estimate, using a learning model during a driving scenario, first grip intensity on a steering device for a vehicle according to first image data depicting a hand of an operator gripping outside a first area and a second area that have pressure sensors;
calibrate and modify the learning model for the operator and the steering device using grip measurements acquired from the pressure sensors and second image data acquired by the camera from the operator gripping inside the first area;
compute, using the learning model during the driving scenario, second grip intensity from outside the first area on the steering device according to hand images acquired about the operator using the camera; and
adapt a vehicle parameter of the vehicle and control the vehicle according to the second grip intensity that indicates a driving state of the operator.
2. The detection system of claim 1 , further including instructions to:
detect a shape of the hand on the steering device using the second grip intensity and the hand images;
adjust the second grip intensity according to the shape; and
modify automated braking automatically and output an operator warning automatically by calculating a mechanical coupling between the hand and the steering device according to the second grip intensity without detection of the mechanical coupling by a steer-by-wire system, and the mechanical coupling that is limited indicates anomalous handling of the steering device.
3. The detection system of claim 1 , further including instructions to:
measure, offline in a testing environment, pressure intensities of hand grips using the pressure sensors outside the first area and the second area located throughout the steering device; and
train the learning model to output grip intensities inferred by correlating test images of grip patterns with the pressure intensities and a steering angle of the steering device.
4. The detection system of claim 1 , further including instructions to:
measure blood flow and pulse of the operator using grip data from an infrared camera; and
adjust, using the learning model, the second grip intensity using the blood flow and the pulse.
5. The detection system of claim 1 , further including instructions to:
calculate a grip force using the second grip intensity, the grip force is a mechanical coupling for the operator handling the steering device in the vehicle; and
determine an intent of the operator and altering a steering angle according to the mechanical coupling.
6. The detection system of claim 1 , further including instructions to:
calculate a grip pressure using the second grip intensity, the grip pressure is a mechanical coupling for the operator handling the steering device in the vehicle; and
alter a driving command in the vehicle according to the grip pressure.
7. The detection system of claim 1 , further including instructions to:
generate a notification by a driver monitoring system (DMS) using the vehicle parameter, wherein the vehicle parameter indicates an intent for altering a vehicle command.
8. The detection system of claim 1 , wherein the hand images are from a back view of the operator.
9. The detection system of claim 1 , wherein the steering device is one of a joystick, a knob, a yoke wheel, and a circular steering wheel.
10. A non-transitory computer-readable medium comprising:
instructions that when executed by a processor cause the processor to:
estimate, using a learning model during a driving scenario, first grip intensity on a steering device for a vehicle according to first image data depicting a hand of an operator gripping outside a first area and a second area that have pressure sensors;
calibrate and modify the learning model for the operator and the steering device using grip measurements acquired from the pressure sensors and second image data acquired by a camera in communication with the processor from the operator gripping inside the first area;
compute, using the learning model during the driving scenario, second grip intensity from outside the first area on the steering device according to hand images acquired about the operator using the camera; and
adapt a vehicle parameter of the vehicle and control the vehicle according to the second grip intensity that indicates a driving state of the operator.
11. The non-transitory computer-readable medium of claim 10 , further including instructions to:
detect a shape of the hand on the steering device using the second grip intensity and the hand images;
adjust the second grip intensity according to the shape; and
modify automated braking automatically and output an operator warning automatically by calculating a mechanical coupling between the hand and the steering device according to the second grip intensity without detection of the mechanical coupling by a steer-by-wire system, and the mechanical coupling that is limited indicates anomalous handling of the steering device.
12. The non-transitory computer-readable medium of claim 10 , further including instructions to:
measure, offline in a testing environment, pressure intensities of hand grips using the pressure sensors outside the first area and the second area located throughout the steering device; and
train the learning model to output grip intensities inferred by correlating test images of grip patterns with the pressure intensities and a steering angle of the steering device.
13. A method comprising:
estimating, using a learning model during a driving scenario, first grip intensity on a steering device for a vehicle according to first image data depicting a hand of an operator gripping outside a first area and a second area that have pressure sensors;
calibrating and modifying the learning model for the operator and the steering device using grip measurements acquired from the pressure sensors and second image data acquired by a camera from the operator gripping inside the first area;
computing, using the learning model during the driving scenario, second grip intensity from outside the first area on the steering device according to hand images acquired about the operator using the camera; and
adapting a vehicle parameter of the vehicle and controlling the vehicle according to the second grip intensity that indicates a driving state of the operator.
14. The method of claim 13 , further comprising:
detecting a shape of the hand on the steering device using the second grip intensity and the hand images;
adjusting the second grip intensity according to the shape; and
modifying automated braking automatically and outputting an operator warning automatically by calculating a mechanical coupling between the hand and the steering device according to the second grip intensity without detection of the mechanical coupling by a steer-by-wire system, and the mechanical coupling that is limited indicates anomalous handling of the steering device.
15. The method of claim 13 , further comprising:
measuring, offline in a testing environment, pressure intensities of hand grips using the pressure sensors outside the first area and the second area located throughout the steering device; and
training the learning model to output grip intensities inferred by correlating test images of grip patterns with the pressure intensities and a steering angle of the steering device.
16. The method of claim 13 , further comprising:
measuring blood flow and pulse of the operator using grip data from an infrared camera; and
adjusting, using the learning model, the second grip intensity using the blood flow and the pulse.
17. The method of claim 13 , further comprising:
calculating a grip force using the second grip intensity, the grip force is a mechanical coupling for the operator handling the steering device in the vehicle; and
determining an intent of the operator and altering a steering angle according to the mechanical coupling.
18. The method of claim 13 , further comprising:
calculating a grip pressure using the second grip intensity, the grip pressure is a mechanical coupling for the operator handling the steering device in the vehicle; and
altering a driving command in the vehicle according to the grip pressure.
19. The method of claim 13 , further comprising:
generating a notification by a driver monitoring system (DMS) using the vehicle parameter, wherein the vehicle parameter indicates an intent for altering a vehicle command.
20. The method of claim 13 , wherein the hand images are from a back view of the operator.