Robust radar-based gesture-recognition by user equipment
Systems and techniques are described for robust radar-based gesture-recognition. A radar system ( 104 ) detects radar-based gestures on behalf of application subscribers. A state machine ( 2000 ) transitions between multiple states based on inertial sensor data. A no-gating state ( 2002 ) enables the radar system ( 104 ) to output radar-based gestures to application subscribers. The state machine ( 2000 ) also includes a soft-gating state ( 2004 ) that prevents the radar system ( 104 ) from outputting the radar-based gestures to the application subscribers. A hard-gating state ( 2006 ) prevents the radar system ( 104 ) from detecting radar-based gestures altogether. The techniques and systems enable the radar system ( 104 ) to determine when not to perform gesture-recognition, enabling user equipment ( 102 ) to automatically reconfigure the radar system ( 104 ) to meet user demand. By so doing, the techniques conserve power, improve accuracy, or reduce latency relative to many common techniques and systems for radar-based gesture-recognition.
1. An apparatus comprising:
a radar system that detects radar-based gestures on behalf of application subscribers;
an inertial measurement unit that produces inertial data; and
a state machine configured to transition between multiple states for controlling the radar system based on the inertial data and one or more context-sensitive transition functions, the multiple states including:
a no-gating state in which the state machine enables the radar system to output indications of the radar-based gestures to the application subscribers;
a soft-gating state in which the state machine prevents the radar system from outputting the indications of the radar-based gestures to one or more of the application subscribers by shielding a gesture-recognition model from radar data collected by the radar system or by shielding one or more of the application subscribers from the output of the gesture-recognition model; and
a hard-gating state in which the state machine prevents the radar system from detecting the radar-based gestures by disabling the gesture-recognition model.
2. The apparatus of claim 1 , further comprising:
a low-pass filter,
wherein the inertial sensor data is filtered through the low-pass filter.
3. The apparatus of claim 1 , wherein:
the inertial data comprises an orientation of the device; and
the state machine is further configured to transition between the multiple states based on the orientation of the device.
4. The apparatus of claim 1 , further comprising:
a proximity sensor that detects a proximity of an object to the apparatus,
the state machine is further configured to transition between the multiple states based on the proximity of the object.
5. The apparatus of claim 1 , wherein the state machine is further configured to:
enter the no-gating state; and
transition to the soft-gating state responsive to a first context-sensitive transition function of the context-sensitive transition functions producing a first contextual score that meets a first threshold.
6. The apparatus of claim 1 , wherein the state machine is further configured to:
enter the no-gating state; and
transition to the hard-gating state responsive to a second context-sensitive transition function of the context-sensitive transition functions producing a second contextual score that meets a second threshold.
7. The apparatus of claim 1 , wherein the state machine is further configured to:
enter the no-gating state; and
transition to the soft-gating state responsive to a first context-sensitive transition function of the context-sensitive transition functions producing a first contextual score that meets a first threshold and a second context-sensitive transition function of the context-sensitive transition functions producing a second contextual score that meets a second threshold, wherein the first contextual score is higher than the second contextual score.
8. The apparatus of claim 1 , wherein the state machine is further configured to:
enter the no-gating state; and
transition to the soft-gating state responsive to a first context-sensitive transition function of the context-sensitive transition functions producing a first contextual score that meets a first threshold and a second context-sensitive transition function of the context-sensitive transition functions producing a second contextual score that meets a second threshold, wherein the first contextual score is lower than the second contextual score.
9. The apparatus of claim 1 , wherein the soft-gating state causes the apparatus to consume more power than the hard-gating state.
10. The apparatus of claim 1 , wherein:
at least one of the context-sensitive transition functions outputs a contextual score based on a comparison between a position, orientation, or movement inferred from the inertial data and a position, orientation, or movement threshold; and
the state machine is further configured to transition between the multiple states based on the contextual score.
11. The apparatus of claim 1 , wherein:
one of the context-sensitive transition functions outputs a contextual score based on whether the radar system is occluded by an object; and
the state machine is further configured to transition between the multiple states based on the contextual score.
12. The apparatus of claim 11 , wherein the state machine is further configured to:
enter the no-gating state; and
transition to the soft-gating state in response to the contextual score indicating that the apparatus is not occluded by the object.
13. The apparatus of claim 11 , wherein the state machine is further configured to:
enter the no-gating state; and
transition to the hard-gating state in response to the contextual score indicating that the apparatus is occluded by the object.
14. A method of controlling a radar system of a device, the method performed by the device and comprising:
receiving inertial data from an inertial measurement unit of the device; and
configuring, based on the inertial data and via at least one or more context-sensitive transition functions, the device to one of multiple radar system gating states, the radar system detecting radar-based gestures on behalf of application subscribers, the application subscribers configured to receive an output from a gesture-recognition model, the multiple states including:
a no-gating radar system gating state in which the radar system is enabled to output indications of the radar-based gestures to the application subscribers;
a soft-gating radar system gating state in which the radar system is prevented from outputting the indications of the radar-based gestures to one or more of the application subscribers by shielding the gesture-recognition model from radar data collected by the radar system or by shielding one or more of the application subscribers from the output of the gesture-recognition model; and
a hard-gating radar system gating state in which the radar system is prevented from detecting the radar-based gestures by disabling the gesture-recognition model.
15. The method of claim 14 , wherein;
the inertial data indicates an orientation of the device; and
the configuring is further based on the orientation of the device.
16. The method of claim 14 , further comprising:
receiving an indication of proximity from the apparatus to an object,
wherein the configuring is further based on the proximity from the apparatus to the object.
17. The method of claim 14 , wherein the configuring comprises:
configuring the device to the no-gating radar system gating state; and
configuring the device to the soft-gating radar system gating state responsive to a first context-sensitive transition function of the context-sensitive transition functions producing a first contextual score that meets a first threshold.
18. The method of claim 14 , wherein the configuring comprises:
configuring the device to the no-gating radar system gating state; and
configuring the device to the hard-gating radar system gating state responsive to a second context-sensitive transition function of the context-sensitive transition functions producing a second contextual score that meets a second threshold.
19. The method of claim 14 , wherein the configuring comprises:
configuring the device to the no-gating radar system gating state; and
configuring the device to the soft-gating radar system gating state responsive to a first context-sensitive transition function of the context-sensitive transition functions producing a first contextual score that meets a first threshold and a second context-sensitive transition function of the context-sensitive transition functions producing a second contextual score that meets a second threshold, wherein the first contextual score is greater than the second contextual score.
20. The method of claim 14 , wherein the configuring comprises:
configuring the device to the no-gating radar system gating state; and
configuring the device to the hard-gating radar system gating state responsive to a first context-sensitive transition function of the context-sensitive transition functions producing a first contextual score that meets a first threshold and a second context-sensitive transition function of the context-sensitive transition functions producing a second contextual score that meets a second threshold, wherein the first contextual score is less than the second contextual score.
21. The apparatus of claim 1 , wherein the hard-gating state in which the state machine prevents the radar system from detecting the radar-based gestures disables the gesture-recognition model by operating in a low-power mode or an intermediate-power mode, the low-power mode or the intermediate-power mode effective to disable the gesture recognition module.
22. The method of claim 14 , wherein the configuring the device to the hard-gating radar system gating state in which the radar system is prevented from detecting the radar-based gesture disables the gesture-recognition model by causing the radar system to operate in a low-power mode or an intermediate-power mode, the low-power mode or the intermediate-power mode effective to disable the gesture recognition module.