Radar recognition-aided search
This document describes techniques and devices for a radar recognition-aided search. Through use of a radar-based recognition system, gestures made by, and physiological information about, persons can be determined. In the case of physiological information, the techniques can use this information to refine a search. For example, if a person requests a search for a coffee shop, the techniques may refine the search to coffee shops in the direction that the person is walking. In the case of a gesture, the techniques may refine or base a search solely on the gesture. Thus, a search for information about a store, car, or tree can be made responsive to a gesture pointing at the store, car, or tree with or without explicit entry of a search query.
1. A computer-implemented method comprising:
identifying verbalization expressed by a user, the verbalization comprising a question;
recognizing an in-the-air gesture performed by the user that has multiple potential meanings and is received through a radar-based recognition system, the recognizing based on reflections of a radar field received by an antenna of the radar-based recognition system;
causing a search to be performed, the search based on the question and refined using:
a determined meaning of an in-the-air gesture that is relevant to the question from the multiple potential meanings, the determining of the meaning based on a comparison of the multiple potential meanings associated with the in- the-air gesture in combination with the question; or
physiological information associated with the user, the physiological information relating to at least one of a body or skin temperature, heart rate, perspiration, stress level, hydration level, or orientation of the user; and
providing, by a device, search results responsive to the question.
2. The computer-implemented method as described in claim 1 , wherein physiological information associated with the user is obtained via the radar-based recognition system.
3. The computer-implemented method as described in claim 1 , wherein receiving the verbalization having the question receives the verbalization through an input of the device, determining the meaning about the in-the-air gesture that is relevant to the question determines a geographic direction and providing the search results provides the search results in the form of locations on a map, a list of terms or webpages, or a media presentation.
4. The computer-implemented method as described in claim 3 , wherein:
the geographic direction indicates a street; or
the geographic direction indicates a building, and the search is refined based on the determined meaning of the in-the-air gesture sufficient to provide search results indicative of addresses or stores associated with the building.
5. The computer-implemented method as described in claim 3 , wherein receiving the verbalization receives the question through an audio communication system and determining the meaning about the in-the-air gesture that is relevant to the question is responsive to and based on the in-the-air gesture being recognized commensurate with receiving the verbalization through the audio communication system.
6. The computer-implemented method as described in claim 1 , wherein the search is refined sufficient to provide search results that vary in at least one of relevancy, quality, verbosity, temporal proximity, or geographic proximity.
7. The computer-implemented method as described in claim 1 , wherein:
the question indicates a desire for a drink:
determining the meaning of the in-the-air gesture of the multiple potential meanings determines the in-the-air gesture is indicative of an exhaustion of the user; and
the search is refined based on physiological information associated with the user indicative of at least one of an orientation, temperature, or perspiration of the user sufficient to provide search results prioritizing a proximity to the user and a relevancy to the question.
8. The computer-implemented method as described in claim 1 , wherein determining the meaning of the in-the-air gesture of the multiple potential meanings determines the in-the-air gesture is indicative of a pointing gesture, the method further comprising:
obtaining, responsive to identifying the verbalization expressed by the user, sensor data of an environment surrounding the device, and wherein the search is refined based on the pointing gesture and is further refined based on the sensor data.
9. The computer-implemented method as described in claim 8 , wherein the sensor data comprises image data of a scene captured by an on-device camera, the method further comprising:
analyzing the image data to determine one or more objects, locations, or activities, and wherein the search is refined based on the one or more objects, locations, or activities sufficient to provide search results responsive to the question and relevant to the one or more objects, locations, or activities.
10. The computer-implemented method as described in claim 1 , wherein the in-the-air gesture comprises at least one of a movement of a part of a body of the user or an object associated with the user.
11. The computer-implemented method as described in claim 10 , wherein the in-the-air gesture comprises a stepped gesture, and wherein the determination of the meaning about the stepped gesture that is relevant to the question from the multiple potential meanings determines an intended path.
12. The computer-implemented method as described in claim 1 , further comprising:
receiving, prior to identifying, the verbalization expressed by the user at an audio input mechanism; and
transmitting, responsive to receiving, the verbalization from the audio input mechanism to the device.
13. A computing device comprising:
a processor;
a radar-based recognition system; and
a computer-readable media including a search manager, the search manager configured to cause the computing device to:
identify a verbalization expressed by a user, the verbalization comprising a question;
recognize an in-the-air gesture performed by the user that has multiple potential meanings and is received through a radar-based recognition system, the recognizing based on reflections of a radar field received by an antenna of the radar-based recognition system;
cause a search to be performed, the search based on the question and refined using:
a determined meaning of the in-the-air gesture that is relevant to the question from the multiple potential meanings, the determination of the meaning based on a comparison of the multiple potential meanings associated with the in-the-air gesture in combination with the question; or
physiological information associated with the user, the physiological information relating to at least one of a body or skin temperature, heart rate, perspiration, stress level, hydration level, or orientation of the user; and
provide, by the computing device, search results responsive to the question.
14. The computing device as described in claim 13 , wherein the radar based-recognition system includes a transceiver that is configured to transmit gesture information to a remote device.
15. The computing device as described in claim 13 , wherein computing device is configured to receive the verbalization through an audio communication system and the determination of the meaning about the in-the-air gesture that is relevant to the question is responsive to and based on the in-the-air gesture being recognized commensurate with receiving the verbalization through the audio communication system.
16. The computing device as described in claim 13 , wherein the computing device further comprises a display, and wherein the search manager is configured to cause the computing device to provide the search results of the performed search via the display.
17. The computing device as described in claim 13 , wherein:
the computing device further comprises an image sensor;
the determination of the meaning of the in-the-air gesture of the multiple potential meanings determines the in-the-air gesture is indicative of a pointing gesture; and
the search manager is further configured to cause the computing device to:
obtain image data from the image sensor; and
analyze the image data to determine one or more objects, locations, or activities, and wherein the search is refined based on the one or more objects, locations, or activities sufficient to provide search results responsive to the question and relevant to the one or more objects, locations, or activities.
18. The computing device as described in claim 13 , wherein the search is refined sufficient to provide search results that vary in at least one of relevancy, quality, verbosity, temporal proximity, or geographic proximity.
19. The computing device as described in claim 13 , wherein the computing device comprises a portable wireless communication device being a smartphone or wearable device.
20. The computing device as described in claim 13 , wherein the in-the-air gesture comprises at least one of a movement of a part of a body of the user or an object associated with the user.