IP Library Granted Patent US 10,817,070
Granted Patent B2
US 10,817,070 · App. 16/252,477 · Granted Oct 27, 2020

RF-based micro-motion tracking for gesture tracking and recognition

Inventors: Jaime Lien (Mountain View, CA); Erik M. Olson (Mountain View, CA); Patrick M. Amihood (Mountain View, CA); Ivan Poupyrev (Sunnyvale, CA)
Assignee: Google LLC
G06F3/017G01S7/415G01S13/58G01S13/66G01S13/88G01S13/89G06F3/011
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,817,070
App. No.
16/252,477
Granted
Oct 27, 2020
Kind
B2
Abstract

This document describes techniques for radio frequency (RF) based micro-motion tracking. These techniques enable even millimeter-scale hand motions to be tracked. To do so, radar signals are used from radar systems that, with conventional techniques, would only permit resolutions of a centimeter or more.

Claims (52)

1. One or more non-transitory computer-readable storage media having instructions stored thereon that, responsive to execution by one or more computer processors, implement a micro-motion tracking module and a recognition module:

the micro-motion tracking module configured to:

receive a first radar signal representing reflections of a radar field off first and second points of a hand at a first time, the first and second points of the hand moving relative to one another within the radar field;

determine, based on energies of the first radar signal, velocities of the first and second points of the hand at the first time;

receive a second radar signal representing reflections of the radar field off the first and second points of the hand at a second time;

determine, based on energies of the second radar signal, velocities of the first and second points of the hand at the second time;

calculate, based on the velocities of the first and second points of the hand at the first and second times, a relative velocity between the first and second point of the hand; and

determine, based on the relative velocity between the first and second point of the hand and an elapsed time between the first and second times, a millimeter-scale change in displacement between the first and second points of the hand; and

the recognition module configured to:

determine a gesture based on the change in displacement gesture of between the first and second points of the hand; and

pass the gesture effective to control or alter a display, function, or capability of a device.

2. The one or more non-transitory computer-readable storage media of claim 1 , wherein the velocities of the first and second points of the hand at the first and second times are determined based on micro-Doppler centroids of the first and second radar signals.

3. The one or more non-transitory computer-readable storage media of claim 1 , wherein the energies of the first and second radar signals are functions of ranges to the first and second points of the hand at the first and second times.

4. The one or more non-transitory computer-readable storage media of claim 1 , wherein the first and second radar signals are received via a single antenna element.

5. The one or more non-transitory computer-readable storage media of claim 1 , wherein an extended Kalman filter is used in conjunction with Doppler centroids corresponding to the first and second points of the hand to allow for nonlinear phase unwrapping of the first and second radar signals.

6. A computer-implemented method comprising:

receiving a first radar signal representing reflections of a radar field off first and second points of a hand at a first time, the first and second points of the hand moving relative to one another within the radar field;

determining, based on energies of the first radar signal, velocities of the first and second points of the hand at the first time;

receiving a second radar signal representing reflections of the radar field off the first and second points of the hand at a second time;

determining, based on energies of the second radar signal, velocities of the first and second points of the hand at the second time;

calculating, based on the velocities of the first and second points of the hand at the first and second times, a relative velocity between the first and second points of the hand; and

determining, based on the relative velocity between the first and second points of the hand and an elapsed time between the first and second times, a millimeter-scale change in displacement between the first and second points of the hand.

7. The method of claim 6 , wherein the radar field is emitted by a single radar element and the first and second radar signals are received by a single antenna.

8. The method of claim 6 , wherein the energies of the first and second radar signals are functions of ranges to the first and second points of the hand at the first and second times.

9. The method of claim 6 , wherein the velocities of the first and second points of the hand at the first and second times are radial velocities determined through Doppler frequencies of the first and second radar signals.

10. The method of claim 9 , wherein the velocities of the first and second points of the hand at the first and second times are based on micro-Doppler centroids of the first and second radar signals.

11. The method of claim 6 , wherein the first and second radar signals comprise a superposition of energies corresponding to the first and second points of the hand.

12. The method of claim 6 , further comprising creating a range-Doppler-time data cube that contains data corresponding to the first and second radar signals and other received radar signals representing reflections of the radar field off the first and second points of the hand at other times, wherein the velocities of the first and second points of the hand at the first and second times are determined from portions of the range-Doppler-time data cube corresponding to the first and second times.

13. The method of claim 6 , wherein the radar field has a wavelength greater than the millimeter-scale.

14. The method of claim 6 , wherein the change in displacement is determined by integrating the relative velocity.

15. An apparatus comprising:

one or more computer processors;

a radar system comprising:

one or more radar-emitting elements configured to provide a radar field; and

one or more antenna elements configured to receive radar signals representing reflections of the radar field off two or more points of a hand that are moving relative to one another within the radar field; and

one or more computer-readable storage media having instructions stored thereon that, responsive to execution by the one or more computer processors, implement a micro-motion tracking module and a recognition module:

the micro-motion tracking module configured to:

cause the radar-emitting elements to provide the radar field;

receive a first radar signal from the antenna elements representing reflections of the radar field off the first and second points of the hand at a first time;

determine, based on energies of the first radar signal, velocities of the first and second points of the hand at the first time;

receive a second radar signal from the antenna elements representing reflections of the radar field off the first and second points of the hand at a second time;

determine, based on energies of the second radar signal, velocities of the first and second points of the hand at the second time;

calculate, based on the velocities of the first and second points of the hand at the first and second times, a relative velocity between the first and second point of the hand; and

determine, based on the relative velocity between the first and second point of the hand and an elapsed time between the first and second times, a millimeter-scale change in displacement between the first and second points of the hand; and

the recognition module configured to:

determine a gesture based on the change in displacement between the first and second points of the hand; and

pass the gesture effective to control or alter a display, function, or capability of a device.

16. The apparatus of claim 15 , wherein the velocities of the first and second points of the hand at the first and second times are determined based on micro-Doppler centroids of the first and second radar signals.

17. The apparatus of claim 15 , wherein the energies of the first and second radar signals are functions of ranges to the first and second points of the hand at the first and second times.

18. The apparatus of claim 15 , wherein the one or more antenna elements are a single antenna element.

19. The apparatus of claim 15 , wherein an extended Kalman filter is used in conjunction with Doppler centroids corresponding to the first and second points of the hand to allow for nonlinear phase unwrapping of the first and second radar signals.

20. The apparatus of claim 15 , wherein the apparatus comprises a mobile phone or tablet.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 18, 2019
From: OLSON, ERIK M.; AMIHOOD, PATRICK M.; POUPYREV, IVAN
To: GOOGLE INC.
Reel/Frame 048063/0331 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 18, 2019
From: LIEN, JAIME
To: THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY
Reel/Frame 048063/0345 →
CHANGE OF NAME Recorded Jan 18, 2019
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 048098/0509 →
Continuity (4)
Continuation 15142689 · Apr 29, 2016
Provisional Application 62167823 · May 28, 2015
Provisional Application 62155357 · Apr 30, 2015
Related Publication 20190155396A1 · May 23, 2019
Cited By (1)
US 12,340,028