IP Library Granted Patent US 10,977,671
Granted Patent B2
US 10,977,671 · App. 16/525,391 · Granted Apr 13, 2021

Item level 3D localization and imaging using radio frequency waves

Inventors: Vivek Jayan (Redwood City, CA); Andrew McElroy (San Jose, CA); Kumar Ujjwal (Palo Alto, CA)
Assignee: PUNCHH INC.
G06Q30/0201G01S7/412G01S13/89G06F16/2379G06F16/9535H04L67/26H04L67/306
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,977,671
App. No.
16/525,391
Granted
Apr 13, 2021
Kind
B2
Abstract

A system and a method are disclosed for tracking user activity with products in an environment. In an embodiment, a processor detects that a user has entered an environment, and responsively uniquely identifies the user. The processor determines that the user interacts with a product of a plurality of products within the environment, and responsively updates a profile of the user with indicia of the product. The processor determines whether the user is attempting to exit the environment, and, in response to determining that the user is attempting to exit the environment, prompts the user to confirm that the user intends to remove the product from the environment based on the updated profile. The processor receives input from the user confirming that the user intends to remove the product from the environment, and responsively further updates the profile based on the input.

Claims (108)

1. A method comprising:

detecting that a user has entered an environment, by:

receiving wireless network signals detected by a receiver array, the receiver array continuously monitoring for wireless network signals;

determining a measure of absorption of a given waveform of the wireless network signals; and

determining, based on inputting the measure of absorption into a data structure and receiving an output from the data structure, that the given waveform represents a human being;

in response to detecting that the user has entered the environment, uniquely identifying the user based on signal patterns of the wireless network signals corresponding to at least two uniquely identifiable physical features of the user;

determining whether the user interacts with a product of a plurality of products within the environment based on absorption characteristics of the product relative to the wireless network signals;

in response to determining that the user interacts with the product, updating a profile of the user with indicia of the product;

determining whether the user is attempting to exit the environment;

in response to determining that the user is attempting to exit the environment, prompting the user to confirm that the user intends to remove the product from the environment based on the updated profile;

receiving input from the user confirming that the user intends to remove the product from the environment; and

further updating the profile based on the input.

2. The method of claim 1 , wherein uniquely identifying the user comprises:

detecting signal patterns corresponding to features of the user;

generating a signature based on the signal patterns;

determining whether the user is known by determining whether the unique signature matches a stored signature in an entry of a database that corresponds signatures to profiles of users; and

responsive to determining that the user is known, attributing an identifier retrieved from the profile to the user.

3. The method of claim 2 , wherein prompting the user to confirm that the user intends to remove the product from the environment based on the updated profile comprises:

determining an address of a mobile device of the user from the profile; and

pushing a notification to the mobile device, using the address, the notification comprising a selectable option to confirm that the user intends to remove the product from the environment.

4. The method of claim 2 , further comprising:

responsive to determining that the user is not known:

adding the unique signature to an entry of the database;

associating an identifier to the entry of the database; and

uniquely identifying the user using the identifier.

5. The method of claim 1 , wherein updating the profile with the indicia of the product comprises:

identifying patterns, by the product, of signals within the environment that are indicative of characteristics of objects encountered by the signals;

determining whether the patterns correspond to known patterns in one or more entries of a database that maps products to known patterns, the database entries indicating at least one of absorption characteristics, reflectivity characteristics, and refraction characteristics; and

responsive to determining that the patterns match an entry of the database, identifying the indicia of the product based on the entry.

6. The method of claim 5 , further comprising:

determining that the patterns match two or more entries of the database;

responsive to determining that the patterns match two or more entries of the database, querying a sensor for a weight of the product;

determining whether one of the two or more entries of the database indicates a weight matching the weight of the product; and

responsive to determining that one of the two or more entries of the database indicates a weight matching a weight of the product, identifying the indicia of the product based on the one of the two or more entries.

7. The method of claim 5 , wherein the patterns are absorption patterns, and wherein identifying the absorption patterns, by the product, of the signals within the environment comprises:

detecting a radio-opaque marker embedded on the product based on locations of the product having a high signal absorption rate, and wherein determining whether the absorption patterns correspond to known patterns in an entry of a database that maps products to known absorption patterns comprises:

determining whether an entry of the database maps a product to the detected radio-opaque marker.

8. A non-transitory computer-readable storage medium comprising stored instructions executable by at least one processor, the instructions when executed causing the at least one processor to:

detect that a user has entered an environment, by:

receiving wireless network signals detected by a receiver array, the receiver array continuously monitoring for wireless network signals;

determining a measure of absorption of a given waveform of the wireless network signals; and

determining, based on inputting the measure of absorption into a data structure and receiving an output from the data structure, that the given waveform represents a human being;

uniquely identify the user based on signal patterns of the wireless network signals corresponding to at least two uniquely identifiable physical features of the user, responsive to detection that the user has entered the environment;

determine whether the user interacts with a product of a plurality of products within the environment based on absorption characteristics of the product relative to the wireless network signals;

update a profile of the user with indicia of the product responsive to a determination that the user interacts with the product;

determine whether the user is attempting to exit the environment;

prompt the user to confirm that the user intends to remove the product from the environment based on the updated profile responsive to a determination that the user is attempting to exit the environment;

receive input from the user confirming that the user intends to remove the product from the environment; and

update the profile based on the received input.

9. The non-transitory computer-readable storage medium of claim 8 , wherein the instructions to uniquely identify the user comprise further instructions that when executed cause the processor to:

detect signal patterns corresponding to features of the user;

generate a signature based on the signal patterns;

determine whether the user is known by determining whether the unique signature matches a stored signature in an entry of a database that corresponds signatures to profiles of users; and

attribute an identifier retrieved from the profile to the user responsive to a determination that the user is known.

10. The non-transitory computer-readable storage medium of claim 9 , wherein the instructions to prompt the user to confirm that the user intends to remove the product from the environment based on the updated profile comprise further instructions that when executed cause the processor to:

determine an address of a mobile device of the user from the profile; and

push a notification to the mobile device, using the address, the notification comprising a selectable option to confirm that the user intends to remove the product from the environment.

11. The non-transitory computer-readable storage medium of claim 9 , the instructions further comprising instructions that when executed cause the processor to, responsive to a determination that the user is not known:

add the unique signature to an entry of the database;

associate an identifier to the entry of the database; and

uniquely identify the user using the identifier.

12. The non-transitory computer-readable storage medium of claim 8 , wherein the instructions to update the profile with the indicia of the product further comprise instructions that when executed cause the processor to:

identify patterns, by the product, of signals within the environment that are indicative of characteristics of objects encountered by the signals;

determine whether the patterns correspond to known patterns in one or more entries of a database that maps products to known absorption patterns; and

identify the indicia of the product based on the entry responsive to a determination that the patterns match an entry of the database.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the instructions further comprise instructions that when executed cause the processor to:

determine that the patterns match two or more entries of the database;

query a sensor for a weight of the product responsive to a determination that the patterns match two or more entries of the database;

determine whether one of the two or more entries of the database indicates a weight matching the weight of the product; and

identify the indicia of the product based on the one of the two or more entries responsive to a determination that one of the two or more entries of the database indicates a weight matching a weight of the product.

14. The non-transitory computer-readable storage medium of claim 12 , wherein the patterns are absorption patterns, and wherein the instructions to identify the absorption patterns, by the product, of the signals within the environment further comprise instructions that when executed causes the processor to:

detect a radio-opaque marker embedded on the product based on locations of the product having a high signal absorption rate, and wherein the instructions to determine whether the absorption patterns correspond to known patterns in an entry of a database that maps products to known absorption patterns comprise instructions to:

determine whether an entry of the database maps a product to the detected radio-opaque marker.

15. A system comprising:

a processor configured to:

detect, using a sensor, that a user has entered an environment, by:

receiving wireless network signals detected by a receiver array, the receiver array continuously monitoring for wireless network signals;

determining a measure of absorption of a given waveform of the wireless network signals; and

determining, based on inputting the measure of absorption into a data structure and receiving an output from the data structure, that the given waveform represents a human being;

in response to detecting that the user has entered the environment, uniquely identify the user based on signal patterns of the wireless network signals corresponding to at least two uniquely identifiable physical features of the user using a signature generation module;

determine, using an inventory module, whether the user interacts with a product of a plurality of products within the environment based on absorption characteristics of the product relative to the wireless network signals;

in response to determining that the user interacts with the product, update, using a user profile module, a profile of the user with indicia of the product;

determine, using a location tracker module, whether the user is attempting to exit the environment;

in response to determining that the user is attempting to exit the environment, prompt the user to confirm that the user intends to remove the product from the environment based on the updated profile;

receive input from the user confirming that the user intends to remove the product from the environment; and

further update the profile, using the user profile module, based on the input.

16. The system of claim 15 , wherein the processor is further configured, when being caused to uniquely identify the user, to:

detect signal patterns corresponding to features of the user;

generate a signature based on the signal patterns;

determine whether the user is known by determining whether the unique signature matches a stored signature in an entry of a database that corresponds signatures to profiles of users; and

responsive to determining that the user is known, attribute an identifier retrieved from the profile to the user.

17. The system of claim 16 , wherein the processor is further configured, when being caused to prompt the user to confirm that the user intends to remove the product from the environment based on the updated profile, to:

determine an address of a mobile device of the user from the profile; and

push a notification to the mobile device, using the address, the notification comprising a selectable option to confirm that the user intends to remove the product from the environment.

18. The system of claim 16 , wherein the processor is further configured to:

responsive to determining that the user is not known:

add the unique signature to an entry of the database;

associate an identifier to the entry of the database; and

uniquely identify the user using the identifier.

19. The system of claim 15 , wherein the processor is further configured, when being caused to update the profile with the indicia of the product, to:

identify patterns, by the product, of signals within the environment that are indicative of characteristics of objects encountered by the signals;

determine whether the patterns correspond to known patterns in one or more entries of a database that maps products to known patterns, the database entries indicating at least one of absorption characteristics, reflectivity characteristics, and refraction characteristics; and

responsive to determining that the patterns match an entry of the database, identify the indicia of the product based on the entry.

20. The system of claim 19 , wherein the processor is further configured to:

determine that the patterns match two or more entries of the database;

responsive to determining that the patterns match two or more entries of the database, query a sensor for a weight of the product;

determine whether one of the two or more entries of the database indicates a weight matching the weight of the product; and

responsive to determining that one of the two or more entries of the database indicates a weight matching a weight of the product, identify the indicia of the product based on the one of the two or more entries.

Assignments (6)
RELEASE OF PATENT SECURITY AGREEMENT RECORDED AT REEL 067924/FRAME 0663 Recorded Jan 31, 2025
From: BLUE OWL CAPITAL CORPORATION
To: PAR TECHNOLOGY CORPORATION; PARTECH, INC.; PUNCHH INC.; STUZO, LLC
Reel/Frame 070078/0575 →
SECURITY INTEREST Recorded Jul 8, 2024
From: PAR TECHNOLOGY CORPORATION; PARTECH, INC.; PUNCHH INC.; STUZO, LLC
To: BLUE OWL CAPITAL CORPORATION, AS COLLATERAL AGENT
Reel/Frame 067924/0663 →
RELEASE OF SECURITY INTEREST Recorded Sep 22, 2021
From: OWL ROCK FIRST LIEN MASTER FUND, L.P.
To: PAR TECHNOLOGY CORPORATION; PARTECH, INC.; PUNCHH INC.
Reel/Frame 057564/0591 →
SECURITY INTEREST Recorded Apr 8, 2021
From: PAR TECHNOLOGY CORPORATION; PARTECH, INC.; PUNCHH INC.
To: OWL ROCK FIRST LIEN MASTER FUND, L.P., AS COLLATERAL AGENT
Reel/Frame 055870/0710 →
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY'S NAME TO REMOVE A COMMA PREVIOUSLY RECORDED ON REEL 050066 FRAME 0936. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT DOCUMENT. Recorded May 6, 2020
From: JAYAN, VIVEK; MCELROY, ANDREW; UJJWAL, KUMAR
To: PUNCHH INC.
Reel/Frame 052595/0125 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 15, 2019
From: JAYAN, VIVEK; MCELROY, ANDREW; UJJWAL, KUMAR
To: PUNCHH, INC,
Reel/Frame 050066/0936 →
Continuity (2)
Provisional Application 62748299 · Oct 19, 2018
Related Publication 20200126097A1 · Apr 23, 2020
Cited By (2)
US 12,400,182 US 12,456,137