IP Library Granted Patent US 11,908,290
Granted Patent B2
US 11,908,290 · App. 17/323,031 · Granted Feb 20, 2024

Visual recognition and sensor fusion weight detection system and method

Inventors: Lin Gao (Syosset, NY); Shiyuan Yang (Hangzhou, CN)
Assignee: Maplebear Inc.
G07G1/14A47F9/048G01G19/415G01G21/22G06Q20/208G06V20/44G07G1/0072G06V20/52G06V20/68
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,908,290
App. No.
17/323,031
Granted
Feb 20, 2024
Kind
B2
Abstract

Disclosed are visual recognition and sensor fusion weight detection system and method. An example method includes: tracking, by a sensor system, objects and motions within a selected area of a store; activating, by the sensor system, a first computing device positioned in the selected area in response to detecting a presence of a customer within the selected area; identifying, by the sensor system, the customer and at least one item carried by the customer; transmitting, by the sensor system, identifying information of the customer and the at least one item to a computing server system via a communication network; measuring, by the first computing device, a weight of the at least one item; transmitting, by the first computing device, the weight to the computing server system via the communication network; and generating, by the computing server system, via the communication network, transaction information of the at least one item.

Claims (59)

1. A system deployed within a communication network, the system comprising:

a processor coupled to a non-transitory computer-readable storage medium and configured to control a plurality of modules to execute instructions for:

tracking objects and motions within a selected area of a store based on one or more sensors or cameras within the store;

in response to detecting a presence of a customer carrying at least one item within the selected area, activating a computing device positioned within the selected area to measure a weight of the at least one item;

weighing the at least one item using the computing device;

obtaining identifying information of the customer and the at least one item carried by the customer; and

transmitting the weight of the at least one item, the identifying information of the customer and the at least one item to the computing server system via the communication network.

2. The system of claim 1 , further comprising a plurality of cameras and sensors configured to track the objects and motions within the selected area of the store.

3. The system of claim 1 , wherein the non-transitory computer-readable storage medium is configured to store information relating to one or more characteristics of each item for sale in the store.

4. The system of claim 1 , wherein the processor is further configured to:

detect the presence of the customer in relation to a location of the computing device within the selected area; and

capture one or more images of the customer in response to detecting that the customer is within a selected distance of the computing device.

5. The system of claim 4 , wherein the processor is further configured to:

generate a unique identifier for the customer;

associate the one and more images of the customer with the unique identifier; and

continuously accumulate and store information relating to the customer on the non-transitory computer-readable storage medium at different locations and times in the store in connection with the unique identifier prior to a final check out by the customer.

6. The system of claim 1 , wherein the processor is further configured to:

detect characteristics of the customer and surrounding objects obtained inside and outside the selected area for identifying the customer.

7. The system of claim 6 , wherein the processor is further configured to:

apply a weight to each detected characteristic of the customer, wherein facial features of the customer are configured to carry a greater weight than that of a color and texture details of an upper body clothes of the customer; and

determine a threshold for the customer presented in the selected area in connection with a total weight of all detected characteristics of the customer.

8. A method, comprising:

tracking, by a system, objects and motions within a selected area of a store based on one or more sensors or cameras within the store;

in response to detecting a presence of a customer carrying at least one item within the selected area, activating, by the system, a computing device positioned within the selected area to measure a weight of the at least one item;

weighing the at least one item using the computing device

obtaining, by the system, identifying information of the customer and the at least one item carried by the customer; and

transmitting, by the system, the weight of the at least one item, the identifying information of the customer and the at least one item to the computing server system via the communication network.

9. The method of claim 8 , further comprising storing, by the system, information relating to one or more characteristics of each item for sale in the store.

10. The method of claim 8 , further comprising:

detecting, by the system, the presence of the customer in relation to a location of the computing device within the selected area; and

capturing, by a plurality of cameras and sensors of the system, one or more images of the customer in response to detecting that the customer is within a selected distance of the computing device.

11. The method of claim 10 , further comprising:

generating, by the system, a unique identifier for the customer;

associating, by the system, the one and more images of the customer with the unique identifier; and

continuously accumulating and storing information relating to the customer, by the system, at different locations and times in the store in connection with the unique identifier prior to a final check out by the customer.

12. The method of claim 8 , further comprising:

detecting, by the system, characteristics of the customer and surrounding objects obtained inside and outside the selected area for identifying the customer.

13. The method of claim 12 , further comprising:

applying, by the system, a weight to each detected characteristic of the customer, wherein facial features of the customer are configured to carry a greater weight than that of a color and texture details of an upper body clothes of the customer; and

determining, by the system, a threshold for the customer presented in the selected area in connection with a total weight of all detected characteristics of the customer.

14. A non-transitory computer readable medium storing computer executable instructions for a system deployed in a communication network, the instructions being configured for:

tracking objects and motions within a selected area of a store based on one or more sensors or cameras within the store;

in response to detecting a presence of a customer carrying at least one item within the selected area, activating a computing device positioned within the selected area to measure a weight of the at least one item;

weighing the at least one item using the computing device;

obtaining identifying information of the customer and the at least one item carried by the customer; and

transmitting the weight of the at least one item, the identifying information of the customer and the at least one item to the computing server system via the communication network.

15. The non-transitory computer readable medium of claim 14 , further comprising instructions for storing information relating to one or more characteristics of each item for sale in the store.

16. The non-transitory computer readable medium of claim 15 , further comprising instructions for:

detecting the presence of the customer in relation to a location of the computing device within the selected area; and

capturing, by a plurality of cameras and sensors of the system, one or more images of the customer in response to detecting that the customer is within a selected distance of the computing device.

17. The non-transitory computer readable medium of claim 16 , further comprising instructions for:

generating a unique identifier for the customer;

associating the one and more images of the customer with the unique identifier; and

continuously accumulating and storing information relating to the customer at different locations and times in the store in connection with the unique identifier prior to a final check out by the customer.

18. The non-transitory computer readable medium of claim 14 , further comprising instructions for:

detecting characteristics of the customer and surrounding objects obtained inside and outside the selected area for identifying the customer.

19. The non-transitory computer readable medium of claim 18 , further comprising instructions for:

applying a weight to each detected characteristic of the customer, wherein facial features of the customer are configured to carry a greater weight than that of a color and texture details of an upper body clothes of the customer; and

determining a threshold for the customer presented in the selected area in connection with a total weight of all detected characteristics of the customer.

Assignments (3)
MERGER AND CHANGE OF NAME Recorded Mar 17, 2022
From: SBOT TECHNOLOGIES INC.; CUCUMBER MERGER SUB II LLC
To: SBOT TECHNOLOGIES LLC
Reel/Frame 059299/0262 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 17, 2022
From: SBOT TECHNOLOGIES LLC
To: MAPLEBEAR INC. (DBA INSTACART)
Reel/Frame 059299/0271 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 18, 2021
From: GAO, LIN; YANG, SHIYUAN
To: SBOT TECHNOLOGIES, INC.
Reel/Frame 056270/0833 →
Continuity (2)
Continuation 16229336 · Dec 21, 2018
Related Publication 20210272423A1 · Sep 2, 2021