IP Library Granted Patent US 11,176,576
Granted Patent B2
US 11,176,576 · App. 16/439,073 · Granted Nov 16, 2021

Mobile visual locator

Inventors: Gajaruban Kandavanam (Chicago, IL); Sarika Oak (Chicago, IL); Gloria Ye (Chicago, IL); Chunjun Chen (Chicago, IL)
Assignee: GROUPON, INC.
G06Q30/0261G06K9/00664G06K9/325G06K9/3208G06K9/6215G06Q30/0259G06Q30/0267
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,176,576
App. No.
16/439,073
Granted
Nov 16, 2021
Kind
B2
Abstract

Techniques for providing remote messages to mobile devices based on image data and other sensor data are discussed herein. Some embodiments may include one or more servers configured to: receive, from a consumer device via a network, location data indicating a consumer device location of a consumer device; receive, from the consumer device via the network, image data captured by a camera of the consumer device; receive, from the consumer device via the network, orientation data defining an orientation of the camera when the image data was captured, wherein the orientation data is captured by an accelerometer of the consumer device; attempt to extract a merchant identifier from the image based on programmatically processing the image data; determine one or more merchants based on a fuzzy search of available ones of the location data, the merchant identifier, and the orientation data.

Claims (58)

1. An apparatus configured to update a machine learning model training data set, the apparatus comprising at least a processor, and a memory associated with the processor having computer coded instructions therein, with the computer coded instructions configured to, when executed by the processor, cause the apparatus to:

receive, from a consumer device via a network, location data indicating a consumer device location of the consumer device;

receive, from the consumer device via the network, image data captured by a camera of the consumer device;

extract a merchant identifier from the image data based on programmatically processing the image data;

determine a candidate merchant set based on a fuzzy search of the location data and the merchant identifier by comparing first orientation data defining an orientation of the camera when the image data was captured with second orientation data defining an orientation of second image data captured by one or more other cameras;

send an indication of the candidate merchant set to the consumer device via the network;

receive a selection of a first merchant profile of the candidate merchant set from the consumer device;

determine the first merchant as a selected merchant;

subsequent to determining the first merchant as the selected merchant, associate the first merchant profile with the location data and the merchant identifier for one or more subsequent fuzzy searches by:

comparing the merchant identifier with actual merchant information associated with the first merchant profile,

based on the comparing, determining that the merchant identifier is different from actual merchant information associated with the first merchant profile,

associating the merchant identifier with the first merchant profile,

associating the location data with the first merchant profile, and

programmatically updating, based at least in part on the merchant identifier, the machine learning model training data set.

2. The apparatus of claim 1 , wherein the computer coded instructions are further configured to, when executed by the processor, cause the apparatus to utilize the machine learning model training data set for improving programmatic location determination accuracy.

3. The apparatus of claim 1 , wherein the merchant identifier is one or more of a merchant sign, a merchant logo, a merchant menu, a merchant interior structure, or a merchant storefront structure.

4. The apparatus of claim 1 , wherein the computer coded instructions are further configured to, when executed by the processor, cause the apparatus to extract the merchant identifier from the image data based on programmatically processing the image data by:

extracting an object from the image data; and

determining the merchant identifier based on the object.

5. The apparatus of claim 1 , wherein the computer coded instructions are further configured to, when executed by the processor, cause the apparatus to:

subsequent to determining the first merchant as the selected merchant:

determine, based on accessing promotion data associated with the selected merchant, whether the selected merchant is associated with an available promotion; and

in response to determining that the selected merchant fails to be associated with an available promotion:

determine merchant parameters associated with the selected merchant;

determine, based on the merchant parameters and promotion parameters defining candidate promotions, one or more promotions associated with merchants that are different from the selected merchant and related to the selected merchant; and

generate an electronic marketing communication including an indication of at least one of the one or more promotions.

6. The apparatus of claim 1 , wherein the computer coded instructions are further configured to, when executed by the processor, cause the apparatus to determine the candidate merchant set based on the fuzzy search of the location data and the merchant identifier by:

accessing merchant identifier data defining merchant identifiers; and

comparing the merchant identifiers defined by the merchant identifier data with the merchant identifier extracted from the image data.

7. A method for updating a machine learning model training data set, the method comprising:

receiving, by one or more servers and from a consumer device via a network, location data indicating a consumer device location of the consumer device;

receiving, from the consumer device via the network, image data captured by a camera of the consumer device;

extracting a merchant identifier from the image data based on programmatically processing the image data;

determining a candidate merchant set based on a fuzzy search of the location data and the merchant identifier by comparing first orientation data defining an orientation of the camera when the image data was captured with second orientation data defining an orientation of second image data captured by one or more other cameras;

sending an indication of the candidate merchant set to the consumer device via the network;

receiving a selection of a first merchant of the candidate merchant set from the consumer device;

determining the first merchant as a selected merchant;

subsequent to determining the first merchant as the selected merchant, associating the first merchant profile with the location data and the merchant identifier for one or more subsequent fuzzy searches by:

comparing the merchant identifier with an actual merchant name associated with the first merchant profile,

based on the comparing, determining that the merchant identifier is different from the actual merchant name associated with the first merchant profile,

associating the merchant identifier with the first merchant profile,

associating the location data with the first merchant profile, and

programmatically updating, based at least in part on the merchant identifier, the machine learning model training data set.

8. The method of claim 7 , further comprising utilizing the machine learning model training data set for improving programmatic location determination accuracy.

9. The method of claim 7 , wherein the merchant identifier is one or more of a merchant sign, a merchant logo, a merchant menu, a merchant interior structure, or a merchant storefront structure.

10. The method of claim 7 , wherein the extracting the merchant identifier from the image based on programmatically processing the image data comprises:

extracting an object from the image data; and

determining the merchant identifier based on the object.

11. The method of claim 7 , further comprising:

subsequent to determining the first merchant as the selected merchant:

determining, based on accessing promotion data associated with the selected merchant, whether the selected merchant is associated with an available promotion; and

in response to determining that the selected merchant fails to be associated with an available promotion:

determining merchant parameters associated with the selected merchant;

determining, based on the merchant parameters and promotion parameters defining candidate promotions, one or more promotions associated with merchants that are different from the selected merchant and related to the selected merchant; and

generating an electronic marketing communication including an indication of at least one of the one or more promotions.

12. The method of claim 7 , wherein the one or more servers configured to determine the candidate merchant set based on the fuzzy search of the location data and the merchant identifier by:

accessing merchant identifier data defining merchant identifiers; and

comparing the merchant identifiers defined by the merchant identifier data with the merchant identifier extracted from the image.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 12, 2024
From: GROUPON, INC.
To: BYTEDANCE INC.
Reel/Frame 068833/0811 →
RELEASE OF SECURITY INTEREST Recorded Feb 26, 2024
From: JPMORGAN CHASE BANK, N.A.
To: GROUPON, INC.; LIVINGSOCIAL, LLC (F/K/A LIVINGSOCIAL, INC.)
Reel/Frame 066676/0001 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN INTELLECTUAL PROPERTY RIGHTS Recorded Feb 26, 2024
From: JPMORGAN CHASE BANK, N.A.
To: GROUPON, INC.; LIVINGSOCIAL, LLC (F/K/A LIVINGSOCIAL, INC.)
Reel/Frame 066676/0251 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 14, 2021
From: KANDAVANAM, GAJARUBAN; OAK, SARIKA; YE, GLORIA; CHEN, CHUJUN
To: GROUPON, INC.
Reel/Frame 055912/0381 →
SECURITY INTEREST Recorded Jul 23, 2020
From: GROUPON, INC.; LIVINGSOCIAL, LLC
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 053294/0495 →