IP Library Granted Patent US 11,651,393
Granted Patent B2
US 11,651,393 · App. 17/451,250 · Granted May 16, 2023

Mobile visual locator

Inventors: Gajaruban Kandavanam (Chicago, IL); Sarika Oak (Chicago, IL); Gloria Ye (Chicago, IL); Chunjun Chen (Chicago, IL)
Assignee: GROUPON, INC.
G06Q30/0261G06K9/6215G06Q30/0259G06Q30/0267G06V10/242G06V20/10G06V20/62
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,651,393
App. No.
17/451,250
Granted
May 16, 2023
Kind
B2
Abstract

Techniques for providing remote messages to mobile devices based on image data and other sensor data are discussed herein. Some embodiments may include one or more servers configured to: receive, from a consumer device via a network, location data indicating a consumer device location of a consumer device; receive, from the consumer device via the network, image data captured by a camera of the consumer device; receive, from the consumer device via the network, orientation data defining an orientation of the camera when the image data was captured, wherein the orientation data is captured by an accelerometer of the consumer device; attempt to extract a merchant identifier from the image based on programmatically processing the image data; determine one or more merchants based on a fuzzy search of available ones of the location data, the merchant identifier, and the orientation data.

Claims (68)

1. An apparatus comprising at least a processor, and a memory associated with the processor having computer coded instructions therein, with the computer coded instructions configured to, when executed by the processor, cause the apparatus to:

retrieve image data captured by a camera of a consumer device;

extract a merchant identifier from the image data;

determine a candidate merchant set based on a fuzzy search of location data and second image data associated with the image data using the merchant identifier by comparing first orientation data defining a first orientation of the camera in relation to a reference normal orientation when the image data was captured with second orientation data defining a second orientation in relation to the reference normal orientation of second image data captured by one or more other cameras;

determine if the candidate merchant subset includes multiple merchants;

in response to determining that the candidate merchant subset includes multiple merchants:

send an indication of the multiple merchants to the consumer device via a network;

receive a selection of a first merchant profile of the multiple merchants from the consumer device indicating that one of the multiple merchants is a selected merchant; and

associate the first merchant profile with the location data and the merchant identifier in a merchant database for one or more subsequent fuzzy searches.

2. The apparatus of claim 1 , wherein the computer coded instructions are further configured to, when executed by the processor, cause the apparatus to:

compare the merchant identifier with an actual merchant name associated with the first merchant profile;

in response to determining that the merchant identifier is different from the actual merchant name associated with the first merchant profile:

associate the merchant identifier with the first merchant profile, and

associate the location data with the first merchant profile.

3. The apparatus of claim 2 , wherein the computer coded instructions are further configured to, when executed by the processor, cause the apparatus to:

associate the image data with the first merchant profile; and

provide an electronic marketing communication associated with the selected merchant to the consumer device.

4. The apparatus of claim 1 , wherein the merchant identifier is one or more of a merchant sign, a merchant logo, a merchant menu, a merchant interior structure, or a merchant storefront structure.

5. The apparatus of claim 1 , wherein the computer coded instructions are further configured to, when executed by the processor, cause the apparatus to extract the merchant identifier from the image data based on programmatically processing the image data by:

using an optical character recognition technique to extract object information from the image data; and

determining the merchant identifier based on the object information.

6. The apparatus of claim 1 , wherein the computer coded instructions are further configured to, when executed by the processor, cause the apparatus to:

determine, based on accessing promotion data associated with the selected merchant, whether the selected merchant is associated with an available promotion; and

in response to determining that the selected merchant fails to be associated with an available promotion:

determine merchant parameters associated with the selected merchant,

determine, based on the merchant parameters and promotion parameters, one or more promotions associated with merchants that are different from the selected merchant and related to the selected merchant, and

generate an electronic marketing communication including an indication of at least one of the one or more promotions.

7. The apparatus of claim 6 , wherein the candidate merchant set is determined based at least in part by comparing first orientation data associated with the image data with second orientation associated with the second image data.

8. The apparatus of claim 7 , wherein the second image data is captured by one or more other cameras.

9. The apparatus of claim 1 , wherein the first image data and the second image data comprise motion sensor data.

10. The apparatus of claim 1 , wherein the fuzzy search comprises:

determining an edit distance for each candidate merchant of the candidate merchant set based on the location data, the merchant identifier, the image data and the second image data;

assigning an edit score to each candidate merchant of the candidate merchant set based on the edit distance; and

comparing the edit score of each candidate merchant of the candidate merchant set to a pre-determined threshold to identify the candidate merchant subset.

11. A computer-implemented method comprising:

retrieving, by one or more processors, image data captured by a camera of a consumer device;

extracting, by the one or more processors, a merchant identifier from the image data;

determining, by the one or more processors, a candidate merchant set based on a fuzzy search of location data and second image data associated with the image data using the merchant identifier by comparing first orientation data defining a first orientation of the camera in relation to a reference normal orientation when the image data was captured with second orientation data defining a second orientation in relation to the reference normal orientation of second image data captured by one or more other cameras;

determining, by the one or more processors, if the candidate merchant subset includes multiple merchants;

in response to determining that the candidate merchant subset includes multiple merchants:

sending, by the one or more processors, an indication of the multiple merchants to the consumer device via a network;

receiving, by the one or more processors, a selection of a first merchant profile of the multiple merchants from the consumer device indicating that one of the multiple merchants is a selected merchant; and

associating by the one or more processors, the first merchant profile with the location data and the merchant identifier in a merchant database for one or more subsequent fuzzy searches.

12. The computer-implemented method of claim 11 , further comprising:

comparing, by the one or more processors, the merchant identifier with an actual merchant name associated with the first merchant profile;

in response to determining that the merchant identifier is different from the actual merchant name associated with the first merchant profile:

associating, by the one or more processors, the merchant identifier with the first merchant profile, and

associating, by the one or more processors, the location data with the first merchant profile.

13. The computer-implemented method of claim 11 , further comprising:

associating, by the one or more processors, the image data with the first merchant profile; and

providing, by the one or more processors, an electronic marketing communication associated with the selected merchant to the consumer device.

14. The computer-implemented method of claim 11 , wherein the merchant identifier is one or more of a merchant sign, a merchant logo, a merchant menu, a merchant interior structure, or a merchant storefront structure.

15. The computer-implemented method of claim 11 , wherein extracting the merchant identifier from the image data further comprises:

applying, by the one or more processors, an optical character recognition technique to extract object information from the image data; and

determining, by the one or more processors, the merchant identifier based on the object information.

16. The computer-implemented method of claim 11 , further comprising:

determining, by the one or more processors, based on accessing promotion data associated with the selected merchant, whether the selected merchant is associated with an available promotion; and

in response to determining that the selected merchant fails to be associated with an available promotion:

determining, by the one or more processors, merchant parameters associated with the selected merchant,

determining, by the one or more processors, based on the merchant parameters and promotion parameters, one or more promotions associated with merchants that are different from the selected merchant and related to the selected merchant, and

generating, by the one or more processors, an electronic marketing communication including an indication of at least one of the one or more promotions.

17. The computer-implemented method of claim 16 , wherein the candidate merchant set is determined based at least in part by comparing first orientation data associated with the image data with second orientation associated with the second image data.

18. The computer-implemented method of claim 17 , wherein the second image data is captured by one or more other cameras.

19. The computer-implemented method of claim 11 , wherein the first image data and the second image data comprise motion sensor data.

20. The computer-implemented method of claim 11 , further comprising:

determining, by the one or more processors, an edit distance for each candidate merchant of the candidate merchant set based on the location data, the merchant identifier, the image data and the second image data;

assigning, by the one or more processors, an edit score to each candidate merchant of the candidate merchant set based on the edit distance; and

comparing, by the one or more processors, the edit score of each candidate merchant of the candidate merchant set to a pre-determined threshold to identify the candidate merchant subset.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 12, 2024
From: GROUPON, INC.
To: BYTEDANCE INC.
Reel/Frame 068833/0811 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 28, 2022
From: KANDAVANAM, GAJARUBAN; OAK, SARIKA; YE, GLORIA; CHEN, CHUJUN
To: GROUPON, INC.
Reel/Frame 059767/0526 →