IP Library Granted Patent US 12,268,313
Granted Patent B2
US 12,268,313 · App. 18/237,245 · Granted Apr 8, 2025

Image display device, image display system, image display method, and program

Inventors: Jun Kobayashi (Tokyo, JP); Shigetsu Saito (Tokyo, JP)
Assignee: NEC CORPORATION
A47F5/0807G06Q30/02G06Q30/06G06V20/52G06V40/103A47F3/00G09G3/20G09G2354/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,268,313
App. No.
18/237,245
Granted
Apr 8, 2025
Kind
B2
Abstract

An image display device includes an approach situation information acquirer that acquires approach situation information indicating approach of at least part of a customer's body to a shelf on which a plurality of items are displayed and indicating, when there is the approach, a position of the approach on the shelf at each time; an associator that associates the approach situation information with one item of the plurality of items according to the position indicated by the approach situation information; an index value calculator that calculates an index value relating to the customer's behavior on the basis of information indicating the item associated with the approach situation information; and a display unit that displays the index value calculated by the index value calculator together with an image of the item on the basis of the association made by the associator.

Claims (45)

1. An information processing device comprising:

a sensor measuring a distance between a customer's hand and the sensor;

at least one memory configured to store instructions; and

at least one processor configured to execute the instructions to:

receive sensing data representing movement of the customer's hand toward a shelf;

determine, based on the sensing data and the distance, from a plurality of layers included in the shelf where a plurality of products is displayed, a layer to which the customer reached out;

count a number of times the customer reached out to the products displayed on the determined layer; and

output the number of times.

2. The information processing device according to claim 1 , wherein the sensor is installed above the shelf.

3. The information processing device according to claim 1 , wherein the processor is further configured to execute the instructions to:

generate a shelf image including an image of the plurality of products displayed on the shelf and an index value based on the number of times, the index value being displayed at a position of the image of the products displayed on the determined layer; and

display the shelf image on a display.

4. The information processing device according to claim 3 , wherein the processor is further configured to execute the instructions to:

generate the shelf image including a highlighting of the products displayed on the determined layer.

5. The information processing device according to claim 3 , wherein the image indicates the index value with a predetermined color which is superimposed on the position of the image of the products displayed on the determined layer and selected from among a plurality of colors,

wherein the colors are preset to a heat map and the predetermined color is changeable according to the number of times.

6. The information processing device according to claim 5 , wherein the image includes a legend indicating an association between the number of times and the plurality of the colors.

7. The information processing device according to claim 3 , wherein the image indicates the index value with a predetermined color which is superimposed on the position of the image of the products displayed on the determined layer,

wherein an intensity of the predetermined color is changeable according to the number of times.

8. The information processing device according to claim 3 , wherein the image indicates the index value with a predetermined pattern which is superimposed on the position of the image of the products displayed on the determined layer and selected from among a plurality of patterns,

wherein the patterns are preset to a heat map and the predetermined pattern is changeable according to the number of times.

9. An information processing method, by at least one processor, comprising:

measuring, by a sensor, a distance between a customer's hand and the sensor;

receiving sensing data representing movement of the customer's hand toward a shelf;

determining, based on the sensing data and the distance, from a plurality of layers included in the shelf where a plurality of products is displayed, a layer to which the customer reached out;

counting a number of times the customer reached out to the products displayed on the determined layer; and

outputting the number of times.

10. The information processing method according to laim claim 9 , wherein the sensor is installed above the shelf.

11. The information processing method according to claim 9 , further comprising:

generating a shelf image including an image of the plurality of products displayed on the shelf and an index value based on the number of times, the index value being displayed at a position of the image of the products displayed on the determined layer; and

displaying the shelf image on a display.

12. The information processing method according to claim 11 , further comprising:

generating the shelf image including a highlighting of the products displayed on the determined layer.

13. The information processing method according to claim 11 , wherein the image indicates the index value with a predetermined color which is superimposed on the position of the image of the products displayed on the determined layer and selected from among a plurality of colors,

wherein the colors are preset to a heat map and the predetermined color is changeable according to the number of times.

14. The information processing method according to claim 11 , wherein the image indicates the index value with a predetermined color which is superimposed on the position of the image of the products displayed on the determined layer,

wherein an intensity of the predetermined color is changeable according to the number of times.

15. The information processing method according to claim 11 , wherein the image indicates the index value with a predetermined pattern which is superimposed on the position of the image of the products displayed on the determined layer and selected from among a plurality of patterns,

wherein the patterns are preset to a heat map and the predetermined pattern is changeable according to the number of times.

16. A non-transitory computer-readable storage medium having a program causing a computer to:

measuring, by a sensor, a distance between a customer's hand and the sensor;

receiving sensing data representing movement of the customer's hand toward a shelf;

determine, based on the sensing data and the distance, from a plurality of layers included in the shelf where a plurality of products is displayed, a layer to which the customer reached out;

count a number of times the customer reached out to the products displayed on the determined layer; and

output the number of times.

Priority Claims (1)
JP 2016-057537 · Mar 22, 2016 · national
Continuity (4)
Continuation 17325824 · May 20, 2021
Continuation 16854458 · Apr 21, 2020
Continuation 16085638
Related Publication 20230389725A1 · Dec 7, 2023
References Cited (40)
US 6283377B1 · Takuma · 2001 [cited by applicant]
US 8380558B1 · Sharma et al. · 2013 [cited by applicant]
US 10133933B1 · Fisher · 2018 [cited by applicant]
US 10387897B2 · Sakata et al. · 2019 [cited by applicant]
US 20020184082A1 · Nakano · 2002 [cited by applicant]
US 20090153328A1 · Otani et al. · 2009 [cited by applicant]
US 20140132728A1 · Verano et al. · 2014 [cited by applicant]
US 20140214547A1 · Signorelli · 2014 [cited by applicant]
US 20150193716A1 · Tanigawa et al. · 2015 [cited by applicant]
US 20150206188A1 · Tanigawa · 2015 [cited by applicant]
US 20150242863A1 · Sato et al. · 2015 [cited by applicant]
US 20160055566A1 · Otani · 2016 [cited by applicant]
US 20160063712A1 · Matsumoto · 2016 [cited by applicant]
US 20170255947A1 · Horikawa · 2017 [cited by examiner]
JP 2007114851A · 2007 [cited by applicant]
JP 2009048430A · 2009 [cited by applicant]
JP 2009151409A · 2009 [cited by applicant]
JP 4972491B2 · 2012 [cited by applicant]
JP 2014026350A · 2014 [cited by applicant]
JP 2014106628A · 2014 [cited by applicant]
JP 2015011649A · 2015 [cited by applicant]
JP 2015149064A · 2015 [cited by applicant]
JP 2015179390A · 2015 [cited by applicant]
JP 5866564B1 · 2016 [cited by applicant]
JP 5881022B1 · 2016 [cited by applicant]
WO 2010053194A1 · 2010 [cited by applicant]
WO 2014010143A1 · 2014 [cited by applicant]
WO 2014045816A1 · 2014 [cited by applicant]
WO 2015040661A1 · 2015 [cited by applicant]
U.S. Notice of Allowance for U.S. Appl. No. 18/204,145, mailed on Mar. 8, 2024. [cited by applicant]
U.S. Office Action for U.S. Appl. No. 18/237,240, mailed on Apr. 12, 2024. [cited by applicant]
Communication dated Dec. 18, 2018 issued by the Japanese Patent Office in counterpart application No. 2018-507208. [cited by applicant]
International Search Report of PCT/JP2017/009533 dated Apr. 18, 2017 [PCT/ISA/210]. [cited by applicant]
Communication dated Jan. 23, 2020, from the United States Patent and Trademark Office in U.S. Appl. No. 16/085,638. [cited by applicant]
Josh Constine, “Shopper Tracker's Kinect Hack Is Like Google Analytics for Retail Store Shelves”, TechCrunch, Dec. 5, 2011, Retrieved from the Internet: URL:https://techcrunch.com/2011/12/05/shopper-tracker-kinect/, (2 … [cited by applicant]
JP Office Action for JP Application No. 2021-112372, mailed on Aug. 2, 2022 with English Translation. [cited by applicant]
U.S. Office Action for U.S. Appl. No. 17/325,824 mailed on Oct. 13, 2022. [cited by applicant]
U.S. Office Action for U.S. Appl. No. 17/325,824 mailed on Feb. 24, 2023. [cited by applicant]
JP Office Action for JP Application No. 2024-046806 , mailed on Oct. 1, 2024 with English Translation. [cited by applicant]
JP Office Communication for JP Application No. 2024-046806, mailed on Jan. 21, 2025 with English Translation. [cited by applicant]