IP Library Granted Patent US 9,852,329
Granted Patent B2
US 9,852,329 · App. 14/807,909 · Granted Dec 26, 2017

Calculation of a characteristic of a hotspot in an event

Inventors: Hirofumi Nishikawa (Tokyo, JP); Tomohiro Shioya (Tokyo, JP); Kei Sugano (Tokyo, JP); Shoichiro Watanabe (Tokyo, JP)
Assignee: International Business Machines Corporation
G06K9/00335G06K9/00778G06K9/3233H04L67/04H04L67/18H04L67/22G06K9/00604
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,852,329
App. No.
14/807,909
Granted
Dec 26, 2017
Kind
B2
Abstract

A computer characterizes an area of interest in an event attended by an audience by receiving behavior information for an audience attending the event and event activity information for one or more areas in the event venue. The computer analyzes the audience behavior and event activity to determine the areas in the event venue to which the audience pays attention. The computer determines the objects in the areas in the event venue and determines one or more areas of interest based on the areas in the event venue to which the audience pays attention and the objects in the areas in the event venue to which the audience pays attention. The computer calculates a characteristic of the areas of interest in the event based on: the areas of interest, the objects in the areas in the event venue, and the analyzed audience behavior information related to the areas of interest.

Claims (92)

1. A method for characterizing an area of interest in an event in a venue attended by an audience, the method comprising:

receiving, by a computer from a first plurality of cameras in the venue, audience video data of the audience;

receiving, by the computer from a second plurality of cameras in the venue, venue video data of event activity information for one or more areas in the event venue;

determining, by the computer, one or more areas in the event venue to which the audience pays attention by analyzing the received audience video data to estimate a gaze of members of the audience;

identifying, by the computer, one or more objects in the one or more areas in the event venue by analyzing the received venue video data based on movement of the object and further based on metadata for the objects;

determining, by the computer, one or more objects of interest in the event based on the determined one or more areas in the event venue to which the audience pays attention and the one or more identified objects in the one or more areas in the event venue;

calculating, by the computer, a characteristic of the one or more objects of interest in the event based on

the audience video data related to the one or more areas of interest in the event; and

displaying, by the computer, the characteristic of the one or more objects of interest in the event.

2. The method according to claim 1 , wherein determining, by the computer, the one or more areas of interest in the event further comprises mapping, by the computer, the determined one or more objects in the one or more areas in the event venue to the determined one or more areas in the event venue to which the audience pays attention.

3. The method according to claim 1 , further comprising

receiving, by a computer from a plurality of microphones in the venue, audience audio data of the audience;

and wherein determining, by the computer, one or more areas in the event venue to which the audience pays attention further comprises

determining, by the computer, one or more areas in the event venue to which the audience pays attention by analyzing the received audience video data and the received audience audio data to estimate a gaze of members of the audience.

4. The method according to claim 1 , wherein determining, by the computer, one or more areas in the event venue to which the audience pays attention further comprises:

determining, by the computer, one or more areas in the event venue to which the audience pays attention by analyzing the received audience video data to estimate a gaze of members of the audience based at least on a location of one or more video cameras providing the video data of the audience attending the event, and a line of sight of the audience in the video data.

5. The method according to claim 3 , wherein processing, by the computer, received audio data of the audience attending the event to obtain information relating to the audience attending the event further comprises one or more of:

processing, by the computer, the received audio data using voice recognition processing; and

processing, by the computer, the received audio data using natural language processing.

6. The method according to claim 3 , wherein analyzing the received audience video data includes:

analyzing the received audience video data by using voice recognition processing and natural language processing to identify:

a proper noun included in an utterance of the audience;

a sense or feeling of the audience based on an utterance of the audience;

a spoken language of the audience based on an utterance of the audience;

a birthplace of the audience based on an utterance of the audience;

a gender of the audience based on an utterance of the audience;

an age group of the audience based on an utterance of the audience; and

a degree of mastery of the audience based on an utterance of the audience, wherein the degree of mastery of the audience includes a degree of knowledge or familiarity of the audience about the event or the area of interest.

7. The method according to claim 1 , wherein displaying, by the computer, the characteristic of the one or more objects of interest in the event further comprises:

preparing, by the computer, display data including:

the one or more objects in the event venue to which the audience attending the event pays attention;

a stationary state and a state of rapid change that are visually distinguished from each other; and

an audience behavior attribute related to the characteristic of the one or more objects of interest in the event.

8. The method according to claim 7 , wherein displaying, by the computer, the characteristic of the one or more objects of interest in the event further comprises:

displaying, by the computer, a display which includes:

a degree of audience attention for each of the one or more objects of interest in the event at a defined point of time;

a sequential change, over a defined time period, of the degree of audience attention for each of the one or more objects of interest in the event;

an audience behavior attribute value for each audience behavior attribute related to the characteristic of the one or more objects of interest in the event; and

a broadcasting source used by the audience to view an objects of interest in the event.

9. A computer program product for characterizing an area of interest in an event attended by an audience, the computer program product comprising:

one or more computer readable storage tangible media and program instructions stored on at least one of the one or more computer readable tangible storage media, the program instructions executable by a computer to cause the computer to perform a method comprising:

receiving, by a computer from a first plurality of cameras in the venue, audience video data of the audience;

receiving, by the computer from a second plurality of cameras in the venue, venue video data of event activity information for one or more areas in the event venue;

determining, by the computer, one or more areas in the event venue to which the audience pays attention;

identifying, by the computer, one or more objects in the one or more areas in the event venue by analyzing the received venue video data based on movement of the object and metadata for the objects;

determining, by the computer, one or more objects of interest in the event based on the determined one or more areas in the event venue to which the audience pays attention and the one or more identified objects in the one or more areas in the event venue;

calculating, by the computer, a characteristic of the one or more objects of interest in the event based on

the audience video data related to the one or more areas of interest; and

displaying, by the computer, the characteristic of the one or more objects of interest in the event.

10. The computer program product according to claim 9 , wherein determining, by the computer, the one or more areas of interest in the event further comprises mapping, by the computer, the determined one or more objects in the one or more areas in the event venue to the determined one or more areas in the event venue to which the audience pays attention.

11. The computer program product according to claim 9 , further comprising

receiving, by a computer from a plurality of microphones in the venue, audience audio data of the audience;

and wherein determining, by the computer, one or more areas in the event venue to which the audience pays attention further comprises

determining, by the computer, one or more areas in the event venue to which the audience pays attention by analyzing the received audience video data and the received audience audio data to estimate a gaze of members of the audience.

12. The computer program product according to claim 9 , wherein determining, by the computer, one or more areas in the event venue to which the audience pays attention further comprises:

determining, by the computer, one or more areas in the event venue to which the audience pays attention by analyzing the received audience video data to estimate a gaze of members of the audience based at least on a location of one or more video cameras providing the video data of the audience attending the event, and a line of sight of the audience in the video data.

13. The computer program product according to claim 11 , wherein processing, by the computer, received audio data of the audience attending the event to obtain information relating to the audience attending the event further comprises one or more of:

processing, by the computer, the received audio data using voice recognition processing; and

processing, by the computer, the received audio data using natural language processing.

14. The computer program product according to claim 11 , wherein analyzing the received audience video data includes:

analyzing the received audience video data by using voice recognition processing and natural language processing to identify:

a proper noun included in an utterance of the audience;

a sense or feeling of the audience based on an utterance of the audience;

a spoken language of the audience based on an utterance of the audience;

a birthplace of the audience based on an utterance of the audience;

a gender of the audience based on an utterance of the audience;

an age group of the audience based on an utterance of the audience; and

a degree of mastery of the audience based on an utterance of the audience, wherein the degree of mastery of the audience includes a degree of knowledge or familiarity of the audience about the event or the area of interest.

15. The computer program product according to claim 9 , wherein displaying, by the computer, the characteristic of the one or more objects of interest in the event further comprises:

preparing, by the computer, display data including:

the one or more objects in the event venue to which the audience attending the event pays attention;

a stationary state and a state of rapid change that are visually distinguished from each other; and

an audience behavior attribute related to the characteristic of the one or more objects of interest in the event.

16. The computer program product according to claim 15 , wherein displaying, by the computer, the characteristic of the one or more objects of interest in the event further comprises:

displaying, by the computer, a display which includes:

a degree of audience attention for each of the one or more objects of interest in the event at a defined point of time;

a sequential change, over a defined time period, of the degree of audience attention for each of the one or more objects of interest in the event;

an audience behavior attribute value for each audience behavior attribute related to the characteristic of the one or more objects of interest in the event; and

a broadcasting source used by the audience to view an objects of interest in the event.

17. A computer system for characterizing an area of interest in an event attended by an audience, the computer system comprising one or more processors, one or more computer readable tangible storage media, and program instructions stored on at least one of the one or more computer readable tangible storage media, which when executed cause at least one of the one or more processors to perform a method comprising:

receiving, by a computer from a first plurality of cameras in the venue, audience video data of the audience;

receiving, by the computer from a second plurality of cameras in the venue, venue video data of event activity information for one or more areas in the event venue;

determining, by the computer, one or more areas in the event venue to which the audience pays attention;

identifying, by the computer, one or more objects in the one or more areas in the event venue by analyzing the received venue video data based on movement of the object and metadata for the objects;

determining, by the computer, one or more objects of interest in the event based on the determined one or more areas in the event venue to which the audience pays attention and the one or more identified objects in the one or more areas in the event venue;

calculating, by the computer, a characteristic of the one or more objects of interest in the event based on

the audience video data related to the one or more areas of interest; and

displaying, by the computer, the characteristic of the one or more objects of interest in the event.

18. The computer system according to claim 17 , further comprising

receiving, by a computer from a plurality of microphones in the venue, audience audio data of the audience;

and wherein determining, by the computer, one or more areas in the event venue to which the audience pays attention further comprises

determining, by the computer, one or more areas in the event venue to which the audience pays attention by analyzing the received audience video data and the received audience audio data to estimate a gaze of members of the audience.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 15, 2021
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: DOORDASH, INC.
Reel/Frame 057826/0939 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 24, 2015
From: NISHIKAWA, HIROFUMI; SHIOYA, TOMOHIRO; SUGANO, KEI; WATANABE, SHOICHIRO
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 036176/0705 →
Continuity (1)
Related Publication 20170024609A1 · Jan 26, 2017