IP Library Granted Patent US 10,311,303
Granted Patent B2
US 10,311,303 · App. 15/303,391 · Granted Jun 4, 2019

Information processing apparatus, information processing method, and program

Inventors: Makoto Murata (Tokyo, JP); Naoki Shibuya (Tokyo, JP); Junko Takabayashi (Tokyo, JP); Yuuji Takimoto (Kanagawa, JP); Koji Sato (Tokyo, JP)
Assignee: SONY CORPORATION
G06K9/00671G06F1/1686G06F3/002G06F3/011G06F3/0304G06F3/04895G06F16/24575G06F16/435G06F17/20G06K9/00342G06K9/00664G06K9/6293G06K9/72G06Q50/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,311,303
App. No.
15/303,391
Granted
Jun 4, 2019
Kind
B2
Abstract

There is provided an information processing apparatus for automatically generating information representing a context surrounding a user, the information processing apparatus including: a recognition processing unit configured to perform, on the basis of user environment information including at least any of location information representing a location where a user is present, image information relating to an environment surrounding a user, and audio information relating to the environment, an analysis process of at least any of the location information, the image information, and the audio information included in the user environment information, at a predetermined time interval, and to recognize a context surrounding the user, using the acquired result of analysis relating to the user environment; and a context candidate information generating unit configured to generate context candidate information representing a candidate of the context surrounding the user, the context candidate information including, at least, information representing the context surrounding the user and information representing the user's emotion in the context, using the result of context recognition performed by the recognition processing unit.

Claims (48)

1. An information processing apparatus comprising:

a processor configured to

receive an image of a user as an input,

perform a recognition process on the image of the user using a machine learning model,

output a classification and a score corresponding to the classification as a recognition result from the recognition process performed on the image of the user, and

generate context candidate information representing a context surrounding the user based on the recognition result and a recognized emotion of the user,

wherein the machine learning model is constructed from a plurality of images collected in advance.

2. The information processing apparatus according to claim 1 ,

wherein the outputs the recognition result each time the context surrounding the user changes.

3. The information processing apparatus according to claim 2 ,

wherein the recognized emotion of the user is generated using an emotion representation table preliminarily provided for each of the recognized contexts.

4. The information processing apparatus according to claim 2 ,

wherein the context candidate information includes image information of the received image relating to the context surrounding the user.

5. The information processing apparatus according to claim 1 , wherein the processor is further configured to

initiate display the recognition result in a predetermined area of a predetermined display screen.

6. The information processing apparatus according to claim 1 , wherein the processor is further configured to

perform the recognition process by recognizing the context by a time-series pattern recognition process based on time course of the recognition result, or a rule-based process based on predetermined conditional processing, using the recognition result.

7. The information processing apparatus according to claim 1 , wherein the processor is further configured to

update an expression representing the recognized emotion of the user included in an emotion representation table preliminarily provided for each of the recognized contexts, using at least any of a result of an analysis process relating to an environment surrounding the user, a remark or a sentence expression provided by the user, and an output from a biological sensor provided in the information processing apparatus.

8. The information processing apparatus according to claim 1 , wherein the processor is further configured to

cooperate with another information processing apparatus with which mutual communication is possible, and

improve precision of, or share, the generated context candidate information.

9. The information processing apparatus according to claim 1 , wherein the processor is further configured to

transmit a piece of information selected by the user from the generated context candidate information to an information posting server providing a social network service, as context information representing the context surrounding the user.

10. The information processing apparatus according to claim 1 ,

wherein the information processing apparatus comprises a personal digital assistant carried by the user, or a wearable terminal worn by the user.

11. An information processing method, executed via at least one processor, the method comprising:

receiving an image of a user as an input,

performing a recognition process on the image of the user by a machine learning model,

outputting a classification and a score corresponding to the classification as a recognition result from the recognition process performed on the image of the user, and

generating context candidate information representing a context surrounding the user based on the recognition result and a recognized emotion of the user,

wherein the machine learning model is constructed from a plurality of images collected in advance.

12. A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the compute to execute a method, the method comprising:

receiving an image of a user as an input,

performing a recognition process on the image of the user by a machine learning model,

outputting a classification and a score corresponding to the classification as a recognition result from the recognition process performed on the image of the user, and

generating context candidate information representing a context surrounding the user based on the recognition result and a recognized emotion of the user,

wherein the machine learning model is constructed from a plurality of images collected in advance.

13. The information processing apparatus according to claim 1 ,

wherein the classification includes at least one of a face, a landscape, a dish, or an object.

14. The information processing apparatus according to claim 13 ,

wherein, when the processor outputs the classification including at least one face, the processor further outputs at least one of a number of faces, coordinates, angles, presence or absence of smile, age, or race.

15. The information processing apparatus according to claim 2 ,

wherein, when the processor outputs the classification including at least one face, a face detection frame is displayed on the image of the at least one face in the image.

16. The information processing apparatus according to claim 1 ,

wherein the image received at the processor is sent from an outside terminal.

17. The information processing apparatus according to claim 1 ,

wherein the processor generates a sentence for the context candidate information.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 11, 2016
From: MURATA, MAKOTO; SHIBUYA, NAOKI; TAKABAYASHI, JUNKO; TAKIMOTO, YUUJI; SATO, KOJI
To: SONY CORPORATION
Reel/Frame 040314/0367 →
Priority Claims (1)
JP 2014-106276 · May 22, 2014 · national
Continuity (1)
Related Publication 20170032186A1 · Feb 2, 2017