IP Library Granted Patent US 11,908,055
Granted Patent B2
US 11,908,055 · App. 17/434,970 · Granted Feb 20, 2024

Information processing device, information processing method, and recording medium

Inventors: Mari Saito (Tokyo, JP); Kenji Sugihara (Tokyo, JP)
Assignee: SONY GROUP CORPORATION
G06T13/00G06F3/013G06F3/16G06T7/70G10L15/22G10L25/51G06T2207/30196G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,908,055
App. No.
17/434,970
Granted
Feb 20, 2024
Kind
B2
Abstract

An information processing device includes an acquisition unit that acquires position information, visual line detection information, and voice information of a first user, and position information of a second user, a determination unit that determines a positional relationship between the first user and the second user on a basis of the position information of the first user and the position information of the second user, a specification unit that specifies a voice dialogue area in which a voice dialogue is possible to be performed between the first user and the second user on a basis of the determined positional relationship, and an avatar control unit that sets a position of an avatar outside the specified voice dialogue area, in which in a case where the voice information of the first user is acquired when the first user is looking outside the voice dialogue area, the avatar control unit causes the avatar to execute a predetermined action, and in a case where the voice information of the first user is acquired when the first user is looking inside the voice dialogue area, the avatar control unit suppresses the avatar from executing the predetermined action.

Claims (62)

1. An information processing device comprising:

an acquisition unit that acquires position information, visual line detection information, and voice information of a first user, and position information of a second user;

a determination unit that determines a positional relationship between the first user and the second user on a basis of the position information of the first user and the position information of the second user;

a specification unit that specifies a voice dialogue area in which a voice dialogue is possible to be performed between the first user and the second user on a basis of the determined positional relationship; and

an avatar control unit that sets a position of an avatar outside the specified voice dialogue area,

wherein in a case where the voice information of the first user is acquired when the first user is looking outside the voice dialogue area, the avatar control unit causes the avatar to execute a predetermined action, and

in a case where the voice information of the first user is acquired when the first user is looking inside the voice dialogue area, the avatar control unit suppresses the avatar from executing the predetermined action.

2. The information processing device according to claim 1 , further comprising a detection unit that detects visual line direction information and a peripheral visual field of the first user,

wherein the specification unit specifies the voice dialogue area on a basis of the visual line direction information of the first user when the first user is having a dialogue with the second user.

3. The information processing device according to claim 2 , wherein

the detection unit detects gaze points of the first user as the visual line direction information on a basis of the visual line detection information, and

the specification unit specifies a gaze area that is an area in which the gaze points of the first user are distributed, and specifies the voice dialogue area on a basis of the gaze area and the positional relationship.

4. The information processing device according to claim 3 , wherein

in a case where the voice dialogue area is specified on a basis of the gaze points of the first user,

the detection unit further detects a central visual field of the first user, and

the avatar control unit sets the position of the avatar at a position outside the voice dialogue area, outside the central visual field of the first user, and inside the peripheral visual field of the first user.

5. The information processing device according to claim 2 , wherein

the detection unit detects a direction of a face of the first user as the visual line direction information on a basis of inertia information of the first user acquired as the visual line detection information, and

the specification unit specifies the voice dialogue area on a basis of the direction of the face of the first user.

6. The information processing device according to claim 5 , wherein

in a case where the voice dialogue area is identified based on the direction of the face of the first user,

the specification unit further specifies a visual line movement area that is an area in which the first user is able to move a visual line, and

the avatar control unit sets the position of the avatar at a position outside the voice dialogue area, outside the visual line movement area, and inside the peripheral visual field of the first user.

7. The information processing device according to claim 2 , wherein

in a case where the first user and the second user use a common avatar,

the avatar control unit sets a position of the common avatar in a common space between the peripheral visual field of the first user and the peripheral visual field of the second user.

8. The information processing device according to claim 2 , wherein

in a case where the first user and the second user use a common avatar,

the avatar control unit selects either the peripheral visual field of the first user or the peripheral visual field of the second user, and sets the position of the common avatar in the selected peripheral visual field.

9. The information processing device according to claim 2 , wherein

in a case where the first user is having a dialogue with the second user without looking at the second user,

the specification unit specifies a space between the first user and the second user as the voice dialogue area on a basis of the positional relationship.

10. The information processing device according to claim 9 , wherein

in a case where the first user and the second user are moving side by side in parallel,

the avatar control unit sets the position of the avatar at a position outside a space in a moving direction of the first user and the second user.

11. The information processing device according to claim 9 , wherein

in a case where the first user is working,

the avatar control unit sets the position of the avatar at a position outside a work area that is an area where the first user performs work.

12. The information processing device according to claim 1 , wherein the specification unit specifies in advance an area in which it is estimated that the voice dialogue is possible to be performed between the first user and the second user as the voice dialogue area.

13. The information processing device according to claim 1 , wherein the avatar control unit causes a position of the avatar to be presented by displaying an object or outputting a voice.

14. The information processing device according to claim 1 , further comprising a recognition unit that performs voice recognition on a basis of the voice information,

wherein the predetermined action is an action determined on a basis of a result of the voice recognition by the recognition unit.

15. The information processing device according to claim 1 , wherein the acquisition unit includes a microphone that acquires the voice information of the first user and the second user, and acquires the position information of the second user on a basis of the voice information acquired by the microphone.

16. The information processing device according to claim 1 , wherein the acquisition unit includes an imaging device capable of imaging a periphery of the first user, and acquires the position information of the second user on a basis of an image captured by the imaging device.

17. The information processing device according to claim 1 , further comprising a communication unit,

wherein the avatar control unit acquires information regarding a position where it is possible to display the avatar for the second user from an external device via communication by the communication unit.

18. The information processing device according to claim 1 , further comprising a display unit,

wherein the display unit displays the avatar at the position set by the avatar control unit.

19. An information processing method comprising:

acquiring position information, visual line detection information, and voice information of a first user, and position information of a second user;

determining a positional relationship between the first user and the second user on a basis of the position information of the first user and the position information of the second user; and

specifying a voice dialogue area in which a voice dialogue is possible to be performed between the first user and the second user on a basis of the determined positional relationship; and

setting a position of an avatar outside the specified voice dialogue area,

wherein in a case where the voice information of the first user is acquired when the first user is looking outside the voice dialogue area, the setting causes the avatar to execute a predetermined action, and

in a case where the voice information of the first user is acquired when the first user is looking inside the voice dialogue area, the setting suppresses the avatar from executing the predetermined action.

20. A recording medium recording a program causing a computer to function as:

an acquisition unit that acquires position information, visual line detection information, and voice information of a first user, and position information of a second user;

a determination unit that determines a positional relationship between the first user and the second user on a basis of the position information of the first user and the position information of the second user;

a specification unit that specifies a voice dialogue area in which a voice dialogue is possible to be performed between the first user and the second user on a basis of the determined positional relationship; and

an avatar control unit that sets a position of an avatar outside the specified voice dialogue area,

wherein in a case where the voice information of the first user is acquired when the first user is looking outside the voice dialogue area, the avatar control unit causes the avatar to execute a predetermined action, and

in a case where the voice information of the first user is acquired when the first user is looking inside the voice dialogue area, the avatar control unit suppresses the avatar from executing the predetermined action.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 30, 2021
From: SUGIHARA, KENJI; SAITO, MARI
To: SONY GROUP CORPORATION
Reel/Frame 057330/0799 →
Priority Claims (1)
JP 2019-045552 · Mar 13, 2019 · national
Continuity (1)
Related Publication 20220180583A1 · Jun 9, 2022
Cited By (2)
US 12,411,546 US 12,452,356