IP Library › Granted Patent US 11,250,636
Granted Patent B2
US 11,250,636 · App. 17/254,305 · Granted Feb 15, 2022

Information processing device, information processing method, and program

Inventors: Takayoshi Shimizu (Tokyo, JP); Tsuyoshi Ishikawa (Tokyo, JP)
Assignee: SONY CORPORATION
G06T19/006G06F3/011
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,250,636
App. No.
17/254,305
Granted
Feb 15, 2022
Kind
B2
Abstract

An information processing device includes: an acquisition unit ( 110, 140 ) configured to acquire a captured image including a subject and three-dimensional subject position information indicating a three-dimensional position of the subject; and a content configuration information generation unit ( 150 ) configured to generate content configuration information including the captured image, the three-dimensional subject position information, and virtual space association information which is information used for an interaction in which the subject in the captured image displayed in the virtual space is involved and is information for associating the subject in the captured image with the three-dimensional subject position information.

Claims (27)

1. An information processing device comprising:

circuitry configured to

acquire a captured image including a subject and three-dimensional subject position information indicating a three-dimensional position of the subject; and

generate content configuration information including the captured image, the three-dimensional subject position information, and virtual space association information which is information used for an interaction in which the subject in the captured image displayed in the virtual space is involved and is information for associating the subject in the captured image with the three-dimensional subject position information, wherein the content configuration information further includes feedback setting information for setting feedback content when an interaction in which the subject in the captured image displayed in the virtual space is performed, the feedback setting information including information for setting content fed back at the time of viewpoint movement in the virtual space.

2. The information processing device according to claim 1 , wherein the virtual space association information includes identification information for associating the captured image with the three-dimensional subject position information.

3. The information processing device according to claim 1 , wherein the virtual space association information includes information regarding a coordinate system of a camera capturing the captured image and a coordinate system of a sensor detecting the three-dimensional subject position information.

4. The information processing device according to claim 1 , wherein the virtual space association information includes information corresponding to an imaging time of the captured image and information corresponding to a detection time of the three-dimensional subject position information.

5. The information processing device according to claim 1 , wherein the feedback setting information includes information for setting content fed back visually, tactually, auditorily, or olfactorily.

6. The information processing device according to claim 1 , wherein the content configuration information includes information indicating a physical coefficient of the subject.

7. The information processing device according to claim 1 , wherein the three-dimensional subject position information is three-dimensional position information of a plurality of points on a surface of the subject.

8. The information processing device according to claim 7 , wherein the three-dimensional subject position information is three-dimensional position information of a plurality of points of the subject satisfying a predetermined condition on the surface.

9. The information processing device according to claim 7 , wherein the three-dimensional subject position information is three-dimensional position information of the number of points in accordance with a transmission speed of the content configuration information.

10. The information processing device according to claim 1 , wherein the three-dimensional subject position information is three-dimensional position information of one point of a tracking target.

11. An information processing device comprising:

circuitry configured to

acquire content configuration information including a captured image including a subject, three-dimensional subject position information indicating a three-dimensional position of the subject, and virtual space association information which is information used for an interaction in which the subject in the captured image displayed in the virtual space is involved and is information for associating the subject in the captured image with the three-dimensional subject position information; and

display the captured image in the virtual space based on the content configuration information and control a feedback output corresponding to an interaction in which the subject in the captured image displayed in the virtual space is involved, wherein the virtual space association information includes information regarding a coordinate system of a camera capturing the captured image and a coordinate system of a sensor detecting the three-dimensional subject position information, the coordinate system of the camera being based on a position of the camera and coordinate axes set based on an attitude of the camera and the coordinate system of the sensor being based on a position of the sensor and coordinate axes set based on an attitude of the sensor, the circuitry being configured to convert data detected by the sensor into the coordinate system of the camera.

12. The information processing device according to claim 11 , wherein the virtual space association information includes identification information for associating the captured image with the three-dimensional subject position information.

13. The information processing device according to claim 11 , wherein the virtual space association information includes information corresponding to an imaging time of the captured image and information corresponding to a detection time of the three-dimensional subject position information.

14. The information processing device according to claim 11 , wherein the content configuration information further includes feedback setting information for setting feedback content when an interaction in which the subject in the captured image displayed in the virtual space is involved is performed.

15. The information processing device according to claim 11 , wherein the content configuration information includes information indicating a physical coefficient of the subject.

16. The information processing device according to claim 14 , wherein the feedback setting information including information for setting content fed back at the time of viewpoint movement in the virtual space.

17. The information processing device according to claim 11 , wherein the circuitry is further configured to select the content configuration information corresponding to a position of a user in the virtual space as the content configuration information to be acquired.

18. The information processing device according to claim 17 , wherein the circuitry is configured to switch the content configuration information selected using arrival of the position of the user in the virtual space at a predetermined position as a trigger.

19. An information processing method performed by a processor, the method comprising:

acquiring a captured image including a subject and three-dimensional subject position information indicating a three-dimensional position of the subject; and

generating content configuration information including the captured image, the three-dimensional subject position information, and virtual space association information which is information used for an interaction in which the subject in the captured image displayed in the virtual space is involved and is information for associating the subject in the captured image with the three-dimensional subject position information, wherein the content configuration information further includes feedback setting information for setting feedback content when an interaction in which the subject in the captured image displayed in the virtual space is performed, the feedback setting information including information for setting content fed back at the time of viewpoint movement in the virtual space.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 21, 2020
From: SHIMIZU, TAKAYOSHI; ISHIKAWA, TSUYOSHI
To: SONY CORPORATION
Reel/Frame 054704/0340 →
Priority Claims (1)
JP JP2018-136547 · Jul 20, 2018 · national
Continuity (1)
Related Publication 20210217248A1 · Jul 15, 2021