IP Library › Granted Patent US 12,063,344
Granted Patent B2
US 12,063,344 · App. 17/640,983 · Granted Aug 13, 2024

Reproduction device, reproduction method, and recording medium

Inventor: Takayoshi Shimizu (Tokyo, JP)
Assignee: SONY GROUP CORPORATION
H04N13/373G02B27/0101H04N13/376G02B2027/011
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,063,344
App. No.
17/640,983
Granted
Aug 13, 2024
Kind
B2
Abstract

The present technology relates to a reproduction device, a reproduction method, and a recording medium capable of suppressing influence of motion parallax and image distortion. There is provided a reproduction device including an image processing unit that performs processing including: projecting an image captured from a specific position in a real space on a projection surface arranged according to a reference position set in a virtual space; acquiring a head movement amount of a user viewing the projected image; changing, according to the head movement amount with respect to the reference position in a first direction that is at least one of a left-right direction or an up-down direction as viewed from the user, a first relative distance between the reference position and a user position in the first direction; changing, according to the head movement amount with respect to the reference position.

Claims (57)

1. A reproduction device comprising an image processing unit that performs processing including:

projecting an image captured from a specific position in a real space on a projection surface arranged according to a reference position set in a virtual space;

acquiring a head movement amount of a user viewing the projected image;

changing, according to the head movement amount with respect to the reference position in a first direction that is at least one of a left-right direction or an up-down direction as viewed from the user, a first relative distance between the reference position and a user position in the virtual space in the first direction;

changing, according to the head movement amount with respect to the reference position in a second direction that is a front-back direction as viewed from the user, a second relative distance between the reference position and the user position in the second direction; and

causing a change amount of the first relative distance with respect to the head movement amount in the first direction to be smaller than a change amount of the second relative distance with respect to the head movement amount in the second direction.

2. The reproduction device according to claim 1 , wherein

the image processing unit multiplies by a first coefficient that reduces a change amount of the first relative distance with respect to the head movement amount as a component in the first direction.

3. The reproduction device according to claim 2 , wherein

the image processing unit changes the first relative distance by multiplying an inverse vector of a movement vector corresponding to the head movement amount by the first coefficient that reduces a change amount thereof as a component in the first direction on the projection surface.

4. The reproduction device according to claim 2 , wherein

the image processing unit multiplies by a second coefficient corresponding to a size of the projection surface as a component in the second direction.

5. The reproduction device according to claim 4 , wherein

the second coefficient is a value that generates natural motion parallax in the second direction according to movement of a head of the user.

6. The reproduction device according to claim 4 , further comprising

a relationship of the second coefficient>the first coefficient>0.

7. The reproduction device according to claim 1 , wherein

in a case where the head movement amount in the second direction changes in a state where the user position is separated from the reference position in the first direction, the image processing unit brings the user position closer to the reference position in the first direction and changes the second relative distance according to the head movement amount in the second direction.

8. The reproduction device according to claim 7 , wherein

the image processing unit

arranges, in a case of moving toward a viewpoint of the user on the projected image, a second virtual camera different from a first virtual camera arranged corresponding to the user position at a predetermined position on a straight line connecting the viewpoint and the reference position, and

switches, when a position of the first virtual camera that has moved on a first movement track from the user position crosses a second movement track of the second virtual camera, from an image corresponding to the first virtual camera to an image corresponding to the second virtual camera.

9. The reproduction device according to claim 8 , wherein

the image processing unit

moves the position of the first virtual camera on the first movement track at predetermined time intervals so as to approach the second movement track,

switches, when the position of the first virtual camera crosses the second movement track, from the image corresponding to the first virtual camera to the image corresponding to the second virtual camera, and

moves a position of the second virtual camera on the second movement track at predetermined time intervals so as to approach a position of the viewpoint.

10. The reproduction device according to claim 9 , wherein

movement of the first virtual camera on the first movement track and movement of the second virtual camera on the second movement track temporally correspond to movement from the user position toward the direction of the viewpoint.

11. The reproduction device according to claim 8 , wherein

the image processing unit sets a boundary region that is a region in which the first virtual camera and the second virtual camera are allowed to move in a region corresponding to the projection surface.

12. The reproduction device according to claim 1 , wherein

in a case of cutting out a trimmed image including a viewpoint of the user on the projected image in a state where the user position is separated from the reference position, the image processing unit cuts out a trimmed image when the viewpoint is viewed from a predetermined position corresponding to the reference position.

13. The reproduction device according to claim 12 , wherein

the image processing unit arranges a second virtual camera, which is different from a first virtual camera arranged corresponding to the user position, at a position that is on a straight line connecting the viewpoint and the reference position and is separated from the viewpoint by a same distance as the user position.

14. The reproduction device according to claim 13 , wherein

the image processing unit switches an image corresponding to the first virtual camera to an image corresponding to the second virtual camera in a case of cutting out the trimmed image.

15. The reproduction device according to claim 1 , wherein

the projection surface is a full spherical screen.

16. The reproduction device according to claim 15 , wherein

the reference position is a position corresponding to a center of the full spherical screen.

17. The reproduction device according to claim 15 , wherein

the reproduction device is configured as a head mounted display.

18. The reproduction device according to claim 17 , wherein

the image processing unit converts the full spherical screen into a sufficient diameter size in which a change amount of each component of XYZ axes of the head mounted display is ignorable.

19. A reproduction method comprising, by a reproduction device:

projecting an image captured from a specific position in a real space on a projection surface arranged according to a reference position set in a virtual space;

acquiring a head movement amount of a user viewing the projected image;

changing, according to the head movement amount with respect to the reference position in a first direction that is at least one of a left-right direction or an up-down direction as viewed from the user, a first relative distance between the reference position and a user position in the virtual space in the first direction;

changing, according to the head movement amount with respect to the reference position in a second direction that is a front-back direction as viewed from the user, a second relative distance between the reference position and the user position in the second direction; and

causing a change amount of the first relative distance with respect to the head movement amount in the first direction to be smaller than a change amount of the second relative distance with respect to the head movement amount in the second direction.

20. A non-transitory computer readable medium recording a program for causing a computer to function as an image processing unit that performs processing comprising:

projecting an image captured from a specific position in a real space on a projection surface arranged according to a reference position set in a virtual space;

acquiring a head movement amount of a user viewing the projected image;

changing, according to the head movement amount with respect to the reference position in a first direction that is at least one of a left-right direction or an up-down direction as viewed from the user, a first relative distance between the reference position and a user position in the virtual space in the first direction;

changing, according to the head movement amount with respect to the reference position in a second direction that is a front-back direction as viewed from the user, a second relative distance between the reference position and the user position in the second direction; and

causing a change amount of the first relative distance with respect to the head movement amount in the first direction to be smaller than a change amount of the second relative distance with respect to the head movement amount in the second direction.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 7, 2022
From: SHIMIZU, TAKAYOSHI
To: SONY GROUP CORPORATION
Reel/Frame 059186/0146 →
Priority Claims (1)
JP 2019-166893 · Sep 13, 2019 · national
Continuity (1)
Related Publication 20220337805A1 · Oct 20, 2022