Storage medium having stored therein stereoscopic image display program, stereoscopic image display device, stereoscopic image display system, and stereoscopic image display method

1. A non-transitory computer-readable storage medium having stored therein an image display program for stereoscopically displaying a captured image on a stereoscopically-viewable stereoscopic display by displaying, in substantially a same display area, a right-eye image and a left-eye image by using a right captured image outputted from a right-eye real camera and a left captured image outputted from a left-eye real camera, wherein the image display program, when executed, causes a computer to at least:

set a reference distance for a distance from the real cameras in an image-capturing direction, the setting of the reference distance comprising calculating a distance to a predetermined image-capture object, and setting the reference distance based on the calculated distance;

display the right and left captured images in the display area so that a reference point common to the right and left captured images is displayed at a center of the display area, wherein the reference point is a point that is away from the real cameras by the reference distance in the image-capturing direction of the two real cameras on a straight line passing through a median point between the two real cameras and extending in the real camera image-capturing direction;

set a right-eye virtual camera and a left-eye virtual camera in a virtual space; and

stereoscopically display a virtual image by displaying in the display area a right virtual image obtained by capturing an image of the virtual space with the right-eye virtual camera and a left virtual image obtained by capturing an image of the virtual space with the left-eye virtual camera, wherein the left and right virtual images have a common virtual reference point and a projection matrix of the right virtual camera is set so that the virtual reference point in the right virtual image is displayed at the center of the display area and a projection matrix of the left virtual camera is set so that the virtual reference point in the left virtual image is displayed at the center of the display area,

wherein the virtual reference point is a point that is away from the virtual cameras by a distance corresponding to the reference distance in an image-capturing direction of the two virtual cameras on a straight line passing through a median point between the two virtual cameras and extending in the virtual camera image-capturing direction, and

wherein the image display program further causes the computer to recognize, using at least one of the right captured image and the left captured image, a position and an orientation of the predetermined image-capture object in the captured image so as to calculate position/orientation information representing a relative position and orientation relationship between the predetermined image-capture object and the real cameras, and to calculate a distance between the predetermined image-capture object and the real cameras based on the position/orientation information.

2. The storage medium according to claim 1 , wherein the image display program further causes the computer to at least:

determine a right display area which is an area of the right captured image to be displayed in the display area and which is centered about a point corresponding to the reference point in the right captured image, and determine a left display area which is an area of the left captured image to be displayed in the display area and which is centered about a point corresponding to the reference point in the left captured image; and display the right display area of the right captured image and the left display area of the left captured image in the display area based on the determined areas.

3. The storage medium according to claim 1 , wherein the reference distance is fixedly set.

4. The storage medium according to claim 1 ,

wherein the image display program further causes the computer to set the reference distance using the calculated distance as a reference.

5. The storage medium according to claim 4 , wherein the image display program further causes the computer to set, as the reference distance, a value obtained by adding or subtracting a value to/from the calculated distance.

6. The storage medium according to claim 1 ,

wherein the image display program further causes the computer to recognize, using both of the right captured image and the left captured image, the position and the orientation of the predetermined image-capture object in the captured image so as to calculate the position/orientation information representing the relative position and orientation

relationship between the predetermined image-capture object and the real cameras, thereby calculating a distance between the predetermined image-capture object and the real cameras based on the position/orientation information.

7. The storage medium according to claim 1 , wherein the image display program further causes the computer to display the right captured image and the left captured image in the display area so that the reference point is displayed at the center of the display area by using an angle of view of the real cameras and a distance between the real cameras.

8. An image display device for stereoscopically displaying a captured image on a stereoscopically-viewable stereoscopic display by displaying, in substantially a same display area, a right-eye image and a left-eye image by using a right captured image outputted from a right-eye real camera and a left captured image outputted from a left-eye real camera, the image display device comprising:

processing circuitry including a processor, the processing circuitry being configured to control the image display device to at least:

set a reference distance for a distance from the real cameras in an image-capturing direction, the setting of the reference distance comprising calculating a distance to a predetermined image-capture object, and setting the reference distance based on the calculated distance;

display the right and left captured images in the display area so that a reference point common to the right and left captured images is displayed at a center of the display area, wherein the reference point is a point that is away from the real cameras by the reference distance in the image-capturing direction of the two real cameras on a straight line passing through a median point between the two real cameras and extending in the real camera image-capturing direction;

set a right-eye virtual camera and a left-eye virtual camera in a virtual space; and

stereoscopically display a virtual image by displaying in the display area a right virtual image obtained by capturing an image of the virtual space with the right-eye virtual camera and a left virtual image obtained by capturing an image of the virtual space with the left-eye virtual camera,

wherein the left and right virtual images have a common virtual reference point and the virtual camera setting section sets a projection matrix of the right virtual camera so that the virtual reference point in the right virtual image is displayed at the center of the display area and sets a projection matrix of the left virtual camera so that the virtual reference point in the left virtual image is displayed at the center of the display area,

wherein the virtual reference point is a point that is away from the virtual cameras by a distance corresponding to the reference distance in an image-capturing direction of the two virtual cameras on a straight line passing through a median point between the two virtual cameras and extending in the virtual camera image-capturing direction, and

wherein the processing circuitry is configured to control the image display device to recognize, using at least one of the right captured image and the left captured image, a position and an orientation of the predetermined image-capture object in the captured image so as to calculate position/orientation information representing a relative position and orientation relationship between the predetermined image-capture object and the real cameras, and to calculate a distance between the predetermined image-capture object and the real cameras based on the position/orientation information.

9. An image display system for stereoscopically

displaying a captured image on a stereoscopically-viewable stereoscopic display by displaying, in substantially a same display area, a right-eye image and a left-eye image by using a right captured image outputted from a right-eye real camera and a left captured image outputted from a left-eye real camera, the system comprising:

a reference distance setting section for setting a reference distance for a distance from the real cameras in an image-capturing direction, the reference distance setting section being configured to calculate a distance to a predetermined image-capture object, and to set the reference distance based on the calculated distance;

a stereoscopic display controller for displaying the right and left captured images in the display area so that a reference point common to the right and left captured images is displayed at a center of the display area, wherein the reference point is a point that is away from the real cameras by the reference distance in the image-capturing direction of the two real cameras on a straight line passing through a median point between the two real cameras and extending in the real camera image-capturing direction;

a virtual camera setting section for setting a right-eye virtual camera and a left-eye virtual camera in a virtual space; and

a virtual image stereoscopic display controller for stereoscopically displaying a virtual image by displaying in the display area a right virtual image obtained by capturing an image of the virtual space with the right-eye virtual camera and a left virtual image obtained by capturing an image of the virtual space with the left-eye virtual camera,

wherein the left and right virtual images have a common virtual reference point and a virtual camera setting section sets a projection matrix of the right virtual camera so that the virtual reference point in the right virtual image is displayed at the center of the display area and sets a projection matrix of the left virtual camera so that the virtual reference point in the left virtual image is displayed at the center of the display area,

wherein the virtual reference point is a point that is away from the virtual cameras by a distance corresponding to the reference distance in an image-capturing direction of the two virtual cameras on a straight line passing through a median point between the two virtual cameras and extending in the virtual camera image-capturing direction, and

wherein the system is configured to recognize, using at least one of the right captured image and the left captured image, a position and an orientation of the predetermined imagecapture object in the captured image so as to calculate position/orientation information representing a relative position and orientation relationship between the predetermined imagecapture object and the real cameras, and to calculate a distance between the predetermined image-capture object and the real cameras based on the position/orientation information.

10. An image display method to be executed by a computer for stereoscopically displaying a captured image on a stereoscopically-viewable stereoscopic display by displaying, in substantially a same display area, a right-eye image and a left-eye image by using a right captured image outputted from a right-eye real camera and a left captured image outputted from a left-eye real camera, the method comprising:

setting a reference distance for a distance from the real cameras in an image-capturing direction, the setting of the reference distance comprising calculating a distance to a predetermined image-capture object, and setting the reference distance based on the calculated distance;

displaying the right and left captured images in the display area so that a reference point common to the right and left captured images is displayed at a center of the display area, wherein the reference point is a point that is away from the real cameras by the reference distance in the image-capturing direction of the two real cameras on a straight line passing through a median point between the two real cameras and extending in the real camera image-capturing direction;

setting a right-eye virtual camera and a left-eye virtual camera in a virtual space; and stereoscopically displaying a virtual image by displaying in the display area a right virtual image obtained by capturing an image of the virtual space with the right-eye virtual camera and a left virtual image obtained by capturing an image of the virtual space with the left eye virtual camera,

wherein the left and right images have a common virtual reference point and a projection matrix of the right virtual camera is set so that the virtual reference point in the right virtual image is displayed at the center of the display area and a projection matrix of the left virtual camera is set so that the virtual reference point in the left virtual image is displayed at the center of the display area,

wherein the virtual reference point is a point that is away from the virtual cameras by a distance corresponding to the reference distance in an image-capturing direction of the two virtual cameras on a straight line passing through a median point between the two virtual cameras and extending in the virtual camera image-capturing direction, and

wherein the method further comprises recognizing, using at least one of the right captured image and the left captured image, a position and an orientation of the predetermined image-capture object in the captured image so as to calculate position/orientation information representing a relative position and orientation relationship between the predetermined image-capture object and the real cameras, and to calculate a distance between the predetermined image-capture object and the real cameras based on the position/orientation information.