All-around moving image distribution system, all-around moving image distribution method, image processing apparatus, communication terminal apparatus, and control methods and control programs of image processing apparatus and communication terminal apparatus

1. An all-around moving image distribution system comprising:

an all-around camera;

an all-around image generator that acquires an all-around moving image captured by said all-around camera and generates time-series all-around frame image data;

microphones that acquire a stereoscopic sound in synchronism with image capturing of said all-around camera;

a stereoscopic sound data generator that generates time-series stereoscopic sound data from the stereoscopic sound, and uploads the time-series stereoscopic sound data;

an all-around moving image data generator that encodes the time-series all-around frame image data, generates all-around moving image data, and uploads the all-around moving image data;

a distribution server that distributes the all-around moving image data uploaded from said all-around moving image data generator and the stereoscopic sound data uploaded from said stereoscopic sound data generator; and

an all-around moving image reproducer that reproduces the all-around moving image data distributed from said distribution server, displays a moving image of a range selected from a reproduced all-around moving image in accordance with an instruction of changing a line-of-sight by a user, and reproduces and outputs a stereoscopic sound corresponding to a display direction of the moving image from the stereoscopic sound data distributed from said distribution server,

wherein said all-around moving image reproducer performs one of a zoom-in operation and a zoom-out operation of the moving image, and reproduces and outputs a stereoscopic sound corresponding to one of the zoom-in operation and the zoom-out operation of the moving image from the stereoscopic sound data.

2. The all-around moving image distribution system according to claim 1 , wherein said all-around moving image reproducer includes an all-around moving image developer that decodes the all-around moving image data received from said distribution server, and develops, for each all-around frame image, the decoded all-around moving image data onto a three-dimensional mapping surface including a viewpoint inside, and selects and displays a moving image of a line-of-sight direction from the viewpoint, and

the zoom-in operation of the moving image is performed by moving the viewpoint closer to the mapping surface, and the zoom-out operation of the moving image is performed by moving the viewpoint away from the mapping surface.

3. The all-around moving image distribution system according to claim 1 , wherein

said all-around camera comprises at least two all-around cameras fixed, and

said all-around moving image reproducer includes a moving image selector that selects the all-around moving image data distributed from said distribution server in accordance with a selection instruction of the at least two all-around cameras from the user.

4. The all-around moving image distribution system according to claim 1 , wherein

said all-around camera comprises a moving all-around camera,

said all-around moving image distribution system further comprises a position determiner that determines, based on matching between a feature of the all-around moving image and a feature of an image captured in advance from a known position, a position at which the all-around moving image is captured, and

said all-around moving image reproducer displays the moving image of the range corresponding to the instruction of changing the line-of-sight by the user, and the position determined by said position determiner.

5. The all-around moving image distribution system according to claim 1 , further comprising:

an acquirer that acquires information including at least one of display information and sound information corresponding to a known target; and

a determiner that determines, based on matching between a feature of all-around moving image data of the range according to the instruction of changing the line-of-sight by the user and a feature of image data of the known target, whether the all-around moving image data includes the known target,

wherein if the all-around moving image data includes the known target, said all-around moving image reproducer combines the information in correspondence with the known target, and outputs the all-around moving image.

6. The all-around moving image distribution system according to claim 1 , further comprising:

a television station that generates television moving image data from the all-around moving image data distributed from said distribution server, and transmits the television moving image data,

wherein said all-around moving image reproducer is included in a television set, scans and displays a moving image of a range selected from an all-around moving image based on the television moving image data transmitted from said television station in accordance with the instruction of changing the line-of-sight by the user, and reproduces and outputs a stereoscopic sound corresponding to a display direction of the moving image from the stereoscopic sound data transmitted from said television station.

7. An all-around moving image distribution method comprising:

acquiring an all-around moving image captured by an all-around camera, and generating, time-series all-around frame image data, by an all-around image generator;

generating time-series stereoscopic sound data from a stereoscopic sound acquired by microphones in synchronism with image capturing of the all-around camera, and uploading the time-series stereoscopic sound data, by a stereoscopic sound data generator;

generating all-around moving image data by encoding the time-series all-around frame image data, and uploading the all-around moving image data, by an all-around moving image data generator;

distributing the uploaded all-around moving image data and the uploaded stereoscopic sound data, by a distribution server; and

reproducing the all-around moving image data distributed from the distribution server, displaying a moving image of a range selected from a reproduced all-around moving image in accordance with an instruction of changing a line-of-sight by a user, and reproducing and outputting a stereoscopic sound corresponding to a display direction of the moving image from the stereoscopic sound data distributed from the distribution server, by an all-around moving image reproducer,

wherein, in said moving image displaying and stereoscopic sound outputting step, one of a zoom-in operation and a zoom-out operation of the moving image is performed, and a stereoscopic sound corresponding to one of the zoom-in operation and the zoom-out operation of the moving image is reproduced and output from the stereoscopic sound data.

8. A communication terminal apparatus comprising:

an all-around moving image data receiver that receives all-around moving image data from a distribution server distributing an all-around moving image;

a stereoscopic sound data receiver that receives, from the distribution server, stereoscopic sound data generated from a stereoscopic sound acquired in synchronism with the all-around moving image data;

an all-around moving image developer that decodes the received all-around moving image data, and develops, for each all-around frame image, the received all-around moving image data onto a three-dimensional mapping surface including a viewpoint inside; and

an all-around moving image reproducer that selects moving image data in a display range selected from the developed all-around frame image in accordance with an instruction of changing a line-of-sight by a user, and displays the selected moving image in the display range, and reproduces and outputs, from the stereoscopic sound data, a stereoscopic sound corresponding to a direction in the all-around frame image of the display range,

wherein said all-around moving image reproducer performs one of a zoom-in operation and a zoom-out operation of the moving image, and reproduces and outputs a stereoscopic sound corresponding to one of the zoom-in operation and the zoom-out operation of the moving image from the stereoscopic sound data.

9. The communication terminal apparatus according to claim 8 , wherein said all-around moving image reproducer includes a head mounted display that implements three-dimensional viewing.

10. A control method of a communication terminal apparatus, comprising:

receiving all-around moving image data from a distribution server distributing an all-around moving image, by an all-around moving image data receiver of the communication terminal apparatus;

receiving, from the distribution server, stereoscopic sound data generated from a stereoscopic sound acquired in synchronism with the all-around moving image data, by a stereoscopic sound data receiver of the communication terminal apparatus;

decoding the received all-around moving image data, and developing, for each all-around frame image, the received all-around moving image data onto a three-dimensional mapping surface including a viewpoint inside, by an all-around moving image mapper of the communication terminal apparatus; and

selecting moving image in a display range from the developed all-around frame image in accordance with an instruction of changing a line-of-sight by a user, and displaying the selected moving image in the display range, and reproducing and outputting, from the stereoscopic sound data, a stereoscopic sound corresponding to a direction in the all-around frame image of the display range by an all-around moving image reproducer of the communication terminal apparatus,

wherein in said moving image displaying and stereoscopic sound outputting step, one of a zoom-in operation and a zoom-out operation of the moving image is performed, and a stereoscopic sound corresponding to one of the zoom-in operation and the zoom-out operation of the moving image is reproduced and output from the stereoscopic sound data.

11. A non-transitory computer-readable storage medium storing a control program of a communication terminal apparatus, for causing a computer of the communication terminal apparatus to execute a method, comprising:

receiving all-around moving image data from a distribution server that distributes an all-around moving image;

receiving, from the distribution server, stereoscopic sound data generated from a stereoscopic sound acquired in synchronism with the all-around moving image data;

decoding the received all-around moving image data, and developing, for each all-around frame image, the received all-around moving image data onto a three dimensional mapping surface including a viewpoint inside; and

selecting moving image in a display range from the developed all-around frame image in accordance with an instruction of changing a line-of-sight by a user, and displaying the selected moving image in the display range, and reproducing and outputting, from the stereoscopic sound data, a stereoscopic sound corresponding to a direction in the all-around frame image of the display range,

wherein in moving image displaying and stereoscopic sound outputting step, one of a zoom-in operation and a zoom-out operation of the moving image is performed, and a stereoscopic sound corresponding to one of the zoom-in operation and the zoom-out operation of the moving image is reproduced and output from the stereoscopic sound data.