Image data transfer apparatus and image compression method
An image contents acquisition section in a compression coding section of a server acquires information associated with contents indicated by a moving image generated by an image generation section. A communication status acquisition section acquires a status of communication with an image processing apparatus corresponding to a data transmission destination. A compression coding processing section adjusts a data size of the moving image according to a change of the communication status, by using means determined on the basis of the contents indicated by the moving image, and compression-codes data of the moving image. A communication section transmits compression-coded data to the image processing apparatus.
1 . An image data transfer apparatus comprising:
a drawing section that draws each frame of a moving image to be displayed by an image processing apparatus, each frame having a respective formation time;
a compression coding section comprising:
an image contents acquisition section that acquires information associated with contents indicated by the moving image;
a communication status acquisition section that acquires, at a predetermined rate, a communication status used for communication of the moving image, wherein the communication status comprises data associated with a timing period that occurs from communicating each frame of the moving image between the image data transfer apparatus and the image processing apparatus;
a compression coding processing section that compression codes the moving image in units of partial images to thereby generate compression-coded data, wherein a data size of each partial image of a respective frame is smaller than the respective frame, wherein the compression coding processing section adjusts the data size of each partial image according to a change of the communication status, by using means determined on a basis of the contents indicated by the moving image, and based on one or more predefined rules for determining adjustment parameters on the basis of the contents; and
a communication section that transfers the compression-coded data of the moving image and a history of formation times of each frame to the image processing apparatus for display, wherein the history of formation times of each frame comprises formation times associated with each partial image.
2 . The image data transfer apparatus according to claim 1 , wherein the drawing section dynamically draws a moving image that is to be transmitted and has not been present before.
3 . The image data transfer apparatus according to claim 1 , wherein the compression coding processing section adjusts the data size by changing at least any one of a frame rate, a resolution, and a quantization parameter of the moving image.
4 . The image data transfer apparatus according to claim 1 , wherein the image contents acquisition section acquires the information associated with the contents on a basis of at least any one of information received from the drawing section, an analysis result of the moving image drawn by the drawing section, and bibliographic information associated with the moving image.
5 . The image data transfer apparatus according to claim 4 , wherein the image contents acquisition section uses, as the analysis result of the moving image, information acquired in a compression coding process by the compression coding processing section.
6 . The image data transfer apparatus according to claim 1 , wherein the image contents acquisition section acquires, as the information associated with the contents, at least any one of a timing of a user operation associated with content that specifies the moving image, an interval of the timing, contents of the user operation, a status of the content, and a status of audio generated according to the content.
7 . The image data transfer apparatus according to claim 6 , wherein the image contents acquisition section acquires, as the status of the content, from the drawing section, information for determining at least any one of a) whether a current scene is a scene requiring a user operation and is one influencing processing contents of the content, b) whether the current scene is a movie scene not requiring a user operation, c) whether the current scene is a scene other than a movie scene and is one not requiring a user operation, and d) whether the current scene is a scene requiring a user operation and is one not included in an original story of the content.
8 . The image data transfer apparatus according to claim 1 , wherein the image contents acquisition section acquires, as the information associated with the contents, at least any one of an amount of an optical flow, an amount of motion estimation, an allocation status of a coding unit, whether or not a current timing is a scene switching timing, a type of image texture expressed in a frame, a distribution of feature points, depth information, an amount of objects, a use amount of each level of mipmap texture used for three-dimensional graphics, an LOD (Level Of Detail), a use amount of each level of tessellation, an amount of characters and symbols, and a type of an expressed scene.
9 . The image data transfer apparatus according to claim 1 , wherein the compression coding processing section derives a score reflecting the contents indicated by the moving image, under a predetermined rule of the one or more predefined rules, and determines means and an adjustment amount used for adjustment of the data size, on a basis of the score.
10 . The image data transfer apparatus according to claim 9 , wherein the compression coding processing section adjusts the score derived under the predetermined rule in a direction of an increase when an object that meets a predetermined criterion for determining that a user operation is reflected is present in a screen in an immediately preceding predetermined period.
11 . The image data transfer apparatus according to claim 9 , wherein a predetermined rule of the one or more predefined rules for determining the means and the adjustment amount on the basis of the score is optimized by machine learning or deep learning.
12 . The image data transfer apparatus according to claim 9 , wherein the compression coding processing section changes the score according to a change of an apparent size of an identical object expressed by the moving image, as the information indicated by the contents.
13 . The image data transfer apparatus according to claim 9 , wherein the compression coding processing section changes the score according to a rate of an occupation area of an object in an entire image, the object being included in objects expressed by the moving image and smaller than a predetermined value, as the information indicated by the contents.
14 . The image data transfer apparatus according to claim 9 , wherein the compression coding processing section changes the score according to a rate of an occupation area of a region in an entire image, the region being a region where contrast based on a pixel value distribution or a dynamic range based on a luminance distribution is higher than a predetermined criterion, as the information indicated by the contents.
15 . The image data transfer apparatus according to claim 9 , wherein the compression coding processing section changes the score according to an amount of an object that is included in objects expressed by the moving image and moves by an amount larger than a predetermined value, and a rate of an occupation area of the object in an entire image, as the information indicated by the contents.
16 . The image data transfer apparatus according to claim 9 , wherein the compression coding processing section changes the score according to a rate of an occupation area of a predetermined type of texture in an entire image, as the information indicated by the contents.
17 . The image data transfer apparatus according to claim 1 , wherein the compression coding processing section derives a plurality of scores indicating respective weights given to a frame rate, a resolution, and quality of quantization, and determines a combination of the frame rate, the resolution, and a value of a quantization parameter so as to obtain a data size corresponding to the communication status and follow a balance indicated by the scores.
18 . The image data transfer apparatus according to claim 10 , wherein the compression coding processing section increases a score of the plurality of scores indicating the weight of the frame rate as a change of an input value of a user operation associated with content specifying the moving image during an immediately preceding predetermined period increases.
19 . The image data transfer apparatus according to claim 1 , wherein the compression coding processing section compression-codes the moving image in units of a partial image smaller than one frame and adjusts the data size in units of any one of the partial image, one frame, and a predetermined number of frames.
20 . The image data transfer apparatus according to claim 1 , wherein the compression coding processing section resets means and an adjustment amount used for adjustment of the data size at predetermined time intervals.
21 . The image data transfer apparatus according to claim 1 , wherein the communication status acquisition section obtains the communication status by acquiring at least either a data arrival rate or a delay time period of the moving image at the image processing apparatus corresponding to a transmission destination at predetermined time intervals.
22 . The image data transfer apparatus according to claim 1 , wherein the compression coding processing section compression-codes, using an algorithm of SVC (Scalable Video Coding), an image at a level defined on a basis of at least a resolution and adjusts a combination of parameters used for compression coding, on the basis of the contents indicated by the moving image.
23 . The image data transfer apparatus according to claim 22 , wherein the compression coding processing section changes a parameter used when data is divided into levels, on a basis of at least either the contents indicated by the moving image or the communication status.
24 . The image data transfer apparatus according to claim 22 , wherein the compression coding processing section changes a number of levels of compression-coded data or changes a level of compression-coded data to be referred to at a time of compression coding, on a basis of at least either the contents indicated by the moving image or the communication status.
25 . The image data transfer apparatus according to claim 1 , wherein the drawing section draws each frame of the moving image by projecting an object model disposed in a space to be drawn, and the image contents acquisition section acquires, as the information associated with the contents indicated by the moving image, a parameter adjusted by the drawing section in a drawing process.
26 . An image compression method performed by an image data transfer apparatus, the image compression method comprising:
drawing, using a drawing section of the image data transfer apparatus, each frame of a moving image to be displayed by an image processing apparatus, each frame having a respective formation time;
receiving, by a compression coding section of the image data transfer apparatus, each frame of the moving image and the respective formation time of each frame;
acquiring, using an image contexts acquisition section of the compression coding section, information associated with contents indicated by the moving image;
acquiring, using a communication status acquisition section of the compression coding section, a communication status used for communication of the moving image, wherein the communication status comprises data associated with a timing period that occurs from communicating each frame of the moving image between the image data transfer apparatus and the image processing apparatus;
compression coding the moving image in partial images
compressing, using a compression coding processing section of the compression coding section, the moving image in units of partial images to thereby generate compression-coded data, wherein a data size of each partial image of a respective frame is smaller than the respective frame, wherein the compression coding processing section adjusts the data size of each partial image according to a change of the communication status, by using means determined on a basis of the contents indicated by the moving image, and based on one or more predefined rules for determining adjustment parameters on the basis of the contents; and
transferring, using a communication section of the image data transfer apparatus, the compression-coded data of the moving image and a history of formation times of each frame to the image processing apparatus for display, wherein the history of formation times of each frame comprises formation times associated with each partial image.
27 . A non-transitory, computer readable storage medium containing a computer program, which when executed by a an image data transfer apparatus, causes the an image data transfer apparatus to perform an image compression method by carrying out actions, comprising:
drawing, using a drawing section of the image data transfer apparatus, each frame of a moving image to be displayed by an image processing apparatus, each frame having a respective formation time;
receiving, by a compression coding section of the image data transfer apparatus, each frame of the moving image and the respective formation time of each frame;
acquiring, using an image contexts acquisition section of the compression coding section, information associated with contents indicated by the moving image;
acquiring, using a communication status acquisition section of the compression coding section, a communication status used for communication of the moving image, wherein the communication status comprises data associated with a timing period that occurs from communicating each frame of the moving image between the image data transfer apparatus and the image processing apparatus;
compressing, using a compression coding processing section of the compression coding section, the moving image in units of partial images to thereby generate compression-coded data, wherein a data size of each partial image of a respective frame is smaller than the respective frame, wherein the compression coding processing section adjusts the data size of each partial image according to a change of the communication status, by using means determined on a basis of the contents indicated by the moving image, and based on one or more predefined rules for determining adjustment parameters on the basis of the contents; and
transferring, using a communication section of the image data transfer apparatus, the compression-coded data of the moving image and a history of formation times of each frame to the image processing apparatus for display, wherein the history of formation times of each frame comprises formation times associated with each partial image.