Low delay super-resolution processing
A client terminal includes a data acquiring section, a super-resolution processing section 48 , and a display control section. The data acquiring section of the client terminal acquires video data in units of partial image which is smaller than one frame. The super-resolution processing section of the client terminal executes a super-resolution process in units of partial image acquired by the data acquiring section. The display control section of the client terminal sequentially outputs, to a display panel, partial images having undergone the super-resolution process at the super-resolution processing section.
1. An image processing apparatus comprising:
one or more processors; and
one or more memories storing instruction that, when executed by the one or more processors, cause the image processing apparatus to perform operations comprising:
acquiring video data in units of partial image which is smaller than one frame;
executing a super-resolution process in the units of partial image, wherein executing the super-resolution process comprises acquiring first scene information prior to compression-encoding a unit of partial image of the units of partial image and acquiring second scene information after compression-encoding the unit of partial image, wherein the second scene information is generated based on a compression-encoded partial image, wherein executing the super-resolution process comprises switching a model for the super-resolution process based on the first scene information and the second scene information; and
sequentially outputting, to a display section, partial images that have undergone the super-resolution process.
2. The image processing apparatus according to claim 1 , the operations further comprising:
expansion-decoding a compression-encoded partial image to form an expansion-decoded partial image, and
executing a super-resolution process on the expansion-decoded partial image.
3. The image processing apparatus according to claim 1 , wherein executing the super-resolution process comprises performing a scene analysis on a basis of a partial image, and executing the super-resolution process for the partial image on a basis of a result of the scene analysis.
4. The image processing apparatus according to claim 3 , wherein executing the super-resolution process comprises generating one or more partial images, each partial image of the one or more partial images having a resolution which is lower than a resolution of a partial image, and performing a scene analysis of the one or more partial images in order from a partial image with a relatively low resolution to a partial image with a relatively high resolution.
5. The image processing apparatus according to claim 3 , wherein executing the super-resolution process comprises extracting, from discrete positions in the partial image, a plurality of region images of an analysis unit granularity that is smaller than a partial image, and performing the scene analysis based on the plurality of region images.
6. The image processing apparatus according to claim 3 , wherein executing the super-resolution process comprises performing the scene analysis by referring to internal drawing data held in an application, before the application generates a final drawing result, and uses a result of the scene analysis for the super-resolution process.
7. The image processing apparatus according to claim 1 , wherein executing the super-resolution process comprises referring to internal drawing data held in an application, before the application generates a final drawing result, and uses the internal drawing data for the super-resolution process.
8. The image processing apparatus according to claim 1 , wherein executing the super-resolution process comprises selecting simplification or nonexecution of the super-resolution process on a basis of scene information that indicates a feature in a partial image acquired as a result of a process prior to executing the super-resolution process.
9. The image processing apparatus according to claim 1 , wherein the second scene information is acquired before the compression-encoded partial image is expansion-decoded.
10. The image processing apparatus according to claim 1 , wherein executing the super-resolution process comprises acquiring scene information by using internal drawing data held in an application, before the application generates a final drawing result, and switching a model for the super-resolution process on a basis of the scene information.
11. The image processing apparatus according to claim 1 , wherein executing the super-resolution process comprises dividing a partial image into Y-component data and UV-component data, and does not execute a super-resolution process for the UV-component data but executes a super-resolution process for the Y-component data.
12. The image processing apparatus according to claim 11 , wherein
a common identification is given to the Y-component data and the UV-component data that are originated from a same partial image, and
the UV-component data is buffered during the super-resolution process for the Y-component data, and is combined with the Y-component data with the common identification, the Y-component data having been undergone the super-resolution process.