Machine learning based image processing techniques
A machine learning based image processing architecture and associated applications are disclosed herein. In some embodiments, a machine learning framework is trained to learn low level image attributes such as object/scene types, geometries, placements, materials and textures, camera characteristics, lighting characteristics, contrast, noise statistics, etc. Thereafter, the machine learning framework may be employed to detect such attributes in other images and process the images at the attribute level.
1 . A method, comprising:
identifying a set of one or more attributes shared by a set of input images using a machine learning framework, wherein the set of input images is constrained to a prescribed scene type and wherein the machine learning framework is at least in part trained on a set of training images comprising the prescribed scene type; and
generating a set of one or more output images of the prescribed scene type having at least the same shared set of one or more attributes as the set of input images.
2 . The method of claim 1 , wherein the set of training images comprises different combinations of objects and object arrangements, camera configurations, lighting types and locations, and materials and textures.
3 . The method of claim 1 , wherein a training image of the set of training images is rendered using one or more three-dimensional models.
4 . The method of claim 1 , wherein a training image of the set of training images is captured by an imaging or a scanning device.
5 . The method of claim 1 , wherein a training image of the set of training images is generated from one or more other existing images.
6 . The method of claim 1 , wherein a training image of the set of training images is labeled or tagged with metadata.
7 . The method of claim 1 , wherein a training image of the set of training images is labeled or tagged with ground truth data associated with generating the training image.
8 . The method of claim 1 , wherein the identified set of one or more attributes comprises attributes associated with one or more of: object/scene types, geometries, placements, materials, textures, camera characteristics, lighting characteristics, and image statistics.
9 . The method of claim 1 , wherein an identified attribute comprises a function of a plurality of lower level attributes.
10 . The method of claim 1 , wherein the identified set of one or more attributes comprises characteristics imparted to the set of input images by post-processing.
11 . The method of claim 1 , wherein the identified set of one or more attributes comprises characteristics imparted to the set of input images by manual manipulation.
12 . The method of claim 1 , wherein the identified set of one or more attributes comprises characteristics imparted to the set of input images by retouching or remastering.
13 . The method of claim 1 , wherein the identified set of one or more attributes comprises visual characteristics associated with a signature style or a prescribed aesthetic.
14 . The method of claim 1 , further comprising labeling or tagging the set of input images with the identified set of one or more attributes.
15 . The method of claim 1 , wherein an output image of the set of one or more output images is generated using the identified set of one or more attributes during rendering of the output image.
16 . The method of claim 1 , wherein the set of input images, the set of training images, and the set of one or more output images comprise photographs, renderings, or both.
17 . The method of claim 1 , wherein the generated set of one or more output images comprises video frames.
18 . The method of claim 1 , wherein the machine learning framework comprises a deep neural network or a convolutional neural network.
19 . The method of claim 1 , wherein the prescribed scene type comprises a constrained set of one or more objects.
20 . The method of claim 1 , wherein the prescribed scene type is associated with publishable imagery.
21 . The method of claim 1 , wherein the prescribed scene type is associated with a collection of curated images.
22 . The method of claim 1 , wherein the prescribed scene type is associated with a catalog.
23 . The method of claim 1 , wherein the prescribed scene type is associated with an animation or a video sequence.
24 . A system, comprising:
a processor configured to:
identify a set of one or more attributes shared by a set of input images using a machine learning framework, wherein the set of input images is constrained to a prescribed scene type and wherein the machine learning framework is at least in part trained on a set of training images comprising the prescribed scene type; and
generate a set of one or more output images of the prescribed scene type having at least the same shared set of one or more attributes as the set of input images; and
a memory coupled to the processor and configured to provide the processor with instructions.
25 . A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:
identifying a set of one or more attributes shared by a set of input images using a machine learning framework, wherein the set of input images is constrained to a prescribed scene type and wherein the machine learning framework is at least in part trained on a set of training images comprising the prescribed scene type; and
generating a set of one or more output images of the prescribed scene type having at least the same shared set of one or more attributes as the set of input images.