System and method for automated detection and replacement of photographic scenes

1. A method of identifying a background type in a photograph, comprising:

extracting a background image from a photograph;

feeding the background image into a first convolution neural network by one or more computer processors to obtain a first decision;

automatically extracting color features in the background image by the one or more computer processors;

transforming the color features into a two-dimensional color feature matrix;

feeding the two-dimensional color feature matrix into a second convolution neural network to obtain a second decision by the one or more computer processors;

automatically extracting texture features in the background image;

transforming the texture features into a two-dimensional texture feature matrix image by the one or more computer processors;

feeding the two-dimensional texture feature matrix into a third convolution neural network to obtain a third decision by the one or more computer processors;

computing a hybrid decision based on the first decision, the second decision, and the third decision; and

identifying a background type in the background image based on the hybrid decision.

2. The method of claim 1 , further comprising:

multiplying the first decision by a first weight to produce a first weighted decision;

multiplying the second decision by a second weight to produce a second weighted decision; and

multiplying the decision third by a third weight to produce a third weighted decision,

wherein the hybrid decision is computed based on the first weighted decision, and the second weighted decision, and the third weighted decision.

3. The method of claim 2 , wherein the hybrid decision is an average, a sum, or a root-mean square function of the first weighted decision, the second weighted decision, and the third weighted decision.

4. The method of claim 2 , wherein an error associated with a value of the hybrid decision is minimized by backpropagation.

5. The method of claim 4 , wherein values of the first weight, the second weight, and the third weight are updated by minimizing the error associated with the value of the hybrid decision.

6. The method of claim 1 , wherein the color features include a histogram, a mean, a standard deviation, or popular color locations in the color space.

7. The method of claim 6 , wherein transforming the color features into a two-dimensional color feature matrix comprises:

forming a color feature vector using the color features; and

concatenating the color feature vector into the two-dimensional color feature matrix.

8. The method of claim 1 , wherein the texture features include a mean, a standard deviation, entropy, a range, co-occurrence matrix of image grey levels in the background image.

9. The method of claim 8 , wherein transforming the texture features into a two-dimensional texture feature matrix comprises:

forming a texture feature vector using the texture features; and

concatenating the texture feature vector into the two-dimensional texture feature matrix.

10. The method of claim 1 , wherein the photograph includes one or more subjects in front of a background, wherein extracting a background image comprises:

locating borders of the one or more subjects; and

masking the one or more subjects, wherein the background image is extracted outside the borders of the one or more subjects.

11. The method of claim 1 , further comprising:

storing a set of background types in a computer memory;

training a hybrid deep learning model using sample portrait images comprising the background types, wherein the hybrid deep learning model includes the first convolution neural network, the second convolution neural network, and the third convolution neural network; and

obtaining the first weight, the second weight, and the third weight respectively associated with the first convolution neural network, the second convolution neural network, and the third convolution neural network.

12. The method of claim 1 , further comprising:

replacing at least a portion of the background image in the portrait photograph by another background image.