Compatibility-based identification of incompatible objects in digital representations of real-world environments

1. One or more non-transitory computer readable storage media having program instructions stored thereon that, when executed by a processor, direct a computing apparatus to:

analyze, utilizing one or more neural networks, a viewpoint depicting a two-dimensional digital representation of a real-world environment to identify real-world objects within the viewpoint;

identify, utilizing a part-based alignment algorithm, a three-dimensional (3D) model for each of the identified real-world objects in the viewpoint along with pose and scale;

utilizing the identified 3D models, determine an incompatibility of each object of the identified real-world objects in the viewpoint based on a weighted combination of:

a style incompatibility of the object in relation to other of the identified real-world objects in the viewpoint, and

a color incompatibility of the object in relation to a background of the viewpoint; and

determine a least compatible object in the viewpoint based on the incompatibility of each object of the identified real-world objects in the viewpoint.

2. The one or more non-transitory computer readable storage media of claim 1 , wherein the program instructions, when executed by the processor, further direct the computing apparatus to:

for each object of the identified real-world objects in the viewpoint:

compute, using a style compatibility algorithm, the style incompatibility of the object,

wherein the style incompatibility comprises a score determined by comparing geometric features of a 3D model corresponding to the object with geometric features of 3D models corresponding to the other of the identified real-world objects in the viewpoint.

3. The one or more non-transitory computer readable storage media of claim 1 , wherein the program instructions, when executed by the processor, further direct the computing apparatus to:

for each object of the identified real-world objects in the viewpoint:

compute, using a color compatibility algorithm, the color incompatibility of the object,

wherein the color incompatibility comprises a score determined by comparing a color of the object to a theme of colors extracted from the background of the viewpoint.

4. The one or more non-transitory computer readable storage media of claim 1 , wherein the program instructions, when executed by the processor, further direct the computing apparatus to:

generate a product recommendation based on a weighted combination of:

a style compatibility of the product recommendation in relation to a set of surrounding real-world objects in the viewpoint other than the least compatible object; and

a color compatibility of the product recommendation in relation to the background of the viewpoint.

5. The one or more non-transitory computer readable storage media of claim 4 , wherein the program instructions, when executed by the processor, further direct the computing apparatus to:

for each candidate product of multiple candidate products in a product data store:

compute, using a style compatibility algorithm, the style compatibility of the candidate product,

wherein the style compatibility comprises a score determined by comparing geometric features of a 3D model corresponding to the candidate product with geometric features of 3D models corresponding to the set of surrounding real-world objects in the viewpoint.

6. The one or more non-transitory computer readable storage media of claim 5 , wherein the multiple candidate products have a same product type as the least compatible object.

7. The one or more non-transitory computer readable storage media of claim 4 , wherein the program instructions, when executed by the processor, further direct the computing apparatus to:

for each candidate product of multiple candidate products in a product data store:

compute, using a color compatibility algorithm, the color compatibility of the candidate product,

wherein the color compatibility comprises a score determined by comparing a color of the candidate product to a theme of colors extracted from the background of the viewpoint.

8. The one or more non-transitory computer readable storage media of claim 4 , wherein the program instructions, when executed by the processor, further direct the computing apparatus to:

identify, utilizing the part-based alignment algorithm, a pose and scale of the least compatible object in the viewpoint;

generate a two-dimensional representation of the product recommendation with a pose and scale matching the pose and scale of the least compatible object; and

generate a modified viewpoint by:

removing the least compatible object from the viewpoint; and

embedding the two-dimensional representation of the product recommendation positioned at a location of the removed least compatible object.

9. The one or more non-transitory computer readable storage media of claim 8 , wherein the program instructions, when executed by the processor, further direct the computing apparatus to:

generate a targeted communication including the modified viewpoint; and

provide the targeted communication to a user associated with the viewpoint via one or more marketing channels.

10. The one or more non-transitory computer readable storage media of claim 1 , wherein the program instructions, when executed by the processor, further direct the computing apparatus to:

capture, based on visual data received from a client device, the viewpoint depicting the two-dimensional digital representation of the real-world environment,

wherein the visual data comprises one or more of a digital image, a video, or a digital representation of an augmented reality application session executing on the client device.

11. The one or more non-transitory computer readable storage media of claim 10 , wherein the visual data comprises the digital representation of the augmented reality application session and wherein the program instructions, when executed by the processor, further direct the computing apparatus to:

detect a time instant during the augmented reality application session where the client device satisfies a stillness threshold; and

capture a frame from the digital representation of the augmented reality application session at the time instant.

12. The one or more non-transitory computer readable storage media of claim 10 , wherein the visual data comprises the digital representation of the augmented reality application session and wherein the program instructions, when executed by the processor, further direct the computing apparatus to:

generate an augmented reality representation of a product recommendation; and

provide the augmented reality representation of the product recommendation as an overlay within the augmented reality application session.

13. The one or more non-transitory computer readable storage media of claim 1 , wherein at least one of the one or more neural networks utilized to detect the real-world objects within the viewpoint comprises a region-based convolutional neural network configured to generate proposed regions of the viewpoint with corresponding probabilities of containing objects.

14. A digital content campaign platform comprising:

a processing system; and

one or more computer readable storage media operatively coupled with the processing system, the one or more computer readable storage media storing;

a three-dimensional (3D) model data store including a plurality of 3D models; and

program instructions that, when executed by the processing system, direct the digital content campaign platform to:

analyze, utilizing a region-based convolutional neural network, a viewpoint depicting a two-dimensional digital representation of a real-world environment to identify real-world objects within the viewpoint;

identify, utilizing a part-based alignment algorithm, a three-dimensional (3D) model from the 3D model data store for each of the identified real-world objects in the viewpoint along with a pose and scale of the real-world objects in the viewpoint;

determine, utilizing the identified 3D models, an incompatibility of the real-world objects in the viewpoint based on a style incompatibility of the real-world objects in relation to each other and a color incompatibility of the real-world objects in relation to a background of the viewpoint;

identify, utilizing the identified 3D models, a least compatible object in the viewpoint based on the style incompatibility of the real-world objects in relation to each other and the color incompatibility in relation to a background of the viewpoint; and

select at least one product recommendation based on style compatibility of candidate products in relation to the real-world objects and color compatibility of the candidate products in relation to the background of the viewpoint.

15. The digital content campaign platform of claim 14 , wherein the program instructions, when executed by the processing system, direct the digital content campaign platform to:

identify, utilizing the part-based alignment algorithm, a pose and scale of the least compatible object in the viewpoint;

generate a two-dimensional representation of the at least one product recommendation with a pose and scale matching the pose and scale of the least compatible object; and

modify the viewpoint by replacing the least compatible object in the viewpoint with the two-dimensional representation of the at least one product recommendation.

16. The digital content campaign platform of claim 14 , wherein the program instructions, when executed by the processing system, direct the digital content campaign platform to:

generate a targeted communication including the modified viewpoint; and

provide the targeted communication to a user associated with the viewpoint via one or more marketing channels.

17. The digital content campaign platform of claim 14 , wherein the program instructions, when executed by the processing system, direct the digital content campaign platform to:

capture the viewpoint depicting the two-dimensional digital representation of the real-world environment based on visual data received from a client device associated with a user.

18. A method comprising:

analyzing, utilizing one or more neural networks, a viewpoint depicting a two-dimensional digital representation of a real-world environment to detect real-world objects within the viewpoint;

identifying, utilizing a part-based alignment algorithm, a three-dimensional (3D) model for each of the identified real-world objects in the viewpoint along with pose and scale;

utilizing the identified 3D models, determining an incompatibility of each object of the identified real-world objects in the viewpoint based on a weighted combination of:

a style incompatibility of the object in relation to other of the identified real-world objects in the viewpoint, and

a color incompatibility of the object in relation to a background of the viewpoint; and

determining a least compatible object in the viewpoint based on the incompatibility of each object of the identified real-world objects in the viewpoint.

19. The method of claim 18 , further comprising:

for each object of the identified real-world objects in the viewpoint, computing, using a style compatibility algorithm, the style incompatibility of the object, wherein the style incompatibility comprises a score determined by comparing geometric features of a 3D model corresponding to the object with geometric features of 3D models corresponding to the other of the identified real-world objects in the viewpoint; and

for each object of the identified real-world objects in the viewpoint, computing, using a color compatibility algorithm, the color incompatibility of the object, wherein the color incompatibility comprises a score determined by comparing a color of the object to a theme of colors extracted from the background of the viewpoint.

20. The method of claim 18 , further comprising:

generating a product recommendation based on a weighted combination of a style compatibility of the product recommendation in relation to a set of surrounding real-world objects in the viewpoint other than the least compatible object and a color compatibility of the at least one product recommendation in relation to the background of the viewpoint.