Systems for generating output based on orientation of an input image
An image depicting a portion of a user's body in a first orientation is processed using a classifier to determine a class associated with a second orientation similar to the first orientation. The determined class is associated with an angular value relative to a reference axis. To generate an output that depicts the item in association with the user's body, such as worn, while depicting the item in the correct orientation, a regression process is used based on a difference between the orientation in the input image and the orientation represented by the angular value for the determined class. Classification of an input image to determine an initial orientation associated with the class prior to using a regression process reduces the distance that a three-dimensional model representing the item is rotated when generating an output that aligns the item with the body of the user, resulting in improved accuracy.
1 . A system comprising:
one or more non-transitory memories storing computer-executable instructions; and
one or more hardware processors to execute the computer-executable instructions to:
receive a first image depicting a portion of a body of a user in a first orientation;
provide the first image to a classifier, wherein the classifier accesses a plurality of classes, each class of the plurality of classes representing a respective orientation of an item, and each class of the plurality of classes being associated with a respective angular value relative to a reference axis, and wherein the classifier determines a determined class of the plurality of classes having a second orientation that corresponds to the first orientation;
determine an angular value that is associated with the determined class;
determine a regression angle based on the first orientation and the angular value;
determine a third orientation of the item based on the regression angle, wherein the third orientation corresponds to the first orientation;
render a three-dimensional (3D) model representing the item at the third orientation; and
generate, based on the first image, the 3D model, and the regression angle, a second image that depicts the item in association with the portion of the body of the user, wherein the item is depicted in the third orientation.
2 . The system of claim 1 , wherein each class of the plurality of classes corresponds to a range of angles, the determined class corresponds to a midpoint of the range of angles, and the third orientation is determined based on a difference between an angle associated with the first orientation and the midpoint of the range of angles.
3 . The system of claim 1 , wherein each respective orientation and the third orientation comprise a respective quaternion value.
4 . A system comprising:
one or more non-transitory memories storing computer-executable instructions; and
one or more hardware processors to execute the computer-executable instructions to:
receive a first image depicting a portion of a body of a user in a first orientation;
determine, using a machine learning system that is trained to classify input images as corresponding to one or more of a plurality of classes based on characteristics of pixels in the input images, a second orientation of an item that corresponds to the first orientation, wherein the second orientation is associated with a first angular value relative to a reference axis;
determine a third orientation for the item based on the first orientation and the first angular value;
render a three-dimensional (3D) model representing the item at the third orientation; and
generate an output, based on the 3D model and the first image, that depicts the portion of the body of the user in association with the item having the third orientation.
5 . The system of claim 4 , wherein the machine learning system determines correspondence between the first orientation and a plurality of classes, wherein each class of the plurality of classes is associated with a respective orientation, and wherein the second orientation is determined based on correspondence between the first orientation and a first class of the plurality of classes.
6 . The system of claim 4 , wherein the machine learning system determines at least a first confidence score associated with correspondence between the first image and a first class associated with the second orientation and a second confidence score associated with correspondence between the first image and a second class associated with a fourth orientation, and wherein the second orientation is determined based on the first confidence score.
7 . The system of claim 4 , wherein each class of the plurality of classes corresponds to a respective image of a plurality of images, each image of the plurality of images is associated with a respective orientation, and wherein each image of the plurality of images is associated with a respective angular value that corresponds to the respective orientation.
8 . The system of claim 7 , wherein each image of the plurality of images depicts a respective side of the item, and wherein the item is depicted in the respective orientation.
9 . The system of claim 7 , wherein the plurality of images includes at least:
a second image depicting a top side of the item in the second orientation;
a third image depicting the top side of the item in a fourth orientation that differs from the second orientation;
a fourth image depicting a bottom side of the item in a fifth orientation;
a fifth image depicting the bottom side of the item in a sixth orientation that differs from the fifth orientation;
a sixth image depicting a front side of the item in a seventh orientation;
a seventh image depicting the front side of the item in an eighth orientation that differs from the seventh orientation;
an eighth image depicting the front side of the item in a ninth orientation that differs from the seventh orientation and the eighth orientation;
a ninth image depicting the front side of the item in a tenth orientation that differs from the seventh orientation, the eighth orientation, and the ninth orientation;
a tenth image depicting a rear side of the item in an eleventh orientation;
an eleventh image depicting the rear side of the item in a twelfth orientation that differs from the eleventh orientation;
a twelfth image depicting the rear side of the item in a thirteenth orientation that differs from the eleventh orientation and the twelfth orientation;
a thirteenth image depicting the rear side of the item in a fourteenth orientation that differs from the eleventh orientation, the twelfth orientation, and the thirteenth orientation;
a fourteenth image depicting a left side of the item in a fifteenth orientation;
a fifteenth image depicting the left side of the item in a sixteenth orientation that differs from the fifteenth orientation;
a sixteenth image depicting a right side of the item in a seventeenth orientation; and
a seventeenth image depicting the right side of the item in an eighteenth orientation that differs from the seventeenth orientation.
10 . The system of claim 4 , wherein the first angular value corresponds to a first angle between a reference axis and an axis associated with the second orientation, the system further comprising computer-executable instructions to:
determine a difference between the first angle and a second angle associated with the first orientation;
wherein the third orientation is determined based on the difference.
11 . The system of claim 4 , wherein the first image is received from a computing device at a first time, the system further comprising computer-executable instructions to:
at a second time, receive from the computing device a second image depicting the portion of the body of the user in a fourth orientation that differs from the first orientation;
determine, using the machine learning system, a fifth orientation for the item that corresponds to the fourth orientation, wherein the fifth orientation is associated with a second angular value that differs from the first angular value;
determine a sixth orientation for the item based on the fourth orientation and the second angular value;
render the three-dimensional (3D) model representing the item at the sixth orientation; and
generate a second output, based on the 3D model and the fourth orientation, that depicts the body of the user in association with the item having the sixth orientation.
12 . The system of claim 4 , wherein the second orientation and the third orientation each comprise a respective quaternion value.
13 . The system of claim 4 , wherein the item is a wearable item and the output depicts the item being worn by the portion of the body of the user.
14 . A system comprising:
one or more non-transitory memories storing computer-executable instructions; and
one or more hardware processors to execute the computer-executable instructions to:
receive a first image depicting a first object in a first orientation;
determine correspondence between the first orientation and a plurality of classes, wherein each class of the plurality of classes is associated with a respective orientation;
determine, based on the correspondence between the first orientation and a first class of the plurality of classes, a second orientation, for an item, that corresponds to the first image, wherein the second orientation is associated with a first angular value relative to a reference axis;
determine a third orientation, for the item, based on the first orientation, the second orientation, and the first angular value;
determine a three-dimensional (3D) model that represents a second object at the third orientation; and
generate an output based on the first image, the first angular value, and the 3D model, wherein the output depicts the second object in the third orientation.
15 . The system of claim 14 , further comprising computer-executable instructions to:
use a machine learning system that is trained to classify input images based on characteristics of pixels in the input images to determine at least:
a first confidence score associated with correspondence between the first image and the first class; and
a second confidence score associated with correspondence between the first image and a second class of the plurality of classes;
wherein the second orientation is further determined based on the first confidence score.
16 . The system of claim 14 , wherein each class of the plurality of classes is associated with a respective image of a plurality of images, wherein each image of the plurality of images depicts a respective side of the second object in a respective orientation, and wherein each image of the plurality of images differs from each other image by at least one of the respective side or the respective orientation.
17 . The system of claim 16 , wherein one or more of the respective orientation or the first angular value is represented as a quaternion value.
18 . The system of claim 14 , further comprising computer-executable instructions to:
determine a difference between the first angular value and the first orientation; and
determine the third orientation based at least in part on the difference.
19 . The system of claim 14 , wherein:
the first object is a portion of a body of a user;
the second object includes a wearable item; and
the output depicts the wearable item being worn on the portion of the body of the user.
20 . A system comprising:
one or more non-transitory memories storing computer-executable instructions; and
one or more hardware processors to execute the computer-executable instructions to:
receive a first image depicting a portion of a body of a user in a first orientation;
determine a second orientation of an item that corresponds to the first orientation, wherein the second orientation is associated with a first angular value that corresponds to a first angle between a reference axis and an axis associated with the second orientation;
determine a difference between the first angle and a second angle associated with the first orientation;
determine a third orientation for the item based on the first orientation, the first angular value, and the difference;
render a three-dimensional (3D) model representing the item at the third orientation; and
generate an output, based on the 3D model and the first image, that depicts the portion of the body of the user in association with the item having the third orientation.