Method, computer-readable storage medium, and system for predicting or estimating physical attributes of products shown in images
The disclosed technology includes a computer-implemented technique for predicting or estimating physical attributes of products depicted in images is disclosed. The method includes receiving one or more images—either photographs of real-world products or synthetic images generated from user-provided descriptions—processing the images using one or more machine learning (ML) models trained to detect and measure physical characteristics, and predicting or estimating measures such as volume, weight, and dimensions. The system can identify product categories, select appropriate ML models, and generate natural language descriptions of predicted attributes for user display. Training datasets may include both real and synthetic images with known attributes, and user feedback can be incorporated to improve model accuracy.
1 . A computer-implemented method for predicting or estimating physical attributes of a product shown in an image, the method comprising:
receiving, by at least one hardware processor, one or more images of the product,
wherein the one or more images reflect a version of the product with real-world materials, dimensions, and manufacturability limits,
processing, by the at least one hardware processor, the one or more images using one or more machine learning (ML) models trained to detect physical characteristics and measure physical attributes of conceptual and real-world products shown in images,
wherein the physical characteristics are distinguishable features of conceptual or real-world products shown in images, and
wherein each of the one or more images are of the product with multiple physical attributes; and
predicting or estimating, using the one or more ML models, measures of the multiple physical attributes of the product shown in the one or more images,
wherein the one or more ML models are configured to predict measures of physical attributes of conceptual products that do not exist in the real world and estimate measures of physical attributes of real-world products, and
wherein the prediction or estimation of the measures of the physical attributes includes measures for volume, weight, and dimensions of at least a portion of the product; and
configuring a manufacturing process to produce a purchasable, physical product based on the measures of the physical attributes of the product shown in the one or more images.
2 . The method of claim 1 , wherein the product is a conceptual product, and wherein the method further comprises, prior to receiving the one or more images of the conceptual product:
receiving user input including a description of the conceptual product entered to a user interface of a user device; and
generating the one or more images of the conceptual product as an output of a generative model based on a prompt input including the description of the conceptual product,
wherein the product shown in the one or more images is manufacturable into the purchasable, physical product, and
wherein the measures of the multiple physical attributes of the conceptual product shown in the one or more images are predicted by the one or more ML models.
3 . The method of claim 1 , further comprising, prior to receiving the one or more images of the product:
receiving user input including one or more photographs uploaded via a user interface,
wherein the product shown in the one or more images is a real-world product shown in the one or more photographs, and
wherein the measures of the multiple physical attributes of the real-world product shown in the one or more photographs are estimated by the one or more ML models.
4 . The method of claim 1 , wherein processing the one or more images using the one or more ML models comprises:
processing different images of the conceptual product showing different physical attributes of the product.
5 . The method of claim 1 , further comprising:
training the one or more ML models using a dataset including real-world photographs and images of synthetic products with known physical characteristics of a category of products,
wherein the product belongs to the category of products.
6 . The method of claim 1 , further comprising:
generating, using a generative model, a natural language description of the predicted or estimated measures of the physical attributes of the product; and
causing a display of a user device to present the natural language description of the predicted or estimated measures of the physical attributes of the product including measures for the volume, the weight, and the dimensions of the conceptual product.
7 . The method of claim 1 , further comprising, prior to predicting or estimating the physical attributes of the product:
identifying a product category for the product based on the detected physical characteristics of the product; and
selecting the one or more ML models from among multiple ML models configured to predict or estimate physical attributes for different product categories,
wherein the one or more selected ML models are configured to predict or estimate physical attributes of products in the product category.
8 . The method of claim 1 , further comprising:
receiving user feedback regarding predicted or estimated product attributes of the product; and
incorporating the user feedback into a training dataset configured to improve accuracy of the one or more ML models in predicting or estimating physical attributes of products shown in images.
9 . A non-transitory, computer-readable storage medium comprising instructions recorded thereon, wherein the instructions, when executed by at least one data processor of a system, cause the system to:
process one or more images using an artificial intelligence system configured to detect physical characteristics and physical attributes of products shown in images,
wherein the one or more images reflect a version of the product with real-world materials, dimensions, and manufacturability limits that are manufacturable into physical products,
wherein the artificial intelligence system includes one or more models that are trained on images of products with multiple or different physical attributes to detect physical characteristics and measure physical attributes of conceptual products that do not yet exist and real-world products shown in images;
predict or estimate, by the artificial intelligence system, one or more physical attributes of the product shown in the one or more images,
wherein the artificial intelligence system predicts the physical attributes of the product including measures for volume, weight, and dimensions; and
configure a manufacturing process to produce a purchasable, physical product based on the measures of the physical attributes of the product in the one or more images.
10 . The non-transitory, computer-readable storage medium of claim 9 , wherein the one or more images are of the product having different physical attributes.
11 . The non-transitory, computer-readable storage medium of claim 9 , wherein the product is a conceptual product, and wherein the system is further caused to, prior to the one or more images being processed using an artificial intelligence system:
receive user input including a text-based description of the conceptual product entered at a user interface of a user device; and
generate the one or more images as an output of a generative model based on a prompt input including the text-based description of the conceptual product,
wherein the product shown in the one or more images is the conceptual product that does not exist and is manufacturable into a physical product.
12 . The non-transitory, computer-readable storage medium of claim 9 , wherein the one or more images that show the product are photographs of real-world products.
13 . The non-transitory, computer-readable storage medium of claim 9 , wherein the system is further caused to:
train a model of the artificial intelligence system using a dataset comprising real-world images and synthetic images of products with known physical attributes.
14 . The non-transitory, computer-readable storage medium of claim 9 , wherein the product is a conceptual product, and wherein the system is further caused to:
generate, using a generative model, a natural language description of the predicted physical attributes of at least a portion of the conceptual product; and
cause a user device to present the natural language description of the predicted physical attributes of at least the portion of the conceptual product including measures for the volume, the weight, and the dimensions.
15 . The non-transitory, computer-readable storage medium of claim 9 , wherein the product is a conceptual product, and wherein the system is further caused to, prior to the physical attributes of the conceptual product being predicted or estimated:
identify a product category of the conceptual product based on the detected physical characteristics of the product; and
select a machine learning (ML) model from among multiple ML models configured to predict physical attributes of different product categories,
wherein the selected ML model is configured to predict physical attributes of products in the product category.
16 . The non-transitory, computer-readable storage medium of claim 9 , wherein the product is a conceptual product, and wherein the system is further caused to:
incorporate user feedback into a training dataset configured to improve accuracy of one or more models of the artificial intelligence system in predicting physical attributes of conceptual products.
17 . A system comprising:
at least one hardware processor; and
at least one non-transitory memory storing instructions, which, when executed by the at least one hardware processor, cause the system to:
process one or more images using a machine learning (ML) model trained to predict or estimate physical attributes of conceptual products or real-world products shown in images,
wherein the one or more images reflect a version of a conceptual product with real-world materials, dimensions, and manufacturability limits,
wherein the conceptual product does not exist in the real-world and is manufacturable into a purchasable, physical product,
wherein the ML model is trained on images of products with multiple or different physical attributes;
determine, by the ML model, measures of physical attributes of the conceptual product shown in the one or more images,
wherein the measures of physical attributes include volume, weight, or dimensions of the conceptual product; and
generate product data for producing the purchasable, physical product based on the measures of the physical attributes of the conceptual product.
18 . The system of claim 17 , wherein the conceptual product is rendered as a computer-generated synthetic product in the one or more images, and wherein to determine the physical attributes of the synthetic product comprises further causing the system to:
predict the physical attributes of a real-world product that represents the synthetic product shown in the one or more images.
19 . The system of claim 17 , wherein the conceptual product represents a real-world product, and wherein to determine the physical attributes of the real-world product comprises causing the system to:
estimate the physical attributes of a real-world product shown in one or more photographs.
20 . The system of claim 17 , wherein the system is further caused to:
train the ML model using a dataset comprising real-world images and images of synthetic products with known physical characteristics.