Searching for and retrieving three-dimensional (3D) virtual objects

1 . A computer-implemented method comprising:

receiving a first query for one or more three-dimensional (3D) objects;

identifying at least one constraint from the first query, wherein the at least one constraint comprises a particular descriptor that specifies one or more aspects of matching 3D objects;

generating a second query comprising a set of descriptors identified by associating the first query with the one or more aspects of the matching 3D objects, the set of descriptors comprising the particular descriptor, wherein the one or more aspects specified by the particular descriptor include a region, wherein the region is a semantic region of a 3D object and the 3D object is divided into semantic regions associated with geometric node names during a preprocessing operation, wherein the geometric node names are used as semantic hints for fuzzy text matching to define the semantic regions, and wherein each descriptor in the set of descriptors comprises text;

performing a multidimensional search, using the second query, of a data repository comprising 3D content, to obtain one or more 3D object search results having features that correspond to the at least one constraint identified from the first query, wherein performing the multidimensional search comprises:

generating a plurality of 3D meshes by applying a text-to-3D artificial intelligence (AI) model to the text within respective descriptors from the set of descriptors; and

comparing the plurality of 3D meshes with representations of the 3D content in the data repository to identify the one or more 3D object search results based on shape similarities determined with spherical harmonic descriptors or silhouette similarities determined with sketch-based 3D search methods, the shape similarities or silhouette similarities being determined between at least one of the plurality of 3D meshes and the one or more 3D object search results by representing a global shape appearance, a local shape appearance, or a combination thereof of the plurality of 3D meshes as one or more fixed-size vectors by using machine learning (ML) techniques to encode at least one of the shape appearances using an image embedding model to transform images corresponding to the at least one of the shape appearances into a list of numbers, wherein the ML techniques include neural networks trained to represent similar images as similar numbers;

outputting a response to the first query that comprises the one or more 3D object search results; and

fine-tuning the one or more 3D object search results based on received weights to narrow down the one or more 3D object search results to a specific 3D model being sought by the first query.

2 . The computer-implemented method of claim 1 , wherein the identifying the at least one constraint from the first query comprises:

parsing the first query using a domain-specific parser to extract the particular descriptor; and

determining the one or more aspects specified in the particular descriptor.

3 . The computer-implemented method of claim 1 , wherein the one or more aspects specified in the particular descriptor further include one or more of geometry, material, articulation, animation, or at least one of physical models or simulation.

4 . The computer-implemented method of claim 1 , wherein the shape similarities are further determined using shape semantics with artificial intelligence (AI) recognition models.

5 . The computer-implemented method of claim 1 , further comprising:

calculating a fitness score of individual search results of the one or more 3D object search results by:

computing similarities, on an aspect-by-aspect basis, between features of the individual search results and the set of descriptors; and

obtaining the fitness score by summing the similarities computed between the features of the individual search results and the set of descriptors.

6 . The computer-implemented method of claim 1 , further comprising:

receiving input indicative of aspect weights of individual aspects of the one or more aspects; and

calculating a fitness score of a particular search result from the one or more 3D object search results by:

computing similarities, on an aspect-by-aspect basis, between features of the particular search result and the set of descriptors;

multiplying the similarities by the aspect weights of the individual aspects, providing multiplication products; and

obtaining the fitness score by summing the multiplication products.

7 . The computer-implemented method of claim 6 , wherein the outputting the response to the first query comprises:

ordering the one or more 3D object search results by the respective fitness scores, wherein the response comprises a subset of the one or more 3D object search results.

8 . The system of claim 7 , wherein the shape similarities are further determined using shape semantics with artificial intelligence (AI) recognition models.

9 . A system comprising:

a memory with instructions stored thereon; and

a processing device, coupled to the memory, the processing device configured to access the memory and execute the instructions, wherein the instructions cause the processing device to perform operations comprising:

receiving a first query for one or more three-dimensional (3D) objects;

identifying at least one constraint from the first query, wherein the at least one constraint comprises a particular descriptor that specifies one or more aspects of matching 3D objects;

generating a second query comprising a set of descriptors identified by associating the first query with the one or more aspects of the matching 3D objects, the set of descriptors comprising the particular descriptor, wherein the one or more aspects specified by the particular descriptor include a region, wherein the region is a semantic region of a 3D object and the 3D object is divided into semantic regions associated with geometric node names during a preprocessing operation, wherein the geometric node names are used as semantic hints for fuzzy text matching to define the semantic regions, and wherein each descriptor in the set of descriptors comprises text;

performing a multidimensional search, using the second query, of a data repository comprising 3D content, to obtain one or more 3D object search results having features that correspond to the at least one constraint identified from the first query, wherein performing the multidimensional search comprises:

generating a plurality of 3D meshes by applying a text-to-3D artificial intelligence (AI) model to the text within respective descriptors from the set of descriptors; and

comparing the plurality of 3D meshes with representations of the 3D content in the data repository to identify the one or more 3D object search results based on shape similarities determined with spherical harmonic descriptors or silhouette similarities determined with sketch-based 3D search methods, the shape similarities or silhouette similarities being determined between at least one of the plurality of 3D meshes and the one or more 3D object search results by representing a global shape appearance, a local shape appearance, or a combination thereof of the plurality of 3D meshes as one or more fixed-size vectors by using machine learning (ML) techniques to encode at least one of the shape appearances using an image embedding model to transform images corresponding to the at least one of the shape appearances into a list of numbers, wherein the ML techniques include neural networks trained to represent similar images as similar numbers;

outputting a response to the first query that comprises the one or more 3D object search results; and

fine-tuning the one or more 3D object search results based on received weights to narrow down the one or more 3D object search results to a specific 3D model being sought by the first query.

10 . The system of claim 9 , wherein identifying the at least one constraint from the first query comprises:

parsing the first query using a domain-specific parser to extract the particular descriptor; and

determining the one or more aspects specified by the particular descriptor.

11 . The system of claim 9 , wherein the instructions further cause the processing device to perform operations comprising:

calculating a fitness score of individual search results of the one or more 3D object search results by:

computing similarities, on an aspect-by-aspect basis, between features of the individual search results and the set of descriptors; and

obtaining the fitness score by summing the similarities computed between the features of the individual search results and the set of descriptors.

12 . The system of claim 9 , wherein the instructions further cause the processing device to perform operations comprising:

receiving input indicative of aspect weights of individual aspects of the one or more aspects; and

calculating a fitness score of a particular search result from the one or more 3D object search results by:

computing similarities, on an aspect-by-aspect basis, between features of the particular search result and the set of descriptors;

multiplying the similarities by the aspect weights of the individual aspects, providing multiplication products; and

obtaining the fitness score by summing the multiplication products.

13 . The system of claim 12 , wherein the outputting the response to the first query comprises:

ordering the one or more 3D object search results by the respective fitness scores, wherein the response comprises a subset of the one or more 3D object search results.

14 . The system of claim 9 , wherein the one or more aspects specified in the particular descriptor further include one or more of geometry, material, articulation, animation, or at least one of physical models or simulation.

15 . A non-transitory computer-readable medium with instructions stored thereon that, responsive to execution by a processing device, cause the processing device to perform operations comprising:

receiving a first query for one or more three-dimensional (3D) objects;

identifying at least one constraint from the first query, wherein the at least one constraint comprises a particular descriptor that specifies one or more aspects of matching 3D objects;

generating a second query comprising a set of descriptors identified by associating the first query with the one or more aspects of the matching 3D objects, the set of descriptors comprising the particular descriptor, wherein the one or more aspects specified by the particular descriptor include a region, wherein the region is a semantic region of a 3D object and the 3D object is divided into semantic regions associated with geometric node names during a preprocessing operation, wherein the geometric node names are used as semantic hints for fuzzy text matching to define the semantic regions, and wherein each descriptor in the set of descriptors comprises text;

performing a multidimensional search, using the second query, of a data repository comprising 3D content, to obtain one or more 3D object search results having features that correspond to the at least one constraint identified from the first query, wherein performing the multidimensional search comprises:

generating a plurality of 3D meshes by applying a text-to-3D artificial intelligence (AI) model to the text within respective descriptors from the set of descriptors; and

comparing the plurality of 3D meshes with representations of the 3D content in the data repository to identify the one or more 3D object search results based on shape similarities determined with spherical harmonic descriptors or silhouette similarities determined with sketch-based 3D search methods, the shape similarities or silhouette similarities being determined between at least one of the plurality of 3D meshes and the one or more 3D object search results by representing a global shape appearance, a local shape appearance, or a combination thereof of the plurality of 3D meshes as one or more fixed-size vectors by using machine learning (ML) techniques to encode at least one of the shape appearances using an image embedding model to transform images corresponding to the at least one of the shape appearances into a list of numbers, wherein the ML techniques include neural networks trained to represent similar images as similar numbers;

outputting a response to the first query that comprises the one or more 3D object search results; and

fine-tuning the one or more 3D object search results based on received weights to narrow down the one or more 3D object search results to a specific 3D model being sought by the first query.

16 . The non-transitory computer-readable medium of claim 15 , wherein the identifying the at least one constraint from the first query comprises:

parsing the first query using a domain-specific parser to extract the particular descriptor; and

determining the one or more aspects specified in the particular descriptor.

17 . The non-transitory computer-readable medium of claim 15 , wherein the operations further comprises:

calculating a fitness score of individual search results of the one or more 3D object search results by:

computing similarities, on an aspect-by-aspect basis, between features of the individual search results and the set of descriptors; and

obtaining the fitness score by summing the similarities computed between the features of the individual search results and the set of descriptors.

18 . The non-transitory computer-readable medium of claim 15 , wherein the operations further comprises:

receiving input indicative of aspect weights of individual aspects of the one or more aspects; and

calculating a fitness score of a particular search result from the one or more 3D object search results by:

computing similarities, on an aspect-by-aspect basis, between features of the particular search result and the set of descriptors;

multiplying the similarities by the aspect weights of the individual aspects, providing multiplication products; and

obtaining the fitness score by summing the multiplication products.

19 . The non-transitory computer-readable medium of claim 18 , wherein the outputting the response to the first query comprises:

ordering the one or more 3D object search results by the respective fitness scores, wherein the response comprises a subset of the one or more 3D object search results.

20 . The non-transitory computer-readable medium of claim 15 , wherein the shape similarities are further determined using shape semantics with artificial intelligence (AI) recognition models.