Statistical shape and appearance modeling for volumetric geometry and intensity data
An image-based approach for statistical shape and appearance modeling includes re-orienting (rotating/translating) training masks to align the training masks with a reference mask to provide corresponding re-orientation parameters, where the training masks represent 3-dimensional shapes of a population of objects and the reference mask represents a 3-dimensional shape of a reference object, deforming the re-oriented training masks based on the reference mask to provide displacement fields indicative of differences between a 3-dimensional shape of the reference mask and 3-dimensional shapes of the re-oriented training masks, re-orienting training backgrounds based on the re-orientation parameters, where the training backgrounds represent volumetric intensity data of the 3-dimensional images of the objects, deforming the re-oriented training backgrounds based on the displacement fields, combining the deformed training backgrounds and the displacement fields, and reducing a dimensionality of the combined deformed training backgrounds and displacement fields to provide a statistical shape and appearance model.
1. A machine-implemented method, comprising:
receiving a reference mask, training masks, and training backgrounds, wherein the reference mask represents a 3-dimensional shape of a reference object, wherein the training masks represent external surfaces of training objects, wherein the training backgrounds comprise intensity data of the training objects, and wherein the intensity data represents one or more of color and texture;
re-orienting the training masks to align the training masks with the reference mask to provide re-orientation parameters for the training masks;
deforming the re-oriented training masks based on the reference mask to provide displacement fields indicative of differences between a 3-dimensional shape of the reference mask and 3-dimensional shapes of the re-oriented training masks;
re-orienting the training backgrounds based on the re-orientation parameters determined for the training masks to provide re-oriented training backgrounds;
deforming the re-oriented training backgrounds based on the displacement fields of the respective training masks to provide deformed training backgrounds;
combining the deformed training backgrounds and the displacement fields; and
reducing a dimensionality of the combined deformed training backgrounds and displacement fields to provide a statistical shape and appearance model (SSAM).
2. The method of claim 1 , wherein the re-orienting training masks comprises:
converting the training masks and the reference mask to triangulated meshes; and
re-orienting the triangulated meshes of the training masks to the triangulated meshes of the reference mask based on an iterative closest point method to provide the re-orientation parameters to include iterative closest point transformation matrices.
3. The method of claim 1 , wherein the deforming the re-oriented training masks comprises:
computing signed distance maps of the re-oriented training masks and the reference mask; and
computing the displacement fields based on the signed distance maps.
4. The method of claim 3 , wherein the computing the displacement fields comprises computing the displacement fields with a diffeomorphic demons deformable method.
5. The method of claim 3 , wherein the deforming the re-oriented training masks further comprises representing the displacement fields as images having vector-type pixels with N elements, where N is a dimensionality of the reference mask.
6. The method of claim 3 , wherein:
the combining comprises combining the deformed training backgrounds and the displacement fields in a matrix; and
the method further comprises scaling the matrix, wherein the scaling comprises computing means of rows of the matrix and dividing the rows of the matrix by the means of the respective rows.
7. The method of claim 6 , wherein the reducing the dimensionality comprises:
reducing a dimensionality of the matrix based on principal component analysis to provide the statistical shape and appearance model to include a mean dataset, modes of variation, and weighting, wherein the modes of variation and the weighting reflect variations in the objects relative to a mean of the objects.
8. The method of claim 3 , further comprising:
iteratively refining the displacement fields and the deformed training backgrounds based on one another to provide a refined set of deformed training backgrounds and a refined set of displacement fields;
wherein the combining comprises combining the refined set of deformed training backgrounds and the refined set of displacement fields.
9. The method of claim 1 , further comprising using the statistical shape and appearance model to perform one or more of:
generate virtual objects based on variations amongst the training masks and the training backgrounds;
analyze variability in shape and material properties of the training objects based on variations amongst the training masks and the training backgrounds; and
predict missing information from a partial dataset of masks and backgrounds extracted from 3-dimensional images of other objects.
10. A non-transitory computer readable medium encoded with a computer program that comprises instructions to cause a processor to:
compute signed distance maps based on training masks and of a reference mask, wherein the training masks represent 3-dimensional shapes of a population of objects extracted from 3-dimensional images of the objects, and wherein the reference mask represents a 3-dimensional shape of a reference object;
compute displacement fields for the training masks based on the respective signed distance maps, wherein the displacement fields represent geometric differences between the reference mask and the training masks;
deform training backgrounds based on the displacement fields computed for the training masks to provide deformed training backgrounds, wherein the training backgrounds comprise volumetric intensity data extracted from the 3-dimensional images of the objects, and wherein the intensity data represents one or more of color and texture;
combine the deformed training backgrounds and the displacement fields; and
reduce a dimensionality of the combined deformed training backgrounds and displacement fields to provide a statistical shape and appearance model (SSAM).
11. The non-transitory computer readable medium of claim 10 , further comprising instructions to cause the processor to:
compute the displacement fields with a diffeomorphic demons deformable method.
12. The non-transitory computer readable medium of claim 10 , further comprising instructions to cause the processor to:
represent the displacement fields as images having vector-type pixels with N elements, where N is a dimensionality of the reference mask.
13. The non-transitory computer readable medium of claim 10 , further comprising instructions to cause the processor to:
combine the deformed training backgrounds and the displacement fields in a matrix.
14. The non-transitory computer readable medium of claim 13 , further comprising instructions to cause the processor to scale the matrix, including to:
compute means of rows of the matrix; and
divide the rows of the matrix by the means of the respective rows.
15. The non-transitory computer readable medium of claim 13 , further comprising instructions to cause the processor to:
reduce a dimensionality of the matrix based on principal component analysis to provide the statistical shape and appearance model to include a mean dataset, modes of variation, and weighting, wherein the modes of variation and the weighting reflect variations in the objects relative to a mean of the objects.
16. The non-transitory computer readable medium of claim 10 , further comprising instructions to cause the processor to use the statistical shape and appearance model to perform one or more of:
generate virtual objects based on variations amongst the training masks and the training backgrounds;
analyze variability in shape and material properties of the population of objects based on variations amongst the training masks and the training backgrounds; and
predict missing information from a partial dataset of masks and backgrounds extracted from 3-dimensional images of other objects.
17. An integrated circuit (IC) device, comprising:
re-orientation circuitry to align training masks with a reference mask to provide re-orientation parameters for the training masks, and to re-orient training backgrounds based on the re-orientation parameters determined for the training masks to provide re-oriented training backgrounds, wherein the training masks represent external surfaces of training objects, wherein the training backgrounds comprise intensity data of the training objects, and wherein the intensity data represents one or more of color and texture;
deformable registration circuitry to deform the re-oriented training masks based on the reference mask to provide displacement fields indicative of differences between a 3-dimensional shape of the reference mask and 3-dimensional shapes of the re-oriented training masks, and to deform the re-oriented training backgrounds based on the displacement fields of the respective training masks to provide deformed training backgrounds;
combining circuitry to combine the deformed training backgrounds and the displacement fields; and
dimensionality reduction circuitry configured to reduce a dimensionality of the combined deformed training backgrounds and displacement fields to provide a statistical shape and appearance model (SSAM).
18. The IC device of claim 17 , wherein the deformable registration circuitry is further configured to deform the re-oriented training masks and the re-oriented training backgrounds by:
deformably registering signed distance maps of the re-oriented training masks to provide an initial set of displacement fields;
deformably registering background intensity values of the re-oriented training backgrounds based on the initial set of displacement fields to provide an initial set of deformed training backgrounds and a refined set of displacement fields;
generating a revised set of displacement fields based on the revised set of displacement fields; and
generating a revised set of deformed training backgrounds based on the revised set of displacement fields,
deform the set of re-oriented training backgrounds based on the refined set of displacement fields to provide a final set of deformed training backgrounds.
19. The IC device of claim 17 , further comprising SSAM application circuitry configured to perform one or more of:
generate virtual objects based on variations amongst the training masks and the training backgrounds;
analyze variability in shape and material properties of the training objects based on variations amongst the training masks and the training backgrounds; and
predict missing information from a partial dataset of masks and backgrounds extracted from 3-dimensional images of other objects.