Interpretable embeddings
This specification is generally directed to techniques for creating reduced-dimensionality embeddings (e.g., embedding layers of neural networks) with dimensions that are interpretable by and/or are meaningful to humans. In various implementations, a datum may sampled from a document. A dimensionality reduction process may be performed based on the sampled datum to generate a semantically-interpretable embedding having a number of individually-interpretable dimensions. The dimensionality reduction process may include: analyzing the sampled datum according to a number of distinct semantic queries to determine respective numeric solutions. The number of distinct semantic queries may correspond to the number of individually-interpretable dimensions. Each numeric solution may offer an inconclusive clue about the sampled datum. The dimensionality reduction process may also include populating the dimensions of the semantically-interpretable embedding with respective numeric solutions.
1. A method for generating an explanation of a visual hierarchy of a digital image, the method implemented using one or more processors and comprising:
processing the digital image using at least part of a machine learning pipeline to yield a first latent embedding layer representation of the digital image;
using the first latent embedding layer, generating a first semantically-interpretable embedding having a number of individually-interpretable dimensions, including
populating the dimensions of the first semantically-interpretable embedding with respective probabilities of visual features of a first level of granularity being present in the digital image;
processing the first latent embedding layer using at least part of the machine learning pipeline to yield a second latent embedding layer representation of the digital image;
using the second latent embedding layer, generating a second semantically-interpretable embedding having a number of individually-interpretable dimensions, including populating the dimensions of the second semantically-interpretable embedding with respective probabilities of visual features of a second level of granularity being present in the digital image;
processing the second latent embedding layer using at least part of the machine learning pipeline to yield a third latent embedding layer representation of the digital image; and
using the third latent embedding layer, generating a third semantically-interpretable embedding having a number of individually-interpretable dimensions, including populating the dimensions of the third semantically-interpretable embedding with respective probabilities of visual features of a third level of granularity being present in the digital image.
2. The method of claim 1 , wherein the second level of granularity is greater than the first level of granularity.
3. The method of claim 2 , wherein the third level of granularity is greater than the second level of granularity.
4. The method of claim 3 , wherein the visual features of the first level of granularity include lines.
5. The method of claim 4 , wherein the visual features of the second or third level of granularity include one or more shapes selected from rectangles, squares, circles, and rhombuses.
6. The method of claim 4 , wherein the visual features of the second or third level of granularity include colors.
7. A system for generating an explanation of a visual hierarchy of a digital image, the system comprising one or more processors and memory storing instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to:
process the digital image using at least part of a machine learning pipeline to yield a first latent embedding layer representation of the digital image;
perform a dimensionality reduction process using the digital image to generate the first latent embedding layer, generating a first semantically-interpretable embedding having a number of individually-interpretable dimensions, including populating the dimensions of the first semantically-interpretable embedding with respective probabilities of visual features of a first level of granularity being present in the digital image;
process the first latent embedding layer using at least part of the machine learning pipeline to yield a second latent embedding layer representation of the digital image;
using the second latent embedding layer, generate a second semantically-interpretable embedding having a number of individually-interpretable dimensions, including populating the dimensions of the second semantically-interpretable embedding with respective probabilities of visual features of a second level of granularity being present in the digital image;
process the second latent embedding layer using at least part of the machine learning pipeline to yield a third latent embedding layer representation of the digital image; and
using the third latent embedding layer, generate a third semantically-interpretable embedding having a number of individually-interpretable dimensions, including populating the dimensions of the third semantically-interpretable embedding with respective probabilities of visual features of a third level of granularity being present in the digital image.
8. The system of claim 7 , wherein the second level of granularity is greater than the first level of granularity.
9. The system of claim 8 , wherein the third level of granularity is greater than the second level of granularity.
10. The system of claim 9 , wherein the visual features of the first level of granularity include lines.
11. The system of claim 10 , wherein the visual features of the second or third level of granularity include one or more shapes selected from rectangles, squares, circles, and rhombuses.
12. The system of claim 10 , wherein the visual features of the second or third level of granularity include colors.
13. At least one non-transitory computer-readable medium for generating an explanation of a visual hierarchy of a digital image, the medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to:
process the digital image using at least part of a machine learning pipeline to yield a first latent embedding layer representation of the digital image;
perform a dimensionality reduction process using the digital image to generate the first latent embedding layer, generating a first semantically-interpretable embedding having a number of individually-interpretable dimensions, including populating the dimensions of the first semantically-interpretable embedding with respective probabilities of visual features of a first level of granularity being present in the digital image;
process the first latent embedding layer using at least part of the machine learning pipeline to yield a second latent embedding layer representation of the digital image;
using the second latent embedding layer, generate a second semantically-interpretable embedding having a number of individually-interpretable dimensions, including populating the dimensions of the second semantically-interpretable embedding with respective probabilities of visual features of a second level of granularity being present in the digital image;
process the second latent embedding layer using at least part of the machine learning pipeline to yield a third latent embedding layer representation of the digital image; and
using the third latent embedding layer, generate a third semantically-interpretable embedding having a number of individually-interpretable dimensions, including populating the dimensions of the third semantically-interpretable embedding with respective probabilities of visual features of a third level of granularity being present in the digital image.
14. The at least one non-transitory computer-readable medium of claim 13 , wherein the second level of granularity is greater than the first level of granularity.
15. The at least one non-transitory computer-readable medium of claim 14 , wherein the third level of granularity is greater than the second level of granularity.
16. The at least one non-transitory computer-readable medium of claim 15 , wherein the visual features of the first level of granularity include lines.
17. The at least one non-transitory computer-readable medium of claim 16 , wherein the visual features of the second or third level of granularity include one or more shapes selected from rectangles, squares, circles, and rhombuses.
18. The at least one non-transitory computer-readable medium of claim 16 , wherein the visual features of the second or third level of granularity include colors.