Image descriptor network with imposed hierarchical normalization
Techniques are disclosed for using and training a descriptor network. An image may be received and provided to the descriptor network. The descriptor network may generate an image descriptor based on the image. The image descriptor may include a set of elements distributed between a major vector comprising a first subset of the set of elements and a minor vector comprising a second subset of the set of elements. The second subset of the set of elements may include more elements than the first subset of the set of elements. A hierarchical normalization may be imposed onto the image descriptor by normalizing the major vector to a major normalization amount and normalizing the minor vector to a minor normalization amount. The minor normalization amount may be less than the major normalization amount.
1. A computer-implemented method comprising:
receiving a first image and a second image;
providing the first image and the second image to a descriptor network as input;
generating, using the descriptor network, a first image descriptor based on the first image and a second image descriptor based on the second image, the first image descriptor including a first set of elements distributed between a first major vector and a first minor vector, and the second image descriptor including a second set of elements distributed between a second major vector and a second minor vector;
computing a major distance between the first image and the second image based on the first major vector and the second major vector;
comparing the major distance to a threshold;
if the major distance is greater than the threshold, determining that the first image does not match the second image; and
if the major distance is less than the threshold:
computing a minor distance between the first image and the second image based on the first minor vector and the second minor vector;
comparing a sum of the major distance and the minor distance to the threshold;
if the sum is greater than the threshold, determining that the first image does not match the second image; and
if the sum is less than the threshold, determining that the first image matches the second image.
2. The computer-implemented method of claim 1 , wherein the first major vector comprises a first subset of the first set of elements, the first minor vector comprising a second subset of the first set of elements, the second major vector comprises a first subset of the second set of elements, and the second minor vector comprising a second subset of the second set of elements.
3. The computer-implemented method of claim 2 , wherein the second subset of the first set of elements includes more elements than the first subset of the first set of elements, and the second subset of the second set of elements includes more elements than the first subset of the second set of elements.
4. The computer-implemented method of claim 2 , wherein computing the major distance between the first image and the second image based on the first major vector and the second major vector includes:
computing a sum of squares of differences between the first subset of the first set of elements and the first subset of the second set of elements.
5. The computer-implemented method of claim 1 , further comprising:
imposing a hierarchical normalization onto the first image descriptor by:
normalizing the first major vector to a major normalization amount; and
normalizing the first minor vector to a minor normalization amount.
6. The computer-implemented method of claim 5 , further comprising:
imposing the hierarchical normalization onto the second image descriptor by:
normalizing the second major vector to the major normalization amount; and
normalizing the second minor vector to the minor normalization amount.
7. The computer-implemented method of claim 6 , wherein:
the major normalization amount is equal to 1−α; and
the minor normalization amount is equal to α, wherein α is less than 0.5.
8. The computer-implemented method of claim 1 , wherein the descriptor network is a neural network having a set of weights that are modifiable through a training process.
9. A system comprising:
one or more processors; and
a computer-readable medium storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving a first image and a second image;
providing the first image and the second image to a descriptor network as input;
generating, using the descriptor network, a first image descriptor based on the first image and a second image descriptor based on the second image, the first image descriptor including a first set of elements distributed between a first major vector and a first minor vector, and the second image descriptor including a second set of elements distributed between a second major vector and a second minor vector;
computing a major distance between the first image and the second image based on the first major vector and the second major vector;
comparing the major distance to a threshold;
if the major distance is greater than the threshold, determining that the first image does not match the second image; and
if the major distance is less than the threshold:
computing a minor distance between the first image and the second image based on the first minor vector and the second minor vector;
comparing a sum of the major distance and the minor distance to the threshold;
if the sum is greater than the threshold, determining that the first image does not match the second image; and
if the sum is less than the threshold, determining that the first image matches the second image.
10. The system of claim 9 , wherein the first major vector comprises a first subset of the first set of elements, the first minor vector comprising a second subset of the first set of elements, the second major vector comprises a first subset of the second set of elements, and the second minor vector comprising a second subset of the second set of elements.
11. The system of claim 10 , wherein the second subset of the first set of elements includes more elements than the first subset of the first set of elements, and the second subset of the second set of elements includes more elements than the first subset of the second set of elements.
12. The system of claim 10 , wherein computing the major distance between the first image and the second image based on the first major vector and the second major vector includes:
computing a sum of squares of differences between the first subset of the first set of elements and the first subset of the second set of elements.
13. The system of claim 9 , wherein the operations further comprise:
imposing a hierarchical normalization onto the first image descriptor by:
normalizing the first major vector to a major normalization amount; and
normalizing the first minor vector to a minor normalization amount.
14. The system of claim 13 , wherein the operations further comprise:
imposing the hierarchical normalization onto the second image descriptor by:
normalizing the second major vector to the major normalization amount; and
normalizing the second minor vector to the minor normalization amount.
15. The system of claim 14 , wherein:
the major normalization amount is equal to 1−α; and
the minor normalization amount is equal to α, wherein α is less than 0.5.
16. The system of claim 9 , wherein the descriptor network is a neural network having a set of weights that are modifiable through a training process.
17. A non-transitory machine-readable medium comprising instruction that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving a first image and a second image;
providing the first image and the second image to a descriptor network as input;
generating, using the descriptor network, a first image descriptor based on the first image and a second image descriptor based on the second image, the first image descriptor including a first set of elements distributed between a first major vector and a first minor vector, and the second image descriptor including a second set of elements distributed between a second major vector and a second minor vector;
computing a major distance between the first image and the second image based on the first major vector and the second major vector;
comparing the major distance to a threshold;
if the major distance is greater than the threshold, determining that the first image does not match the second image; and
if the major distance is less than the threshold:
computing a minor distance between the first image and the second image based on the first minor vector and the second minor vector;
comparing a sum of the major distance and the minor distance to the threshold;
if the sum is greater than the threshold, determining that the first image does not match the second image; and
if the sum is less than the threshold, determining that the first image matches the second image.
18. The non-transitory machine-readable medium of claim 17 , wherein the first major vector comprises a first subset of the first set of elements, the first minor vector comprising a second subset of the first set of elements, the second major vector comprises a first subset of the second set of elements, and the second minor vector comprising a second subset of the second set of elements.
19. The non-transitory machine-readable medium of claim 18 , wherein the second subset of the first set of elements includes more elements than the first subset of the first set of elements, and the second subset of the second set of elements includes more elements than the first subset of the second set of elements.
20. The non-transitory machine-readable medium of claim 18 , wherein computing the major distance between the first image and the second image based on the first major vector and the second major vector includes:
computing a sum of squares of differences between the first subset of the first set of elements and the first subset of the second set of elements.