Entity identification using machine learning
Methods, systems, and apparatus, including computer programs encoded on computer storage media for identification and re-identification of fish. In some implementations, first media representative of aquatic cargo is received. Second media based on the first media is generated, wherein a resolution of the second media is higher than a resolution of the first media. A cropped representation of the second media is generated. The cropped representation is provided to the machine learning model. In response to providing the cropped representation to the machine learning model, an embedding representing the cropped representation is generated using the machine learning model. The embedding is mapped to a high dimensional space. Data identifying the aquatic cargo is provided to a database, wherein the data identifying the aquatic cargo comprises an identifier of the aquatic cargo, the embedding, and a mapped region of the high dimensional space.
1. A method, performed by one or more processors, the method comprising:
receiving, by the one or more processors, multiple images of a particular fish;
generating, by the one or more processors and using the multiple images of the particular fish, a synthetic image of the particular fish that has a higher resolution than any of the received images of the particular fish;
generating, by the one or more processors, a cropped representation of the synthetic image, such that the cropped representation includes a set of anatomical key points of the particular fish;
obtaining, by the one or more processors from a machine learning model, an embedding for the cropped representation;
generating a cluster identifier based on clustering the embedding in a multi-dimensional space; and
providing, by the one or more processors, to a database, data identifying the cluster identifier for the particular fish.
2. The method of claim 1 , wherein generating the cluster identifier comprises:
determining, by the one or more processors, that the embedding matches a previously-generated embedding; and
determining, as a result, that the particular fish associated with the embedding has been re-identified.
3. The method of claim 1 , wherein generating the cluster identifier comprises:
determining, by the one or more processors, that the embedding does not match any previously-generated embedding; and
determining, as a result, that the particular fish associated with the embedding is newly identified.
4. The method of claim 1 , wherein the cluster identifies one or more characteristics associated with fish, and wherein the multi-dimensional space comprises multiple clusters.
5. The method of claim 1 , wherein the key points comprise a dorsal fin of the particular fish, a pectoral fin of the particular fish, and an eye of the particular fish.
6. The method of claim 1 , comprising:
generating, by the one or more processors, positive training data representative of a fish type;
generating, by the one or more processors, negative training data representative of other fish types; and
training, by the one or more processors, the machine learning model to generate embeddings using the positive training data and the negative training data.
7. The method of claim 1 , wherein the embedding comprises a 128-or-more dimensional vector.
8. A system comprising one or more processors and one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform operations comprising:
receiving multiple images of a particular fish;
generating, using the multiple images of the particular fish, a synthetic image of the particular fish that has a higher resolution than any of the received images of the particular fish;
generating a cropped representation of the synthetic image, such that the cropped representation includes a set of anatomical key points of the particular fish;
obtaining, from a machine learning model, an embedding for the cropped representation;
generating a cluster identifier based on clustering the embedding in a multi-dimensional space; and
providing, to a database, data identifying the cluster identifier for the particular fish.
9. The system of claim 8 , wherein generating the cluster identifier comprises:
determining that the embedding matches to a previously-generated embedding; and
determining, as a result, that the particular fish associated with the embedding has been re-identified.
10. The system of claim 8 , wherein generating the cluster identifier comprises:
determining that the embedding does not match any previously-generated embedding; and
determining, as a result, that the particular fish associated with the embedding is newly identified.
11. The system of claim 8 , wherein the cluster identifies one or more characteristics associated with fish, and wherein the multi-dimensional space comprises multiple clusters.
12. The system of claim 8 , wherein the key points comprise a dorsal fin of the particular fish, a pectoral fin of the particular fish, and an eye of the particular fish.
13. The system of claim 8 , wherein the embedding comprises a 128-or-more dimensional vector.
14. The system of claim 8 , wherein the operations comprise:
generating positive training data representative of a fish type;
generating negative training data representative of other fish types; and
training the machine learning model to generate embeddings using the positive training data and the negative training data.
15. One or more non-transitory computer-readable media comprising instructions stored thereon that are executable by one or more processing devices and upon such execution cause the one or more processing devices to perform operations comprising:
receiving multiple images of a particular fish;
generating, using the multiple images of the particular fish, a synthetic image of the particular fish that has a higher resolution than any of the received images of the particular fish;
generating a cropped representation of the synthetic image, such that the cropped representation includes a set of anatomical key points of the particular fish;
obtaining, from a machine learning model, an embedding for the cropped representation;
generating a cluster identifier based on clustering the embedding in a multi-dimensional space; and
providing, to a database, data identifying the cluster identifier for the particular fish.
16. The non-transitory computer-readable media of claim 15 , wherein generating the cluster identifier comprises:
determining that the embedding matches to a previously-generated embedding; and
determining, as a result, that the particular fish associated with the embedding has been re-identified.
17. The non-transitory computer-readable media of claim 15 , wherein generating the cluster identifier comprises:
determining that the embedding does not match any previously-generated embedding; and
determining, as a result, that the particular fish associated with the embedding is newly identified.
18. The non-transitory computer-readable media of claim 15 , wherein the cluster identifies one or more characteristics associated with fish, and wherein the multi-dimensional space comprises multiple clusters.
19. The non-transitory computer-readable media of claim 15 , wherein the key points comprise a dorsal fin of the particular fish, a pectoral fin of the particular fish, and an eye of the particular fish.
20. The non-transitory computer-readable media of claim 15 , wherein the operations comprise:
generating positive training data representative of a fish type;
generating negative training data representative of other fish types; and
training the machine learning model to generate embeddings using the positive training data and the negative training data.