Enhanced storage and data retrieval for face-related data

1. A method for reducing a search space represented by representative vectors by compacting facial data, the method comprising:

generating a first representative vector based on a first plurality of face vectors, wherein the first representative vector is associated with the first plurality of face vectors in a collection of representative vectors, and wherein each face vector of the first plurality of face vectors comprises a set of vector values within a latent space;

generating a second representative vector based on a second plurality of face vectors, wherein the second representative vector is associated with the second plurality of face vectors in the collection of representative vectors;

determining a latent space distance based on the first and second plurality of face vectors;

determining whether the latent space distance satisfies a threshold; and

in response to a determination that the latent space distance satisfies the threshold:

associating a combined representative vector with the first plurality of face vectors and the second plurality of face vectors; and

removing the first and second representative vectors from the collection of representative vectors.

2. The method of claim 1 , further comprising:

ingesting a first video segment of a video stream;

generating the first plurality of face vectors based on the first video segment; and

generating the combined representative vector based on the first plurality of face vectors and the second plurality of face vectors, wherein:

generating the respective vectors of the first plurality face vectors occurs in real-time with respect to the ingestion of the first video segment; and

generating the combined representative vector comprises generating the combined representative vector via a background process that occurs with less frequency than a rate at which the first plurality of face vectors is updated.

3. The method of claim 1 , wherein determining the latent space distance comprises determining a distance between the first representative vector and the second representative vector.

4. The method of claim 1 , wherein the first representative vector and the second representative vector are vectors in the latent space.

5. The method of claim 1 , wherein generating the first representative vector comprises:

generating a centroid in the latent space based on the first plurality of face vectors; and

generating the first representative vector based on the centroid.

6. The method of claim 1 , wherein associating the combined representative vector with the first plurality of face vectors and the second plurality of face vectors comprises:

determining a combined centroid based on a combined cluster of vectors comprising the first plurality of face vectors and the second plurality of face vectors; and

generating the combined representative vector based on the combined centroid.

7. The method of claim 6 , further comprising:

determining a radius between the combined centroid and a furthest vector of the combined cluster of vectors from the combined centroid; and

updating a search parameter based on the radius.

8. A non-transitory, machine-readable medium storing program instructions that, when executed by a set of processors, causes the set of processors to perform operations comprising:

generating a first set of representative values based on a first plurality of face vectors, wherein the first set of representative values is associated with the first plurality of face vectors in a collection of representative values, and wherein each face vector of the first plurality of face vectors comprises a set of vector values corresponding with portions of a face;

generating a second set of representative values based on a second plurality of face vectors, wherein the second set of representative values is associated with the second plurality of face vectors in the collection of representative values;

determining a latent space distance based on the first and second plurality of face vectors;

determining whether the latent space distance satisfies a threshold; and

in response to a determination that the latent space distance satisfies the threshold:

associating a combined set of representative values with the first plurality of face vectors and the second plurality of face vectors; and

removing the first set of representative values and the second set of representative values from the collection of representative values.

9. The non-transitory, machine-readable medium of claim 8 , the operations further comprising:

storing the combined set of representative values in a distributed database;

determining whether a video segment occurred after a pre-determined time range, wherein the first plurality of face vectors are obtained from the video segment; and

based on a determination that the video segment occurred after the pre-determined time range, storing the first plurality of face vectors in a local data store that is different from the distributed database.

10. The non-transitory, machine-readable medium of claim 9 , the operations further comprising storing the combined set of representative values in a key-value data store.

11. The non-transitory, machine-readable medium of claim 8 , the operations further comprising:

obtaining a query, wherein a first set of query values of the query comprises a time-related value or an organization-related value;

obtaining metadata associated with the collection of representative values, wherein the metadata comprises time-related data or identifiers of organizations;

filtering the collection of representative values to obtain a subset of representative values based on the metadata and the first set of query values; and

searching the subset of representative values to retrieve the combined set of representative values based on a second set of query values of the query.

12. The non-transitory, machine-readable medium of claim 8 , wherein the latent space distance is a first latent space distance, the operations further comprising:

obtaining a search request comprising an image;

determining a set of facial features based on the image;

generating a candidate face vector based on the set of facial features;

determining a second latent space distance based on the candidate face vector and the combined set of representative values; and

selecting the combined set of representative values based on the second latent space distance.

13. The non-transitory, machine-readable medium of claim 12 , wherein selecting the combined set of representative values comprises performing a nearest neighbor search based on the candidate face vector and a plurality of representative vectors comprising the combined set of representative values.

14. The non-transitory, machine-readable medium of claim 12 , further comprising:

obtaining a library of face vectors, wherein each respective face vector of the library of face vectors is associated with a respective user identity; and

presenting a face image associated with a first user identifier, wherein the first user identifier is associated with the combined set of representative values.

15. The non-transitory, machine-readable medium of claim 8 , wherein determining the latent space distance comprises determining the latent space distance in response to a determination that a recurring duration has passed.

16. A system comprising:

one or more processors; and

memory storing program instructions that, when executed by the one or more processors, cause the one or more processors to effectuate operations comprising:

generating a first set of representative values based on a first plurality of face vectors, wherein the first set of representative values is associated with the first plurality of face vectors in a collection of representative values, and wherein each face vector of the first plurality of face vectors comprises a set of vector values corresponding with portions of a face;

generating a second set of representative values based on a second plurality of face vectors, wherein the second set of representative values is associated with the second plurality of face vectors in the collection of representative values;

determining a latent space distance based on the first and second plurality of face vectors;

determining whether the latent space distance satisfies a threshold; and

in response to a determination that the latent space distance satisfies the threshold:

associating a combined set of representative values with the first plurality of face vectors and the second plurality of face vectors; and

removing the first set of representative values and the second set of representative values from the collection of representative values.

17. The system of claim 16 , the operations further comprising:

generating a bounding box surrounding a face in an image;

detecting a set of features of a sub-image in the bounding box; and

generating a face vector of the first plurality of face vectors based on the set of features.

18. The system of claim 17 , wherein generating the face vector comprises generating the face vector using a set of neural network layers.

19. The system of claim 16 , further comprising:

obtaining a search request comprising a time range;

determining whether the first plurality of face vectors is in the time range;

based on a determination that the first plurality of face vectors is in the time range, obtaining the first plurality of face vectors from a local database;

determining whether the second plurality of face vectors is in the time range; and

based on a determination that the second plurality of face vectors is not in the time range, obtaining the second plurality of face vectors from a distributed database.

20. The system of claim 16 , wherein associating the combined set of representative values with the first plurality of face vectors and the second plurality of face vectors comprises indicating a combined video segment associated with the first plurality of face vectors and the second plurality of face vectors, wherein the combined video segment has a starting time as a first video segment, and wherein the combined video segment has an ending time as a second video segment.