Guided post-deployment training of customized recognition models
Introduced here is a training system that allows users to define what they would like their surveillance systems to detect, recognize, or otherwise identify. For simplicity, the use case provided by a given user for teaching purposes may be referred to as a “skill.” A surveillance system can learn each of the skills provided by its respective user from a limited number of examples. For example, the given user may only provide several examples—or even a single example—in contrast to conventional learning approaches.
1 . A method comprising:
storing a set of embeddings in a database, wherein each embedding in the set of embeddings is generated as part of a training operation in which a user defines multiple categories, each of which is associated with at least one embedding included in the set of embeddings;
appending, to the set of embeddings, an identifier associated with an edge device of a surveillance system, wherein each embedding in the set of embeddings is generated based on a particular sample captured by the edge device;
acquiring a new sample generated by the edge device, wherein the new sample is associated with the identifier;
providing the new sample to a model as input, so as to produce an embedding;
in response to determining, using the identifier, that the new sample and the set of embeddings are associated with the edge device, comparing the embedding against the set of embeddings; and
outputting a prediction based on an outcome of said comparing.
2 . The method of claim 1 , wherein the prediction is representative of an assignment of the new sample to a category of the multiple categories, and wherein the category corresponds to whichever embedding in the set of embeddings is determined to most closely correspond to the embedding.
3 . The method of claim 1 , further comprising:
receiving input indicative of an acquisition of data generated by the surveillance system; and
determining that one or more values included in the data match a pattern-defining parameter;
wherein said acquiring the new sample is performed in response to said determining that one or more values included in the data match the pattern-defining parameter.
4 . The method of claim 3 , wherein the data is generated by another edge device of the surveillance system.
5 . The method of claim 1 , further comprising:
receiving input indicative of a request from a user to initiate an inferencing phase;
wherein said acquiring the new sample is performed in response to said receiving the input.
6 . The method of claim 1 , wherein said acquiring the new sample is performed in response to a determination that a time matches a temporal criterion specified by a user.
7 . The method of claim 1 , wherein said outputting the prediction comprises:
causing display of the prediction on an interface that is accessible via a computer program through which the surveillance system is manageable.
8 . The method of claim 1 , wherein the edge device includes a camera, and wherein the new sample is representative of a digital image.
9 . The method of claim 1 , wherein the edge device includes a microphone, and wherein the new sample is representative of an audio file.
10 . The method of claim 1 , wherein said acquiring the new sample, said providing the new sample, said comparing the embedding, and said outputting the prediction are performed by a system implemented in the edge device.
11 . One or more non-transitory, computer-readable storage media comprising instructions recorded thereon, wherein the instructions, when executed by at least one data processor of a system, cause the system to:
store a set of embeddings in a database, wherein each embedding in the set of embeddings is generated as part of a training operation in which a user defines multiple categories, each of which is associated with at least one embedding included in the set of embeddings;
append, to the set of embeddings, an identifier associated with an edge device of a surveillance system, wherein each embedding in the set of embeddings is generated based on a particular sample captured by the edge device;
acquire a new sample generated by the edge device, wherein the new sample is associated with the identifier;
provide the new sample to a model as input, so as to produce an embedding;
in response to determining, using the identifier, that the new sample and the set of embeddings are associated with the edge device, compare the embedding against the set of embeddings; and
output a prediction based on an outcome of said comparing.
12 . The one or more non-transitory, computer-readable storage media of claim 11 , wherein the prediction is representative of an assignment of the new sample to a category of the multiple categories, and wherein the category corresponds to whichever embedding in the set of embeddings is determined to most closely correspond to the embedding.
13 . The one or more non-transitory, computer-readable storage media of claim 11 , further comprising instructions causing the system to:
receive input indicative of an acquisition of data generated by the surveillance system; and
determine that one or more values included in the data match a pattern-defining parameter;
wherein said acquiring the new sample is performed in response to said determining that one or more values included in the data match the pattern-defining parameter.
14 . The one or more non-transitory, computer-readable storage media of claim 13 , wherein the data is generated by another edge device of the surveillance system.
15 . The one or more non-transitory, computer-readable storage media of claim 11 , further comprising instructions causing the system to:
receive input indicative of a request from a user to initiate an inferencing phase;
wherein said acquiring the new sample is performed in response to said receiving the input.
16 . The one or more non-transitory, computer-readable storage media of claim 11 , wherein said acquiring the new sample is performed in response to a determination that a time matches a temporal criterion specified by a user.
17 . The one or more non-transitory, computer-readable storage media of claim 11 , wherein said outputting the prediction comprises:
causing display of the prediction on an interface that is accessible via a computer program through which the surveillance system is manageable.
18 . The one or more non-transitory, computer-readable storage media of claim 11 , wherein the edge device includes a camera, and wherein the new sample is representative of a digital image.
19 . The one or more non-transitory, computer-readable storage media of claim 11 , wherein the edge device includes a microphone, and wherein the new sample is representative of an audio file.
20 . The one or more non-transitory, computer-readable storage media of claim 11 , wherein said acquiring the new sample, said providing the new sample, said comparing the embedding, and said outputting the prediction are performed by a system implemented in the edge device.