Curation of custom workflows using multiple cameras
A multi-layer technology stack includes a sensor layer including image sensors, a device layer, and a cloud layer, with interfaces between the layers. A method to curate different custom workflows for multiple applications include the following. Requirements for custom sets of data packages for the applications is received. The custom set of data packages include sensor data packages (e.g., SceneData) and contextual metadata packages that contextualize the sensor data packages (e.g., SceneMarks). Based on the received requirements and capabilities of components in the technology stack, the custom workflow for that application is deployed. This includes a selection, configuration and linking of components from the technology stack. The custom workflow is implemented in the components of the technology stack by transmitting workflow control packages directly and/or indirectly via the interfaces to the different layers.
1. A non-transitory computer-readable storage medium storing executable computer program instructions for curating custom workflows for different applications, the instructions executable by a computer system and causing the computer system to perform a method comprising
receiving task requests from multiple applications;
determining custom workflows in response to the task requests; and
deploying the custom workflows on a multi-layer technology stack having components including cameras;
wherein at least one of the custom workflows includes image capture and artificial intelligence (AI) processing and, for that custom workflow:
determining the custom workflow comprises selecting AI models to implement the AI processing; and
deploying the custom workflow comprises distributing the AI models to components in the multi-layer technology stack for execution and configuring cameras in the multi-layer technology stack for image capture.
2. The non-transitory computer-readable storage medium of claim 1 wherein, for the at least one custom workflow, the AI models are selected in response to the custom workflow having detected an event.
3. The non-transitory computer-readable storage medium of claim 2 wherein the event was detected by AI processing of captured images in said custom workflow.
4. The non-transitory computer-readable storage medium of claim 2 wherein the custom workflow generated a metadata package that indicates detection of the event, and the AI models are selected in response to the indication in the metadata package.
5. The non-transitory computer-readable storage medium of claim 2 wherein the event was detected by AI processing of metadata packages generated by different components in the multi-layer technology stack.
6. The non-transitory computer-readable storage medium of claim 5 wherein the AI processing provides a higher level understanding of a situational context.
7. The non-transitory computer-readable storage medium of claim 2 wherein determining the custom workflow further comprises, in response to the custom workflow having detected the event, selecting cameras to capture images for the custom workflow to process to generate inputs to the AI models.
8. The non-transitory computer-readable storage medium of claim 2 wherein determining the custom workflow further comprises, in response to the custom workflow having detected the event, selecting settings for cameras to capture images for the custom workflow to process to generate inputs to the AI models.
9. The non-transitory computer-readable storage medium of claim 2 wherein the AI models are selected based on understanding a context of the detected event.
10. The non-transitory computer-readable storage medium of claim 1 wherein, for the at least one custom workflow, the AI models are selected in response to the custom workflow having detected an object or person, and the AI models are selected for reidentification of the detected object or person.
11. The non-transitory computer-readable storage medium of claim 10 wherein the selected AI models are distributed to components in the multi-layer technology stack, based on a location of the detection of the object or person and based on a proximity map with locations of cameras relative to the location of the detection.
12. The non-transitory computer-readable storage medium of claim 1 wherein the AI models are market-specific AI models, and selecting the AI models is based on a market indicated by the task request.
13. The non-transitory computer-readable storage medium of claim 1 wherein interfaces to layers of the multi-layer technology stack comprise standardized application programming interfaces (APIs), and the AI models are distributed using the standardized APIs.
14. The non-transitory computer-readable storage medium of claim 13 wherein at least some of the cameras are configured indirectly through layers of the multi-layer technology stack.
15. The non-transitory computer-readable storage medium of claim 13 wherein, for the at least one custom workflow, the task request does not specify which cameras are to be used for image capture.
16. The non-transitory computer-readable storage medium of claim 1 wherein the method further comprises:
receiving capabilities of components in the multi-layer technology stack, wherein selecting the AI models is based on the capabilities of the components.
17. The non-transitory computer-readable storage medium of claim 1 wherein, for the at least one custom workflow, the requested tasks require understanding context of images captured by the cameras and understanding events occurring in a scene based on images of the scene captured by the cameras.
18. A method implemented on a computer system for curating custom workflows for different applications, the method comprising
receiving task requests from multiple applications;
determining custom workflows in response to the task requests; and
deploying the custom workflows on a multi-layer technology stack having components including cameras;
wherein at least one of the custom workflows includes image capture and artificial intelligence (AI) processing and, for that custom workflow:
determining the custom workflow comprises selecting AI models to implement the AI processing; and
deploying the custom workflow comprises distributing the AI models to components in the multi-layer technology stack for execution and configuring cameras in the multi-layer technology stack for image capture.
19. The method of claim 18 wherein the method is executed by a curation service separate from the applications.
20. A system comprising:
a memory system storing instructions; and
a processor system, coupled with the memory system and to execute the instructions, the instructions when executed cause the processor system to:
receive task requests from multiple applications;
determine custom workflows in response to the task requests; and
deploy the custom workflows on a multi-layer technology stack having components including cameras;
wherein at least one of the custom workflows includes image capture and artificial intelligence (AI) processing and, for that custom workflow:
determining the custom workflow comprises selecting AI models to implement the AI processing; and
deploying the custom workflow comprises distributing the AI models to components in the multi-layer technology stack for execution and configuring cameras in the multi-layer technology stack for image capture.