Granted Patent B2

US 12,561,389 · App. 17/807,880 · Granted Feb 24, 2026

Inventors: Ori Ziv (Herzliya, IL); Barak Kinarti (Kfar Saba, IL); Ben Bakhar (Tel Aviv, IL); Zvi Figov (Modiin, IL); Fardau Van Neerden (Gelderland, NL); Ohad Jassin (Tel Mond, IL); Avi Neeman (Tel Aviv, IL)

Assignee: Microsoft Technology Licensing, LLC

G06F16/954G06F16/2322G06F16/2456G06F16/951G06F16/957

Abstract

Claims (68)

1 . A system comprising:

a processor system comprising a processor; and

memory coupled to the processor, the memory comprising computer executable instructions that, when executed by the processor, perform operations comprising:

receiving content by a web browser of a client device, the web browser comprising an in-browser architecture that includes an execution environment for executing an artificial intelligence (AI) pipeline, wherein the content includes at least one of audio content or video content;

extracting, by the in-browser architecture, a set of data features from the content;

retrieving, by the in-browser architecture, a set of AI modules;

creating the AI pipeline in the execution environment using the set of AI modules;

providing the set of data features to the AI pipeline;

executing the AI pipeline in the execution environment to process the content and create a set of insights; and

providing, by the in-browser architecture, the set of insights to an interface of the web browser during receipt or playback of the content.

2 . The system of claim 1 , wherein extracting the set of data features from the content comprises:

decoding each data stream in the content; and

extracting the set of data features.

3 . The system of claim 1 , wherein creating the AI pipeline using the set of AI modules comprises:

retrieving the set of AI modules from an AI module repository comprising a plurality of AI modules, the AI module repository being external to the web browser, the set of AI modules representing a subset of the plurality of AI modules.

4 . The system of claim 1 , wherein selecting the set of AI modules comprises:

receiving a manual selection of the set of AI modules via the web browser.

5 . The system of claim 1 , wherein selecting the set of AI modules comprises:

evaluating, by the web browser, attributes of the content; and

based on the attributes of the content, selecting, by the web browser, the set of AI modules.

6 . The system of claim 1 , wherein creating the AI pipeline comprises:

loading the set of AI modules in the execution environment; and

arranging the set of AI modules according to a configuration order, the configuration order defining dependencies for the set of AI modules.

7 . The system of claim 1 , wherein each AI module in the set of AI modules:

represents a step in the AI pipeline; and

comprises software code or instructions for performing the step.

8 . The system of claim 1 , wherein a particular AI module in the set of AI modules performs at least one of:

topic inferencing;

object detection; or

speech classification.

9 . The system of claim 1 , wherein, in addition to the set of AI modules, the AI pipeline further comprises at least one operator, preprocessing step, or postprocessing step.

10 . The system of claim 1 , wherein executing the AI pipeline causes a particular AI module in the set of AI modules to generate an output comprising inference data for the set of data features.

11 . The system of claim 10 , wherein the inference data comprises at least one of:

a timestamp for a video frame;

a bounding box; or

a confidence value for a detected object or event.

12 . A method comprising:

extracting, by a web browser implementing an in-browser architecture, a set of data features from content, wherein:

the web browser is implemented by a user device;

the in-browser architecture comprises an execution environment for executing an artificial intelligence (AI) pipeline; and

the content is a video file comprising video data and audio data;

retrieving, by the in-browser architecture, a set of AI modules;

creating the AI pipeline in the execution environment using the set of AI modules;

providing the set of data features to the AI pipeline;

creating a set of insights by executing the AI pipeline in the execution environment; and

providing, by the in-browser architecture, the set of insights to the web browser.

13 . The method of claim 12 , wherein the user device is a mobile device.

14 . The method of claim 12 , wherein the user device retrieves the set of AI modules from a server environment or a cloud-based environment separate from the user device.

15 . The method of claim 12 , wherein creating the AI pipeline and creating the set of insights is performed on-device of the user device.

16 . The method of claim 12 , the method further comprising:

prior to extracting the set of data features from the content:

receiving the content at the web browser, wherein the content is provided by a data source external to the user device.

17 . The method of claim 16 , the method further comprising:

upon receiving the content at the web browser, receiving an insight objective for the content, wherein the insight objective indicates a type of analysis to be performed by the AI pipeline and one or more types of insights to be provided for the content.

18 . The method of claim 12 , wherein providing the set of insights to the web browser comprises causing the set of insights to be displayed in in interface of the user device in real-time as the content is displayed in the interface.

19 . A user device comprising:

a processor; and

memory comprising computer executable instructions that, when executed, perform operations comprising:

receiving content by a web browser of the user device, the web browser comprising an in-browser architecture that includes an execution environment for executing an artificial intelligence (AI) pipeline on-device of the user device, the content being a video file comprising video data and audio data;

extracting, by the in-browser architecture, a set of data features from the content;

retrieving, by the in-browser architecture, a set of AI modules based on the set of data features;

creating the AI pipeline in the execution environment by:

retrieving the set of AI modules from a data source outside of the execution environment; and

loading the set of AI modules into the execution environment;

providing the set of data features to the AI pipeline;

creating a set of insights by executing the AI pipeline in the execution environment; and

providing, by the in-browser architecture, the set of insights to an interface of the user device such that one or more insights in the set of insights is provided in the interface while at least a portion of the content is provided in the interface.

20 . The user device of claim 19 , wherein the set of AI modules is retrieved from a server environment or a cloud-based environment separate from the user device.

Assignments (2)

Continuity (1)

References Cited (16)