Methods, systems, articles of manufacture, and apparatus to dynamically determine interaction display regions for screen sharing
Methods, apparatus, systems, and articles of manufacture are disclosed to dynamically determine interaction display regions for screen sharing. Example apparatus disclosed herein are to generate a screen share frame corresponding to content rendered on the display and intended for the screen share event; identify application-related contextual data corresponding to the screen share event; determine an interactive context based on at least one of (a) the application-related contextual data and (b) sensor data from at least one sensor, the interactive context including identification of an interaction region of the screen share frame; generate interaction metadata that includes the interactive context; and transmit a transport stream that includes the interaction metadata and the screen share frame.
1 . An electronic device, comprising:
a first display;
a second display;
at least one sensor;
at least one memory;
machine readable instructions; and
processor circuitry to at least one of instantiate or execute the machine readable instructions to:
generate a screen share frame corresponding to content rendered on the first display and intended for a screen share event;
identify application-related contextual data corresponding to the screen share event;
determine whether the first display includes the content that is rendered and intended for the screen share event;
determine whether the second display includes the content that is rendered and intended for the screen share event; and
identify, based on the determinations, parameters for at least one of the first display or the second display, the parameters to include an identifier of the at least one of the first display or the second display;
determine an interactive context based on the parameters and at least one of (a) the application-related contextual data, or (b) sensor data from the at least one sensor, the interactive context including an identification of an interaction region of the screen share frame;
generate interaction metadata that includes the interactive context; and
transmit a transport stream that includes the interaction metadata and the screen share frame.
2 . The electronic device of claim 1 , wherein the processor circuitry is to generate the screen share frame in response to detecting initialization of the screen share event.
3 . The electronic device of claim 1 , wherein the screen share frame includes a rendered application, and wherein the application-related contextual data includes information corresponding to the rendered application.
4 . The electronic device of claim 1 , wherein the identification of the interactive context is based on display-related contextual data, the processor circuitry to identify the display-related contextual data that includes parameters of the first display.
5 . The electronic device of claim 1 , wherein the processor circuitry is to identify the interactive context based on input data relative to the screen share event.
6 . The electronic device of claim 5 , wherein the input data includes human interface device (HID)-related data received from a HID, the HID corresponding to at least one of a mouse, a keyboard, or a touchpad.
7 . The electronic device of claim 5 , wherein the input data includes operating system events corresponding to human interface device (HID)-related data.
8 . The electronic device of claim 5 , wherein the at least one sensor includes a microphone, and wherein the input data includes language data, the processor circuitry to generate the language data by:
receiving audio data from the microphone; and
applying a natural language processing based model to the audio data.
9 . The electronic device of claim 5 , wherein the at least one sensor includes a camera, and wherein the input data includes eye tracking data, the processor circuitry to generate the eye tracking data by:
receiving image data from the camera; and
applying an image recognition model to the image data.
10 . The electronic device of claim 5 , wherein the processor circuitry is to determine the interactive context by:
collecting a sequence of the input data and a corresponding sequence of the application-related contextual data over a period of time;
processing the sequence of the input data and the corresponding sequence of the application-related contextual data to predict user interaction intent relative to the screen share frame for the period of time; and
identifying the interactive context for the period of time based on the predicted user interaction intent relative to the screen share frame for the period of time.
11 . The electronic device of claim 10 , wherein the sequence of the input data and the corresponding sequence of the application contextual data is processed using at least one of a machine learning model or a rule.
12 . The electronic device of claim 1 , wherein the interaction region is represented by coordinates relative to the screen share frame.
13 . At least one non-transitory machine readable storage medium comprising instructions that, when executed, cause processor circuitry to at least:
generate a screen share image corresponding to content rendered on at least one of a first display or a second display used for a screen share event;
identify application-related contextual information corresponding to the screen share event;
determine whether the first display includes the content that is rendered and intended for the screen share event;
determine whether the second display includes the content that is rendered and intended for the screen share event; and
identify, based on the determinations, parameters for the at least one of the first display or the second display, the parameters to include an identifier of the at least one of the first display or the second display;
predict an interactive context based on the parameters and at least one of (a) the application-related contextual information, or (b) sensor data based on signals output by a sensor, the interactive context including an interaction region of the screen share image;
generate interaction metadata corresponding to the interactive context and the interaction region; and
send a transport stream containing the interaction metadata and the screen share image.
14 . The at least one non-transitory machine readable storage medium of claim 13 , wherein the screen share image includes an application rendered on the first display, and wherein the application-related contextual information includes data corresponding to the rendered application.
15 . The at least one non-transitory machine readable storage medium of claim 13 , wherein the prediction of the interactive context is based on display-related contextual data, and wherein the processor circuitry to identify the display-related contextual data that includes parameters of the first display.
16 . The at least one non-transitory machine readable storage medium of claim 13 , wherein the instructions, when executed, cause the processor circuitry to predict the interactive context based on input data relative to the screen share event.
17 . The at least one non-transitory machine readable storage medium of claim 16 , wherein the input data includes user input data received from a human interface device (HID), the HID corresponding to at least one of a mouse, a keyboard, or a touchpad.
18 . The at least one non-transitory machine readable storage medium of claim 16 , wherein the input data includes operating system events corresponding to human interface device (HID)-related user input data.
19 . The at least one non-transitory machine readable storage medium of claim 16 , wherein the sensor is a microphone, and wherein the input data includes language data, the processor circuitry to generate the language data by:
receiving audio data from the microphone; and
applying a natural language processing based model to the audio data.
20 . The at least one non-transitory machine readable storage medium of claim 16 , wherein the sensor is a camera, and wherein the input data includes eye tracking data, the processor circuitry to generate the eye tracking data by:
receiving image data from the camera; and
applying an image recognition model to the image data.
21 . A method comprising:
generating, by executing machine readable instructions with at least one processor, a display share frame corresponding to content rendered on at least one of a first display or a second display, the display share frame selected for a display share event;
determining whether the first display includes the content that is rendered and intended for the display share event;
determining whether the second display includes the content that is rendered and intended for the display share event; and
identifying, based on the determinations, parameters for the at least one of the first display or the second display, the parameters to include an identifier of the at least one of the first display or the second display;
determining, by executing the machine readable instructions with the at least one processor, an interactive intent based on the parameters and at least one of application contextual data and sensor data from at least one sensor, the interactive intent including an interaction region of the display share frame;
generating, by executing the machine readable instructions with the at least one processor, interaction metadata that includes the interactive intent and the interaction region; and
transmitting, by executing the machine readable instructions with the at least one processor, the interaction metadata and the display share frame.