Determining areas of interest in video based at least on a user's interactions with the video
According to one or more embodiments, an interaction device is provided. The interaction device includes processing circuitry configured to render for display a first premises security video comprising a plurality of frames, determine a user interaction with a playback of the first premises security video, determine a plurality of logical weights associated with the plurality of frames based at least on the user interaction, train a machine learning model based at least on the plurality of logical weights, and perform a premises security system action based at least on the trained machine learning model.
1. An interaction device comprising:
processing circuitry configured to:
render for display a first premises security video comprising a plurality of frames;
determine a user interaction with a playback of the first premises security video;
determine a plurality of logical weights associated with the plurality of frames based at least on the user interaction;
train a machine learning model based at least on the plurality of logical weights;
perform a premises security system action based at least on the trained machine learning model;
the user interaction corresponding to at least one of:
viewing at least one of the plurality of frames;
scrolling forward through at least one of the plurality of frames;
scrolling backwards through at least one of the plurality of frames;
zooming in on at least one of the plurality of frames;
pausing at least one of the plurality of frames for at least a predetermined amount of time; or
tagging at least one of the plurality of frames with a corresponding tag; and
each user interaction type is associated with a corresponding one of a plurality of logical weight formulas.
2. The interaction device of claim 1 , wherein at least one of the plurality of logical weight formulas is based at least on multiplying an amount of time a frame has been viewed by a user times a multiplier.
3. The interaction device of claim 1 , wherein the processing circuitry is further configured to perform the premises security system action by at least:
determining a frame of interest of the plurality of frames;
generating a graphical display identifying at least the frame of interest;
receiving a user input comprising at least one label associated with the frame of interest; and
further training the machine learning model based at least on the at least one label.
4. The interaction device of claim 3 , wherein the processing circuitry is further configured to determine the frame of interest by at least:
determining a logical mean weight mean associated with the plurality of frames; and
determining that the frame of interest has an associated logical weight that is greater than the logical weight mean.
5. The interaction device of claim 1 , wherein the processing circuitry is further configured to perform the premises security system action by at least:
predicting a premises security system alarm event based at least on the trained machine learning model and a second premises security video; and
triggering at least one premises security system device based at least on the premises security system alarm event.
6. A method implemented by an interaction device, the method comprising:
rendering for display a first premises security video comprising a plurality of frames;
determining a user interaction with a playback of the first premises security video;
determining a plurality of logical weights associated with the plurality of frames based at least on the user interaction;
training a machine learning model based at least on the plurality of logical weights;
performing a premises security system action based at least on the trained machine learning model;
the user interaction corresponding to at least one of:
viewing at least one of the plurality of frames;
scrolling forward through at least one of the plurality of frames;
scrolling backwards through at least one of the plurality of frames;
zooming in on at least one of the plurality of frames;
pausing at least one of the plurality of frames for at least a predetermined amount of time; or
tagging at least one of the plurality of frames with a corresponding tag; and
each user interaction type is associated with a corresponding one of a plurality of logical weight formulas.
7. The method of claim 6 , wherein at least one of the plurality of logical weight formulas is based at least on multiplying an amount of time a frame has been viewed by a user times a multiplier.
8. The method of claim 6 , further comprising performing the premises security system action by at least:
determining a frame of interest of the plurality of frames;
generating a graphical display identifying at least the frame of interest;
receiving a user input comprising at least one label associated with the frame of interest; and
further training the machine learning model based at least on the at least one label.
9. The method of claim 8 , further comprising determining the frame of interest by at least:
determining a logical mean weight mean associated with the plurality of frames; and
determining that the frame of interest has an associated logical weight that is greater than the logical weight mean.
10. The method of claim 6 , further comprising performing the premises security system action by at least:
predicting a premises security system alarm event based at least on the trained machine learning model and a second premises security video; and
triggering at least one premises security system device based at least on the premises security system alarm event.
11. A non-transitory, computer-readable storage medium comprising instructions configured to cause an interaction device to:
render for display a first premises security video comprising a plurality of frames;
determine a user interaction with a playback of the first premises security video;
determine a plurality of logical weights associated with the plurality of frames based at least on the user interaction;
train a machine learning model based at least on the plurality of logical weights;
perform a premises security system action based at least on the trained machine learning model;
the user interaction corresponding to at least one of:
viewing at least one of the plurality of frames;
scrolling forward through at least one of the plurality of frames;
scrolling backwards through at least one of the plurality of frames;
zooming in on at least one of the plurality of frames;
pausing at least one of the plurality of frames for at least a predetermined amount of time; or
tagging at least one of the plurality of frames with a corresponding tag; and
each user interaction type is associated with a corresponding one of a plurality of logical weight formulas.
12. The non-transitory, computer-readable storage medium of claim 11 , wherein at least one of the plurality of logical weight formulas is based at least on multiplying an amount of time a frame has been viewed by a user times a multiplier.
13. The non-transitory, computer-readable storage medium of claim 11 , wherein the instructions are configured to cause an interaction device to perform the premises security system action by at least:
determining a frame of interest of the plurality of frames;
generating a graphical display identifying at least the frame of interest;
receiving a user input comprising at least one label associated with the frame of interest; and
further training the machine learning model based at least on the at least one label.
14. The non-transitory, computer-readable storage medium of claim 13 , wherein the instructions are configured to cause an interaction device to determine the frame of interest by at least:
determining a logical mean weight mean associated with the plurality of frames; and
determining that the frame of interest has an associated logical weight that is greater than the logical weight mean.