System and method for automatically generating personalized service offers based on a video analysis

1 . A computer implemented method of processing a video for generating a graphical user interface (GUI) presenting personalized services for cleaning a property, comprising:

receiving the video captured by a camera depicting an interior of the property comprising a plurality of rooms;

extracting frame-level features from a plurality of frames of the video on a per-frame basis;

generating structured frame-level metadata from the plurality of frame-level features on the per-frame basis;

clustering a plurality of the structured frame-level metadata into a plurality of clusters according to a type of room;

temporally aggregating the structured frame-level metadata of the frames of each cluster corresponding to each type of room to generate a structured room-level metadata for each room represented by each cluster;

feeding each structured room-level metadata into at least one ML model for predicting at least one cleaning task and a corresponding duration for completing each cleaning task; and

generating and presenting on a display of a client terminal, the at least one cleaning task and corresponding estimated durations within the GUI.

2 . The computer implemented method of claim 1 , further comprising training the at least one ML model by generating a training dataset comprising a plurality of records, wherein a record includes:

a sample structured room-metadata, and a ground truth of at least one sample cleaning task and corresponding duration for completing each sample cleaning task,

wherein the sample structured room-level metadata is generated by:

receiving a video captured by a camera depicting an interior of a sample property comprising a plurality of sample rooms;

extracting frame-level features from a plurality of frames of the video on a per-frame basis;

generating structured frame-level metadata from the plurality of frame-level features on the per-frame basis;

clustering a plurality of the structured frame-level metadata into a plurality of clusters according to a type of room; and

temporally aggregating the structured frame-level metadata of the frames of each cluster corresponding to each type of room to generate the sample structured room-level metadata for each room represented by each cluster; and

training the ML model on the training dataset for predicting at least one target cleaning task and a corresponding target duration for completing each target cleaning task in response to an input of a target structured cluster-level metadata computed from a target video.

3 . The computer implemented method of claim 1 , wherein the frame-level features are extracted by: at least one of: feeding the frame into a room classification model to obtain a type of room, and feeding the frame into a detector model for detecting at least one object and at least one surface.

4 . The computer implemented method of claim 3 , wherein detecting at least one surface comprises detecting a plurality of different types of surfaces and classifying the plurality of different types of surfaces, including: floor, tables, beds, and counters, wherein the plurality of features are computed according to the plurality of different types of surfaces.

5 . The computer implemented method of claim 1 , wherein the plurality of frame-level features include at least a type of room, and the clustering is performed according to the types of rooms.

6 . The computer implemented method of claim 1 , further comprising:

mapping features of the structured room-level metadata of each room to at least one cleaning task, to obtain a plurality of cleaning tasks for a plurality of rooms,

wherein feeding comprises feeding each at least one cleaning task and corresponding structured room-level metadata into the ML model for estimating the duration for completing each cleaning task.

7 . The computer implemented method of claim 1 , wherein the GUI includes at least one interactive element for selecting a subset of at least one cleaning task, and for automatically scheduling a cleaning service for performing the selected subset of the at least one cleaning task.

8 . The computer implemented method of claim 7 , further comprising receiving a message from the client terminal in response to interaction with the interactive element presented in the GUI, and automatically scheduling the cleaning service for performing the selected subset of the at least one cleaning task.

9 . The computer implemented method of claim 1 , wherein the structured room-level metadata includes measured room size and/or measured room cleanliness.

10 . The computer implemented method of claim 9 , wherein the measured room cleanliness is computed as a combination of at least one of: number of objects to pick up, variety of different objects in the room, required sorting, required folding.

11 . The computer implemented method of claim 1 , wherein the at least one cleaning task comprise pre-defined cleaning tasks selected from: sweeping, mopping, dishwashing.

12 . The computer implemented method of claim 1 , wherein extracting is performed by a detector model implemented as a semantic segmentation model generating a plurality of classifications for pixels of the frame.

13 . The computer implemented method of claim 1 , wherein the room type is selected from: living room, kitchen, bedroom, bathroom, hallway, and office.

14 . The computer implemented method of claim 1 , wherein the video is captured by the camera integrated with and/or connected to a mobile device, and the at least one cleaning task is presented on the display of the mobile device.

15 . The computer implemented method of claim 14 , wherein the method is executed by a server in network communication with a plurality of mobile devices.

16 . The computer implemented method of claim 1 , further comprising feeding the at least one cleaning task, corresponding estimated durations, and at least one current parameter into a pricing engine configured for dynamically computing costs for performing each of the at least one cleaning task, and presenting the computed costs within the GUI on the display of the client terminal.

17 . The computer implemented method of claim 16 , wherein the at least one current parameter is selected from: base labor cost, regional adjustment, supply-demand factor, and discount as part of a promotion.

18 . The computer implemented method of claim 1 , further comprising pre-processing the plurality of frames of the video for normalizing resolution and/or lighting conditions.

19 . The computer implemented method of claim 1 , further comprising presenting within the GUI, a plurality of interactive elements configured for a user to enter feedback regarding the at least one cleaning task including actual duration to complete the at least one cleaning task, and performing an update of models for future predictions according to the feedback.

20 . The computer implemented method of claim 1 , further comprising generating and presenting on a display of a client terminal, at least one personalized message based on the at least one cleaning task and corresponding estimated durations within the GUI, the at least one personalized message selected from: a personalized advertisement, a personalized promotion, and a personalized bundle.

21 . The computer implemented method of claim 1 , further comprising:

computing a confidence score for the extracted frame-level features; and

in response to the confidence score being below a threshold, dynamically presenting with the GUI, instructions within the GUI to at least one of: zoom in on specific areas, capture missing angles, and/or provide additional angular coverage, predicted to increase the confidence score,

wherein the receiving the video, the extracting frame-level features, and the computing the confidence score, are iterated until the confidence score exceeds the threshold.

22 . A system for processing a video for generating a graphical user interface (GUI) presenting personalized services for cleaning a property, comprising:

at least one processor executing a code for:

receiving the video captured by a camera depicting an interior of the property comprising a plurality of rooms;

extracting frame-level features from a plurality of frames of the video on a per-frame basis;

generating structured frame-level metadata from the plurality of frame-level features on the per-frame basis;

clustering a plurality of the structured frame-level metadata into a plurality of clusters according to a type of room;

temporally aggregating the structured frame-level metadata of the frames of each cluster corresponding to each type of room to generate a structured room-level metadata for each room represented by each cluster;

feeding each structured room-level metadata into a ML model for predicting at least one cleaning task and a corresponding duration for completing each cleaning task; and

generating and presenting on a display of a client terminal, the at least one cleaning task and corresponding estimated durations within the GUI.

23 . A computer implemented method of training a ML model for predicting at least one cleaning task and a corresponding duration for completing each cleaning task from a video, comprising:

generating a training dataset comprising a plurality of records, wherein a record includes:

a sample structured room-metadata, and a ground truth of at least one sample cleaning task and corresponding duration for completing each sample cleaning task,

wherein the sample structured room-level metadata is generated by:

receiving a video captured by a camera depicting an interior of a sample property comprising a plurality of sample rooms;

extracting frame-level features from a plurality of frames of the video on a per-frame basis;

generating structured frame-level metadata from the plurality of frame-level features on the per-frame basis;

clustering a plurality of the structured frame-level metadata into a plurality of clusters according to a type of room;

temporally aggregating the structured frame-level metadata of the frames of each cluster corresponding to each type of room to generate the sample structured room-level metadata for each room represented by each cluster; and

training the ML model on the training dataset for predicting at least one target cleaning task and a corresponding target duration for completing each target cleaning task in response to an input of a target structured cluster-level metadata computed from a target video.