Context-aware dialogue system providing predicted next user interface steps
In the present application, a method of predicting next UI steps for a user by a context-aware dialogue system is disclosed. A plurality of user interface (UI) events associated with a UI is tracked. A predicted next UI step is determined based on at least a portion of the plurality of UI events. A dialogue system component is caused to indicate the predicted next UI step.
1 . A method, comprising:
tracking a plurality of user interface (UI) events associated with interaction of a user with a UI;
tracking a plurality of user prompts associated with interaction of the user with a dialog system;
serializing the plurality of UI events into serialized UI events in a serialization format;
serializing the plurality of user prompts into serialized prompt events in the serialization format;
receiving, by an event cache manager, the serialized UI events and the serialized prompt events as a set of serialized events;
discarding, by the event cache manager, at least one serialized event from the set of serialized events;
storing, by the event cache manager, the set of serialized events in a cache;
transmitting, by the event cache manager, one or more of the serialized events from the cache to a next step predictor;
predicting, by the next step predictor, a next UI step, wherein predicting the next UI step includes applying a machine learning model to at least a portion of the serialized events transmitted by the event cache manager; and
causing the dialog system to indicate the next UI step.
2 . The method of claim 1 , wherein the plurality of UI events includes one or more of: a mouse event, a keyboard event, a load event, or an unload event.
3 . The method of claim 1 , wherein predicting the next UI step further comprises:
determining a user profile embedding associated with the user, wherein the user profile embedding is a tensor that describes how the user interacted with one or more of a website or an application during previous sessions.
4 . The method of claim 1 , further comprising:
serializing the next UI step into serialized system events in the serialization format;
receiving, by the event cache manager, the serialized system events; and
storing, by the event cache manager, the serialized system events in the cache.
5 . The method of claim 1 , wherein causing the dialog system to indicate the next UI step includes one or more of:
causing a UI region to be highlighted,
causing a URL to be shown, or
causing an inquiry of whether the specific user needs help.
6 . The method of claim 1 , wherein discarding the at least one serialized event from the set of serialized events further comprises:
detecting a duplicate serialized event in the at least one serialized event; and
discarding the duplicate serialized event.
7 . The method of claim 6 , wherein the duplicate serialized event is a serialized UI event, and wherein detecting the duplicate serialized event comprises:
determining that the duplicate serialized event and another serialized UI event are: (i) of a same type, and (ii) an on-screen distance therebetween is below a predetermined distance.
8 . The method of claim 6 , wherein the duplicate serialized event is a serialized UI event, and wherein detecting the duplicate serialized event comprises:
determining that the duplicate serialized event and another serialized UI event: (i) are of a same type but with different parameters, and (ii) both occurred within a time window of a predetermined period.
9 . The method of claim 1 , further comprising:
determining that the cache is full; and
deleting, from the cache, one or more cached serialized events in an ordering of: UI events from a prior session, then other UI events, then system responses, and then user prompts.
10 . The method of claim 1 , further comprising:
providing, by the event cache manager, the one or more of the serialized events to a training and analysis data store.
11 . The method of claim 1 , wherein causing the dialog system to indicate the next UI step comprises:
highlighting, by a UI highlighter, an element of a webpage interfacing with the dialog system; and
removing the highlighting in response to a user interfacing with the element or after a pre-determined period of time passes.
12 . A system, comprising:
a processor configured to:
track a plurality of user interface (UI) events associated with interaction of a user with a UI;
track a plurality of user prompts associated with interaction of the user with a dialog system;
serialize the plurality of UI events into serialized UI events in a serialization format;
serialize the plurality of user prompts into serialized prompt events in the serialization format;
receive, by an event cache manager, the serialized UI events and the serialized prompt events as a set of serialized events;
discard, by the event cache manager, at least one serialized event from the set of serialized events;
store, by the event cache manager, the set of serialized events in a cache;
transmit, by the event cache manager, one or more of the serialized events from the cache to a next step predictor;
predict, by the next step predictor, a next UI step, wherein predicting the next UI step includes applying a machine learning model to at least a portion of the serialized events transmitted by the event cache manager; and
cause the dialog system to indicate the next UI step; and
a memory coupled to the processor and configured to provide the processor with instructions.
13 . The system of claim 12 , wherein discarding the at least one serialized event from the set of serialized events further comprises:
detecting a duplicate serialized event in the at least one serialized event; and
discarding the duplicate serialized event.
14 . The system of claim 13 , wherein the duplicate serialized event is a serialized UI event, and wherein detecting the duplicate serialized event comprises:
determining that the duplicate serialized event and another serialized UI event are: (i) of a same type, and (ii) an on-screen distance therebetween is below a predetermined distance.
15 . The system of claim 13 , wherein the duplicate serialized event is a serialized UI event, and wherein detecting the duplicate serialized event comprises:
determining that the duplicate serialized event and another serialized UI event: (i) are of a same type but with different parameters, and (ii) both occurred within a time window of a predetermined period.
16 . The system of claim 12 , wherein the processor is further configured to:
determine that the cache is full; and
deleting, from the cache, one or more cached serialized events in an ordering of: UI events from a prior session, then other UI events, then system responses, and then user prompts.
17 . The system of claim 12 , wherein the processor is further configured to:
provide, by the event cache manager, the one or more of the serialized events to a training and analysis data store.
18 . The system of claim 12 , wherein the processor is further configured to:
highlight, by a UI highlighter, an element of a webpage interfacing with the dialog system; and
remove the highlight in response to a user interfacing with the element or after a pre-determined period of time passes.
19 . A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:
tracking a plurality of user interface (UI) events associated with interaction of a user with a UI;
tracking a plurality of user prompts associated with interaction of the user with a dialog system;
serializing the plurality of UI events into serialized UI events in a serialization format;
serializing the plurality of user prompts into serialized prompt events in the serialization format;
receiving, by an event cache manager, the serialized UI events and the serialized prompt events as a set of serialized events;
discarding, by the event cache manager, at least one serialized event from the set of serialized events;
storing, by the event cache manager, the set of serialized events in a cache;
transmitting, by the event cache manager, one or more of the serialized events from the cache to a next step predictor;
predicting, by the next step predictor, a next UI step, wherein predicting the next UI step includes applying a machine learning model to at least a portion of the serialized events transmitted by the event cache manager; and
causing the dialog system to indicate the next UI step.