Two-sided machine learning framework for pointer movement-based bot detection
Methods and systems are presented for bot detection. A movement of a pointing device is tracked via a graphical user interface (GUI) of an application executable at a user device. Movement data associated with different locations of the pointing device within the GUI is obtained. The movement data is mapped to functional areas corresponding to a range of the different locations of the pointing device within the GUI over consecutive time intervals. At least one vector representing a sequence of movements for at least one trajectory of the pointing device through one or more of the functional areas and a duration the pointing device stays within each functional area is generated. At least one trained machine learning model is used to determine whether the sequence of movements of the pointing device was produced through human interaction with the pointing device by an actual user of the user device.
1. A system comprising:
one or more hardware processors; and
a non-transitory memory having stored therein instructions that are executable by the one or more hardware processors to cause the system to perform operations comprising:
tracking, via a graphical user interface (GUI) of an application executable at a user device, a movement of a pointing device over time;
obtaining, based on the tracking, movement data associated with different locations of the pointing device within the GUI between consecutive time intervals, the movement data including a set of coordinates and a timestamp for each of the different locations of the pointing device between the consecutive time intervals;
mapping the movement data to functional areas corresponding to a range of the different locations of the pointing device within the GUI over the consecutive time intervals;
generating, based on the mapped movement data, at least one vector representing a sequence of movements for at least one trajectory of the pointing device through one or more of the functional areas and a duration the pointing device stays within each functional area; and
determining, using at least one trained machine learning model and based on the at least one vector, whether the sequence of movements for the at least one trajectory of the pointing device was produced through human interaction with the pointing device by an actual user of the user device.
2. The system of claim 1 , wherein the at least one trained machine learning model includes at least one data encoder and at least one data decoder, and wherein the operations further comprise:
generating, using the at least one data encoder, a feature representation of the at least one vector with encoded features to be decoded and used by the at least one data decoder for predicting a likelihood that the sequence of movements for the at least one trajectory of the pointing device was produced by the actual user of the user device.
3. The system of claim 2 ,
wherein the set of coordinates for each of the different locations of the pointing device includes a coordinate for each axis in a plurality of coordinate axes of a multi-dimensional coordinate space corresponding to an interactive content display area of the GUI, and
wherein the at least one data encoder includes a plurality of encoding networks for encoding the at least one vector, each encoding network encoding a sequence of coordinates for a corresponding axis in the plurality of coordinate axes.
4. The system of claim 2 , wherein:
the at least one data encoder is a front-end encoder implemented in the application executable at the user device;
the at least one data decoder is a back-end decoding classifier implemented at a server communicatively coupled to the user device over a network;
the feature representation is transmitted from the front-end encoder to the back-end decoding classifier via the network; and
the determination of whether the sequence of movements for the at least one trajectory of the pointing device was produced by the actual user of the user device is based on a prediction received from the back-end decoding classifier via the network in response to the transmitted feature representation.
5. The system of claim 4 , wherein the back-end decoding classifier is trained using training data mapped to labeled features of the feature representation generated by the front-end encoder.
6. The system of claim 4 , wherein the application executable at the user device is a web browser, and wherein the back-end decoding classifier at the server is associated with a web service accessible via a corresponding website loaded within the web browser.
7. The system of claim 3 , wherein the encoded features include a plurality of biometric features obtained from the tracked movement of the pointing device within the interactive content display area of the GUI over the consecutive time intervals.
8. The system of claim 7 , wherein the plurality of biometric features include: an acceleration of the pointing device; an angle of the movement; a Euclidean norm of the set of coordinates along each axis of the multi-dimensional coordinate space; a curvature of the movement; a movement efficiency; a maximum time interval; and an absolute distance between the different locations of the pointing device.
9. A method comprising:
obtaining, by a server from an application executable at a user device via a network, a feature representation of movement data corresponding to different locations of a pointing device within a graphical user interface (GUI) of the application, the movement data including a set of coordinates and a timestamp for each of the different locations of the pointing device between consecutive time intervals;
extracting, by the server from the feature representation, a plurality of features representing a sequence of movements for at least one trajectory of the pointing device through one or more functional areas of the GUI and a duration the pointing device stays within each functional area;
predicting, by the server using at least one machine learning model and based on the plurality of features, a likelihood that the sequence of movements for the at least one trajectory of the pointing device was produced through human interaction with the pointing device by an actual user of the user device; and
transmitting the prediction from the server to the application at the user device via the network.
10. The method of claim 9 , wherein the at least one machine learning model includes at least one data decoder, and the feature representation of the movement data was generated by at least one data encoder for the at least one data decoder to predict the likelihood that the sequence of movements for the at least one trajectory of the pointing device was produced by the actual user of the user device.
11. The method of claim 10 ,
wherein the set of coordinates for each of the different locations of the pointing device includes a coordinate for each axis in a plurality of coordinate axes of a multi-dimensional coordinate space corresponding to an interactive content display area of the GUI, and
wherein the at least one data encoder includes a plurality of encoding networks for encoding the at least one vector, each encoding network encoding a sequence of coordinates for a corresponding axis in the plurality of coordinate axes.
12. The method of claim 10 , wherein:
the at least one data encoder is a front-end encoder implemented in the application executable at the user device;
the at least one data decoder is a back-end decoding classifier implemented at a server communicatively coupled to the user device over a network;
the feature representation is transmitted from the front-end encoder to the back-end decoding classifier via the network; and
the determination of whether the sequence of movements for the at least one trajectory of the pointing device was produced by the actual user of the user device is based on a prediction received from the back-end decoding classifier via the network in response to the transmitted feature representation.
13. The method of claim 12 , wherein the back-end decoding classifier is trained using training data mapped to labeled features of the feature representation generated by the front-end encoder.
14. The method of claim 12 , wherein the application executable at the user device is a web browser, and wherein the back-end decoding classifier at the server is associated with a web service accessible via a corresponding website loaded within the web browser.
15. The method of claim 11 , wherein the plurality of features includes a plurality of biometric features obtained from the tracked movement of the pointing device within the interactive content display area of the GUI over the consecutive time intervals.
16. The method of claim 15 , wherein the plurality of biometric features include: an acceleration of the pointing device; an angle of the movement; a Euclidean norm of the set of coordinates along each axis of the multi-dimensional coordinate space; a curvature of the movement; a movement efficiency; a maximum time interval; and an absolute distance between the different locations of the pointing device.
17. A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising:
tracking, via a graphical user interface (GUI) of an application executable at a user device, a movement of a pointing device;
obtaining, based on the tracking, movement data associated with different locations of the pointing device within the GUI between consecutive time intervals, the movement data including a set of coordinates and a timestamp for each of the different locations of the pointing device between the consecutive time intervals;
mapping the movement data to functional areas corresponding to a range of the different locations of the pointing device within the GUI over the consecutive time intervals;
generating a feature representation of the mapped movement data for at least one trajectory of the pointing device through one or more of the functional areas and a duration the pointing device stays within each functional area; and
determining, using at least one trained machine learning model and based on the generated feature representation, whether the at least one trajectory of the pointing device was produced by human interaction with the pointing device.
18. The non-transitory machine-readable medium of claim 17 , wherein the feature representation includes at least one vector representing a sequence of movements for the at least one trajectory of the pointing device through the one or more of the functional areas.
19. The non-transitory machine-readable medium of claim 18 , wherein the at least one trained machine learning model includes at least one data encoder and at least one data decoder, and wherein the operations further comprise:
generating, using the at least one data encoder, the feature representation of the at least one vector with encoded features to be decoded and used by the at least one data decoder for predicting a likelihood that the sequence of movements for the at least one trajectory of the pointing device was produced by human interaction with the pointing device.
20. The non-transitory machine-readable medium of claim 19 ,
wherein the set of coordinates for each of the different locations of the pointing device includes a coordinate for each axis in a plurality of coordinate axes of a multi-dimensional coordinate space corresponding to an interactive content display area of the GUI, and
wherein the at least one data encoder includes a plurality of encoding networks for encoding the at least one vector, each encoding network encoding a sequence of coordinates for a corresponding axis in the plurality of coordinate axes.