Screen-analysis based device security
Systems and methods are provided for a content-based security for computing devices. An example method includes identifying content rendered by a mobile application, the content being rendered during a session, generating feature vectors from the content and determining that the feature vectors do not match a classification model. The method also includes providing, in response to the determination that the feature vectors do not match the classification model, a challenge configured to authenticate a user of the mobile device. Another example method includes determining a computing device is located at a trusted location, capturing information from a session, the information coming from content rendered by a mobile application during the session, generating feature vectors for the session, and repeating this until a training criteria is met. The method also includes training a classification model using the feature vectors and authenticating a user of the device using the trained classification model.
1. A method implemented by one or more processors, the method comprising:
identifying a user profile that corresponds to a user and that is active for an electronic device;
determining whether current activity of the electronic device, while the user profile is active for the electronic device, matches the user profile, wherein determining whether the current activity of the electronic device matches the user profile comprises:
processing, using a classification model, feature vectors that are generated based on content that is displayed on a screen of the electronic device during the current activity of the electronic device to determine whether the feature vectors corresponding to the content that is displayed on the screen of the electronic device during the current activity of the electronic device match the user profile; and
in response to determining the feature vectors corresponding to the current activity of the electronic device do not match the user profile:
determining whether the current activity of the electronic device, while the user profile is active for the electronic device, matches an additional user profile, of a plurality of additional user profiles, the additional user profile corresponding to an additional user; and
in response to determining the current activity of the electronic device matches the additional user profile:
initiating a profile switch, to the additional user profile, at the electronic device.
2. The method of claim 1 , wherein determining whether the current activity of the electronic device matches the additional user profile comprises:
processing, using the classification model, the feature vectors corresponding to the current activity of the electronic device to determine whether the feature vectors corresponding to the current activity of the electronic device match the additional user profile.
3. The method of claim 1 , further comprising:
in response to determining the feature vectors corresponding to the current activity of the electronic device do not match the additional user profile:
determining whether the current activity of the electronic device, while the user profile is active for the electronic device, matches a further additional user profile, of the plurality of additional user profiles, the further additional user profile corresponding to a further additional user; and
in response to determining the current activity of the electronic device matches the further additional user profile:
initiating the profile switch, to the further additional user profile, at the electronic device.
4. The method of claim 3 , wherein determining whether the current activity of the electronic device matches the further additional user profile comprises:
processing, using the classification model, the feature vectors corresponding to the current activity of the electronic device to determine whether the feature vectors corresponding to the current activity of the electronic device match the further additional user profile.
5. The method of claim 1 , wherein determining whether the current activity of the electronic device matches the user profile is in response to determining that the current activity at the electronic device corresponds to outlier activity for the user profile.
6. The method of claim 1 , wherein initiating the profile switch at the electronic device comprises:
automatically signing out of the user profile to deactivate the user profile at the electronic device; and
automatically signing in to the additional user profile to activate the additional user profile at the electronic device.
7. The method of claim 1 , wherein initiating the profile switch at the electronic device comprises:
automatically signing out of the user profile to deactivate the user profile at the electronic device; and
causing a sign-in user interface to be visually rendered at the electronic device to enable the additional user to sign-in to the additional user profile.
8. The method of claim 7 , further comprising:
in response to visually rendering the sign-in user interface at the electronic device:
receiving, at the electronic device, credentials to sign-in to the additional user profile; and
signing in to the additional user profile based on the credentials.
9. The method of claim 1 , wherein initiating the profile switch at the electronic device comprises:
causing a prompt to be visually rendered at the electronic device to request permission to switch to the additional user profile associated with the additional user.
10. The method of claim 9 , further comprising:
in response to visually rendering the prompt at the electronic device:
receiving, at the electronic device, user input to permit switching to the additional user profile.
11. The method of claim 1 , wherein generating the feature vectors based on the content that is displayed on the screen of the electronic device comprises:
processing at least a portion of the content that is displayed on the screen of the electronic device;
identifying one or more features included in the content that is displayed on the screen of the electronic device; and
generating the feature vectors using one or more of the features included in the content that is displayed on the screen of the electronic device.
12. A system comprising:
at least one processor; and
at least one memory storing instructions that, when executed, cause the at least one processor to:
identify a user profile that corresponds to a user and that is active for an electronic device;
determine whether current activity of the electronic device, while the user profile is active for the electronic device, matches the user profile, wherein the instructions to determine whether the current activity of the electronic device matches the user associated with the user profile comprise instructions to:
process, using a classification model, feature vectors that are generated based on content that is displayed on a screen of the electronic device during the current activity of the electronic device to determine whether the feature vectors corresponding to the content that is displayed on the screen of the electronic device during the current activity of the electronic device match the user profile;
in response to determining the feature vectors corresponding to the current activity of the electronic device do not match the user profile:
determine whether the current activity of the electronic device, while the user profile is active for the electronic device, matches an additional user profile, of a plurality of additional user profiles, the additional user profile corresponding to an additional user; and
in response to determining the current activity of the electronic device matches the additional user profile:
initiate a profile switch, to the additional user profile, at the electronic device.
13. The system of claim 12 , wherein the instructions to determine whether the current activity of the electronic device matches the additional user profile comprise instructions to:
process, using the classification model, the feature vectors corresponding to the current activity of the electronic device to determine whether the feature vectors corresponding to the current activity of the electronic device match the additional user profile.
14. The system of claim 12 , wherein the instructions further comprise instructions to:
in response to determining the feature vectors corresponding to the current activity of the electronic device do not match the additional user profile:
determine whether the current activity of the electronic device, while the user profile is active for the electronic device, matches a further additional user profile, of the plurality of additional user profiles, the further additional user profile corresponding to a further additional user; and
in response to determining the current activity of the electronic device matches the further additional user profile:
initiate the profile switch, to the further additional user profile, at the electronic device.
15. The system of claim 14 , wherein the instructions to determine whether the current activity of the electronic device matches the further additional user profile comprise instructions to:
process, using the classification model, the feature vectors corresponding to the current activity of the electronic device to determine whether the feature vectors corresponding to the current activity of the electronic device match the further additional user profile.
16. The system of claim 12 , wherein determining whether the current activity of the electronic device matches the user profile is in response to determining that the current activity at the electronic device corresponds to outlier activity for the user profile.
17. The system of claim 12 , wherein the instructions to initiate the profile switch at the electronic device comprise instructions to:
automatically sign-in of the user profile to deactivate the user profile at the electronic device; and
automatically sign-in to the additional user profile to activate the additional user profile at the electronic device.
18. The system of claim 12 , wherein the instructions to initiate the profile switch at the electronic device comprise instructions to:
automatically sign out of the user profile to deactivate the user profile at the electronic device; and
cause a sign-in user interface to be visually rendered at the electronic device to enable the additional user to sign-in to the additional user profile.
19. The system of claim 12 , wherein the instructions to initiate the profile switch at the electronic device comprise instructions to:
cause a prompt to be visually rendered at the electronic device to request permission to switch to the additional user profile associated with the additional user.
20. A non-transitory computer-readable storage medium storing instructions that, when executed, cause at least one processor to execute the instructions, the instructions causing the at least one processor to:
identify a user profile that corresponds to a user and that is active for an electronic device;
determine whether current activity of the electronic device, while the user profile is active for the electronic device, matches the user profile, wherein the instructions to determine whether the current activity of the electronic device matches the user profile comprise instructions to:
process, using a classification model, feature vectors that are generated based on content that is displayed on a screen of the electronic device during the current activity of the electronic device to determine whether the feature vectors corresponding to the content that is displayed on the screen of the electronic device during the current activity of the electronic device match the user profile;
in response to determining the feature vectors corresponding to the current activity of the electronic device do not match the user profile:
determine whether the current activity of the electronic device, while the user profile is active for the electronic device, matches an additional user profile, of a plurality of additional user profiles, the additional user profile corresponding to an additional user; and
in response to determining the current activity of the electronic device matches the additional user profile:
initiating a profile switch, to the additional user profile, at the electronic device.