Visual display systems and method for manipulating images of a real scene using augmented reality
The present disclosure relates to a visual display system for manipulating images of a real scene using augmented reality. In one implementation, the system may include at least one processor in communication with a first mobile device; and a storage medium storing instructions that, when executed, configure the at least one processor to perform operations. The operations may include receiving a request from a mobile device to access an account of a user, receiving a first image depicting a real scene from an image sensor of the mobile device, receiving a selection of a virtual object, receiving an augmented reality image comprising the virtual object overlaid on the first image, comparing the augmented reality image to one or more stored augmented reality images, authenticating the user based on the comparison, and authorizing access to the user account based on the authentication.
1 . A system comprising:
one or more processors and non-transitory media storing instructions that, when executed by the one or more processors, cause operations comprising:
obtaining, via mobile device, video data of a real scene;
obtaining a user-selected indication related to a manipulation to the video data;
generating, based on the user-selected indication, augmented reality video data by applying the manipulation to the video data;
generating a confidence score based on comparing images derived from the augmented reality video data with a registration image; and
authenticating a user based on the confidence score being above a threshold.
2 . The system of claim 1 , wherein the augmented reality video data comprises a modified instance of the video data that reflects the manipulation to the video data.
3 . The system of claim 1 , wherein generating the confidence score comprises:
extracting machine learning features from the images derived from the augmented reality video data; and
generating the confidence score based on comparing the machine learning features derived from the augmented reality video data with machine learning features of the registration image.
4 . The system of claim 1 , wherein generating the confidence score comprises obtaining the confidence score by inputting representations of the images derived from the augmented reality video data into an input layer of a neural network comprising the input layer and one or more middle layers, and an output layer, the input layer being configured to receive a matrix for red, green, and blue pixels of an image derived from the augmented reality video data, each of the one or more middle layers and the output layer comprising a function configured to receive one or more values outputted by a preceding layer of the neural network.
5 . A method comprising:
obtaining a user-provided indication related to a manipulation to image data obtained via a user device of a user;
generating, based on the user-provided indication, a modified image by applying the manipulation to the image data obtained via the user device;
generating a confidence score based on comparing the modified image with a prestored image; and
authenticating the user of the user device, wherein authenticating the user comprises:
determining that the confidence score is below a threshold and within a predetermined range of the threshold;
based on the confidence score being below the threshold and within the predetermined range of the threshold, initiating a verification challenge different from the comparison of the modified image with the prestored image; and
authenticating the user based on completion of the verification challenge.
6 . The method of claim 5 , wherein generating the confidence score comprises:
extracting machine learning features from the modified image; and
generating the confidence score based on comparing the machine learning features from the modified image with machine learning features of the prestored image.
7 . The method of claim 5 , wherein generating the confidence score comprises obtaining the confidence score by inputting a representation of the modified image into an input layer of a neural network comprising the input layer and one or more middle layers, and an output layer, the input layer being configured to receive a matrix for red, green, and blue pixels in the modified image, each of the one or more middle layers and the output layer comprising a function configured to receive one or more values outputted by a preceding layer of the neural network.
8 . The method of claim 5 , wherein comparing the modified image comprises comparing features of image regions, modified to overlay a virtual object selected by the user on the image data, with features of the prestored image.
9 . The method of claim 5 , wherein comparing the modified image comprises comparing features of image regions, modified to overlay a first virtual object at a first image location on the image data and a second virtual object at a second image location, different from the first image location, on the image data, with features of the prestored image.
10 . The method of claim 5 , wherein initiating the verification challenge comprises generating a one-time temporary authentication token based on the confidence score, derived from the comparison of the modified image with the prestored image, being below the threshold and within the predetermined range of the threshold.
11 . The method of claim 5 , wherein generating the confidence score comprises generating the confidence score for authenticating the user based on comparing the modified image with the prestored image and one or more lighting conditions associated with the image data with one or more lighting conditions associated with the prestored image.
12 . One or more non-transitory computer-readable media storing instructions, that when executed by one or more processors, cause operations comprising:
obtaining a user-provided indication related to a manipulation to image data obtained via a user device of a user;
generating, based on the user-provided indication, a modified image by applying the manipulation to the image data obtained via the user device;
generating a confidence score based on comparing the modified image with a prestored image; and
authenticating the user of the user device, wherein authenticating the user comprises:
determining that the confidence score is below a threshold and within a predetermined range of the threshold;
based on the confidence score being below the threshold and within the predetermined range of the threshold, initiating a verification challenge different from the comparison of the modified image with the prestored image; and
authenticating the user based on completion of the verification challenge.
13 . The one or more non-transitory computer-readable media of claim 12 , wherein comparing the modified image with the prestored image comprises:
extracting machine learning features from the modified image; and
comparing the modified image with the prestored image based on the machine learning features from the modified image with machine learning features of the prestored image.
14 . The one or more non-transitory computer-readable media of claim 12 , wherein authenticating the user of the user device comprises:
obtaining, as part of an image file comprising the image data, geolocation data in connection with using the image data for authenticating the user; and
authenticating the user based on the completion of the verification challenge and the geolocation data.
15 . The one or more non-transitory computer-readable media of claim 12 , wherein comparing the modified image comprises comparing features of image regions, modified to overlay a virtual object selected by the user on the image data, with features of the prestored image.
16 . The one or more non-transitory computer-readable media of claim 12 , wherein comparing the modified image comprises comparing features of image regions, modified to overlay a first virtual object at a first image location on the image data and a second virtual object at a second image location, different from the first image location, on the image data, with features of the prestored image.
17 . The one or more non-transitory computer-readable media of claim 12 , wherein generating the confidence score comprises generating the confidence score for authenticating the user based on comparing the modified image with the prestored image and geolocation metadata associated with the image data with geolocation metadata associated with the prestored image.
18 . The one or more non-transitory computer-readable media of claim 12 , wherein generating the confidence score comprises generating the confidence score for authenticating the user based on comparing the modified image with the prestored image and a device identifier associated with the image data with a device identifier associated with the prestored image.
19 . The one or more non-transitory computer-readable media of claim 12 , wherein generating the confidence score comprises generating the confidence score for authenticating the user based on comparing the modified image with the prestored image and one or more lighting conditions associated with the image data with one or more lighting conditions associated with the prestored image.