Augmented reality content selection and display based on printed two-dimensional objects
Systems, methods and techniques for automatically recognizing two-dimensional real world objects with an augmented reality display device, and augmenting or enhancing the display of such real world objects by superimposing virtual images such as a still or video advertisement, a story or other virtual image presentation. In non-limiting embodiments, the real world object includes visible features including visible security features and a recognition process takes the visible security features into account when recognizing the object and/or displaying superimposed virtual images.
1 . A messaging method comprising:
(a) capturing an image of a two-dimensional planar object;
(b) using at least one processor, analyzing the captured image and recognizing the two-dimensional planar object using an area of recognition within the captured image without relying on any bar code or serial number on the two-dimensional planar object, the area of recognition within the captured image covering only a portion of the area of the captured image of the two-dimensional planar object, wherein recognizing does not distinguish between an original and a copy or counterfeit;
(c) determining whether the recognized two-dimensional planar object matches data in a database; and
(d) if the recognized two-dimensional planar object is determined to match data in the database, displaying a user-interactive message customized to the recognized two-dimensional planar object anchored to a displayed image of the recognized two-dimensional planar object, the anchoring being configured such that the user-interactive message appears to remain anchored to the displayed captured image of the two-dimensional planar object across different viewing perspectives of the camera relative to the two-dimensional planar object.
2 . The messaging method of claim 1 wherein the two-dimensional planar object comprises a printed document and the further message comprises a video or image.
3 . The messaging method of claim 1 wherein the further message comprises a superimposed virtual image comprising a still or video advertisement.
4 . The messaging method of claim 1 wherein recognizing comprises using artificial intelligence and machine learning to match the two-dimensional planar object.
5 . The messaging method of claim 1 further including changing position and/or orientation of the image of the two-dimensional planar object on the display, and in response moving the further message with the image of the two-dimensional planar object on the display as if it were glued or otherwise adhered to the two-dimensional planar object or were being displayed by the two-dimensional planar object.
6 . The messaging method of claim 1 further including changing the position and/or orientation of the image of the two-dimensional planar object on the display, and in response changing position, orientation and/or perspective of the further message as the two-dimensional planar object changes position, orientation and/or viewing angle in order to provide a photorealistic image in which the further message appears to be part of the displayed two-dimensional planar object.
7 . The messaging method of claim 1 further including using a handheld camera to capture the image.
8 . The messaging method of claim 1 further including accessing the database over a network.
9 . The messaging method of claim 1 further including superimposing the further message onto a display of the captured image.
10 . A messaging system comprising:
a handheld housing having a display thereon;
a camera disposed on the housing and adapted to capture an image of a two-dimensional planar object;
a processing system operatively connected to the camera, the processing system configured to perform operations comprising
recognize the two-dimensional planar object based on an area of recognition within the captured image without relying on any bar code or serial number on the two-dimensional planar object, the area of recognition within the captured image covering only a portion of the area of the captured image of the two-dimensional planar object, wherein recognizing does not distinguish between an original and a copy or counterfeit;
determine whether the recognized two-dimensional planar object matches data in a database; and
if the recognized two-dimensional planar object is determined to match data in the database, display the captured image on the display disposed on the handheld housing with a further user-interactive message customized to the recognized two-dimensional planar object anchored to the two-dimensional planar object within the captured image, the anchoring being configured such that the user-interactive message appears to remain anchored to the displayed captured image of the two-dimensional planar object across different viewing perspectives of the camera relative to the two-dimensional planar object.
11 . The messaging system of claim 10 wherein the further message comprises a video or image.
12 . The messaging system of claim 10 wherein the further message comprises a superimposed virtual image comprising a still or video advertisement.
13 . The messaging system of claim 10 wherein the at least one processor is configured to use artificial intelligence and machine learning to match the two-dimensional planar object.
14 . The messaging system of claim 10 wherein the processor is further configured to change the position and/or orientation of the image of the two-dimensional planar object on the display, and correspondingly move the further message with the image of the two-dimensional planar object on the display as if it were glued or otherwise adhered to the two-dimensional planar object or were being displayed by the two-dimensional planar object.
15 . The messaging system of claim 10 wherein the processor is further configured to change the position and/or orientation of the image of the two-dimensional planar object on the display, and correspondingly the position, orientation and/or perspective of the further message as the two-dimensional planar object changes position, orientation and/or viewing angle in order to provide a photorealistic image in which the further message appears to be part of the underlying two-dimensional planar object.
16 . The messaging system of claim 10 wherein the processor is configured to access the database over a network.
17 . The messaging system of claim 10 wherein the processor is further configured to superimpose the further message onto the displayed captured image.
18 . A messaging system comprising:
a handheld housing having a display thereon;
a camera disposed on the housing and adapted to capture an image of a two-dimensional planar object;
at least one processor operatively connected to the camera, the processor configured to perform operations comprising
recognize the two-dimensional planar object based on an area of recognition within the captured image without relying on any bar code or serial number on the two-dimensional planar object, the area of recognition within the captured image covering only a portion of the area of captured image of the two-dimensional planar object, wherein recognizing does not distinguish between an original and a copy or counterfeit;
connect to a network to determine whether the recognized two-dimensional planar object matches data in a remotely stored database; and
if the recognized two-dimensional planar object is determined to match data in the remotely stored database, display the captured image on the display disposed on the handheld housing with a further user-interactive message customized to the recognized two-dimensional planar object anchored to the two-dimensional planar object within the captured image, the anchoring being configured such that the user-interactive message appears to remain anchored to the displayed captured image of the two-dimensional planar object across different viewing perspectives of the camera relative to the two-dimensional planar object.
19 . The messaging system of claim 18 wherein the further message comprises a video or image.
20 . The messaging system of claim 18 wherein the further message comprises a superimposed virtual image comprising a still or video advertisement.