Augmented reality system using visual object recognition and stored geometry to create and render virtual objects
A three-dimensional geometry of an object is stored, and a representation of the object that can facilitate identification of the object via a camera is also stored. An image of the object is obtained via a scene in the camera. The presence of the object is detected from the image via the stored representation of the object. The three-dimensional geometry to the object is mapped within the scene based on the detection. A virtual object is attached to a point that is at a fixed orientation relative to the three-dimensional geometry based on the mapping. The virtual object is rendered on an augmented reality display as being located at the point and at the fixed orientation regardless of a change in location of the augmented reality display relative to the object.
1. A method comprising:
storing in memory a three-dimensional geometry of an object and a representation of the object that can facilitate identification of the object via a camera;
obtaining an image of the object via a scene in the camera;
based on the image, detecting a presence of the object via the stored representation of the object;
based on detecting the presence of the object, mapping the three-dimensional geometry to the object within the scene;
based on the mapping, attaching a virtual object to a point that is at a fixed orientation relative to the three-dimensional geometry, wherein attaching the virtual object comprises selecting the point on a surface of the object in a live view of the object on a display of a creating device and using a software rendering tool to create the virtual object via the display; and
rendering the virtual object on an augmented reality display as being located at the point and at the fixed orientation regardless of a change in location of the augmented reality display relative to the object.
2. The method of claim 1 , wherein the object comprises a device, and wherein the virtual object comprises an instruction for servicing the device.
3. The method of claim 2 , wherein the instruction comprises an animation superimposed over a part of the object.
4. The method of claim 2 , wherein a user servicing the device causes a change in geometry of the object, the method further comprising:
mapping a second three-dimensional geometry to the object within the scene corresponding to the change in geometry; and
rendering a second instruction at a second point that is fixed relative to the second three-dimensional geometry.
5. The method of claim 1 , wherein attaching the virtual object comprises selecting the point in the live view of the object on the display of the creating device and using the software rendering tool to create the virtual object via the display.
6. The method of claim 5 , wherein the location of the attached virtual object is determined by a position of the object within the live view.
7. The method of claim 6 , wherein the attached virtual object comprises at least one of a graphic, a handwritten note, a video, and an animation.
8. The method of claim 1 , wherein the three-dimensional geometry, data describing the point, and data describing the virtual object are stored on a network database, and wherein mapping the three-dimensional geometry to the object within the scene comprises:
communicating an identifier of the object to the network database in response to detecting the presence of the object; and
retrieving the three-dimensional geometry, the data describing the point, and the data describing the virtual object based on communicating the identifier.
9. The method of claim 1 , further comprising:
detecting a second instance of the object in another location different than the location of the object via a second camera;
mapping the three-dimensional geometry to the second instance of the object within the other location; and
rendering a second video on a second display based on a streaming input from the second camera, the virtual object being rendered with the video as being located at the point on the second instance of the object.
10. The method of claim 1 , wherein the stored representation comprises a machine learning classifier.
11. The method of claim 1 , wherein the stored representation comprises a code that is readable from the object via any of a machine-readable code or human-readable text.
12. A system comprising:
a creating device comprising a first processor coupled to a first camera, the first processor configured to:
obtain an image of an object via a scene in the first camera;
based on the image, detect a presence of the object via a stored representation of the object;
based on detecting the presence of the object, map a three-dimensional geometry to the object within the scene; and
based on the mapping, attach a virtual object to a point that is at a fixed orientation relative to the three-dimensional geometry, wherein attaching the virtual object comprises selecting the point on a surface of the object in a live view of the object on a display of a creating device and using a software rendering tool to create the virtual object via the display; and
a display device comprising a second processor coupled to a second camera memory, the second processor configured to:
detect the object or an equivalent object via the second camera, and in response thereto, map the three-dimensional geometry to the object or the equivalent object; and
render the virtual object on an augmented reality display as being located at the point and at the fixed orientation regardless of a change in location of the display device relative to the object.
13. The system of claim 12 , wherein the object comprises a device, and wherein the virtual object comprises an instruction for servicing the device.
14. The system of claim 13 , wherein a user servicing the device via the viewing device causes a change in geometry of the object or the equivalent object, the system further comprising:
mapping a second three-dimensional geometry to the object within the scene corresponding to the change in geometry; and
rendering a second instruction at a second point that is fixed relative to the second three-dimensional geometry.
15. The system of claim 12 , wherein the location of the attached virtual object is determined by a position of the object within the live view.
16. The system of claim 15 , wherein the attached virtual object comprises at least one of a graphic, a handwritten note, a video, and an animation.
17. The system of claim 12 , wherein the three-dimensional geometry, data describing the point, and data describing the virtual object are stored on a network database, and wherein mapping the three-dimensional geometry to the object within the scene comprises:
communicating an identifier of the object to the network database in response to detecting the presence of the object; and
retrieving the three-dimensional geometry, the data describing the point, and the data describing the virtual object based on communicating the identifier.
18. The system of claim 12 , wherein the object or the equivalent object comprises the equivalent object, and wherein the equivalent object is in a different location than the object.
19. The system of claim 12 , wherein the stored representation comprises a machine learning classifier.