Object detection and anchor position and orientation resolution for augmented reality (AR) applications
Interesting features of an object within a physical environment are identified from a video of the physical environment. A model of the object is obtained, the model includes dimensions of the object. A first array is maintained with the pixel coordinates of the identified features within the video. A second array is maintained with the dimensions of the features obtained from the model. A processing framework is provided for using the first and second arrays and estimating, updating, and tracking the anchor pose in the physical environment within the video along with any AR generated objects placed in the video. The framework provided to an Augmented Reality (AR) app to initiate an AR session and to maintain the session.
1. A method, comprising:
identifying a feature associated with a real-world object in a video of an Augmented Reality (AR) application (app) during AR session initiation of the AR app within a physical environment that includes the real-world object;
maintaining pixel coordinates for the feature within the video;
wherein maintaining the pixel coordinates further includes:
maintaining the pixel coordinates for the feature and additional pixel coordinates for additional features of the real-world object in a two-dimensional (2D) array;
maintaining sizes and dimensions of the feature based on a model that comprises sizes and dimensions of the real-world object;
wherein maintaining the sizes further includes:
maintaining the sizes and dimensions for the feature and additional sizes and dimensions for the additional features of the real-world object in a three-dimensional (3D) array; and
estimating a position and an orientation of the real-world object as depicted in the video based on the pixel coordinates, the sizes of the feature, and the dimensions of the feature;
wherein estimating further includes:
obtaining intrinsic camera parameters for a camera that is supplying the video from the AR app; and
wherein estimating further includes processing a Perspective-n-Point algorithm using the 2D array, the 3D array, and the intrinsic camera parameters to estimate the position and the orientation of the real-world object within the video.
2. The method of claim 1 further comprising, processing the method as a framework for the AR app to track and update the position and the orientation of the real-world object within the video as an anchor object for the AR session.
3. The method of claim 2 further comprising, processing the framework for tracking and updating a position and an orientation associated with an AR-generated object superimposed within the video relative to the position and orientation of the anchor object by the AR app during the AR session.
4. The method of claim 3 further comprising, providing the framework to the AR app through an Application Programming interface.
5. The method of claim 1 , wherein identifying further includes identifying the feature from the video based on a known feature associated with the real-world object.
6. The method of claim 1 , wherein identifying further includes selecting the feature from a plurality of know features associated with the real-world object.
7. A method, comprising:
maintaining a computer-aided design (CAD) model for a terminal;
receiving an Augmented Reality (AR) session initiation request from an AR application (app) that comprises a video of a physical environment with the terminal located within the physical environment;
identifying three or more features of the terminal from the video;
obtaining sizes and dimensions for the three or more features from the CAD model;
wherein obtaining the sizes further includes:
maintaining pixel coordinates, the sizes, and the dimensions for the three or more features in a two-dimensional (2D) array and a three-dimensional (3D) array;
obtaining intrinsic camera parameters for a camera that is supplying the video; and
using pixel coordinates for the three or more features within a screen rendered on a device that comprises the camera, the sizes, the dimensions, and the intrinsic camera parameters to estimate a position and an orientation of the terminal within the video and initializing an AR session for the AR app responsive to the AR session initiation request;
wherein using further includes:
processing a Perspective-n-Point (PnP) algorithm using the 2D array, the 3D array, and the intrinsic camera parameters to estimate the position and the orientation of the terminal within the video.
8. The method of claim 7 further comprising, continuously iterating to the using during the AR session to track and to update the position and the orientation of the terminal within the video and to track and to update a position and an orientation of AR generated objects within the video during the AR session.
9. The method of claim 7 , wherein receiving further includes identifying the AR session initiation request as a maintenance or service action that is to be performed at the terminal by an engineer.
10. The method of claim 7 , wherein identifying further includes selecting the three or more features from a plurality of features based on visual attributes of the three or more features detected in the video.
11. The method of claim 7 further comprising tracking after AR session initiation the video and updating the position and the orientation of the terminal within the video along with any AR-generated objects rendered within the video during the AR session.
12. The method of claim 7 further comprising, providing the method as a cloud-based software-as-a-service (SaaS) to the AR app.