Disposable sticker within augmented reality environment
A sticker system includes at least a head-mounted device and a sticker. In an embodiment, the sticker system receives an image of the sticker captured by an imaging sensor. The sticker system determines registration information by processing a fiducial marker of the sticker shown in the image. The sticker system determines location information of the HMD. Additionally, the sticker system determines a position of the sticker relative to the HMD using the registration information and the location information. The sticker system provides a (e.g., augmented reality) graphic to the HMD for presentation to the user based on the position of the sticker.
1. A system comprising:
a computing device;
a sticker including a fiducial marker; and
a head-mounted device (HMD) worn by a user and
comprising a non-transitory computer-readable storage medium storing instructions, the instructions when executed by one or more processors cause the one or more processors to perform steps including:
receiving an image of the sticker captured by an imaging sensor of the HMD, the image showing the fiducial marker;
determining registration information by processing the fiducial marker in the image;
determining location information of the HMD;
determining a position of the sticker relative to the HMD using the registration information and the location information;
providing a graphic for presentation by the HMD to the user based on the position of the sticker;
generating an encryption key using the registration information for a medical procedure, the encryption key unique to the sticker;
determining that the HMD and the computing device are being used for the medical procedure; and
responsive to determining that the HMD and the computing device are being used for the medical procedure, transmitting the location information from the HMD to the computing device over a wireless network communication using the encryption key.
2. The system of claim 1 , wherein the non-transitory computer-readable storage medium stores further instructions that when executed by the one or more processors cause the one or more processors to perform steps including:
determining additional registration information by processing an additional image showing the fiducial marker of the sticker, the additional image captured by an additional HMD; and
determining that the HMD and the additional HMD are being used for the medical procedure by comparing the registration information and the additional registration information.
3. The system of claim 1 , wherein the non-transitory computer-readable storage medium stores further instructions that when executed by the one or more processors cause the one or more processors to perform steps including:
determining position of the sticker relative to the HMD by determining a distance and angle in 3D between the sticker and the HMD.
4. The system of claim 3 , wherein size or orientation of the graphic is determined using the distance and angle.
5. The system of claim 1 , wherein the non-transitory computer-readable storage medium stores further instructions that when executed by the one or more processors cause the one or more processors to perform steps including:
generating augmented reality content as the graphic, wherein the HMD includes a display configured to present the augmented reality content.
6. The system of claim 1 , wherein the fiducial marker is a QR code or a barcode.
7. The system of claim 1 , wherein the non-transitory computer-readable storage medium stores further instructions that when executed by the one or more processors cause the one or more processors to perform steps including:
responsive to determining that the HMD and the computing device are being used for the medical procedure, synchronizing annotations during the medical procedure on the graphic presented by the HMD and one or more graphics presented by the computing device.
8. A method comprising:
receiving an image of a sticker captured by an imaging sensor, the image showing a fiducial marker of the sticker;
determining registration information by processing the fiducial marker of the sticker in the image;
determining location information of a head-mounted device (HMD) worn by a user;
determining a position of the sticker relative to the HMD using the registration information and the location information;
providing a graphic for presentation by the HMD to the user based on the position of the sticker;
generating an encryption key using the registration information for a medical procedure, the encryption key unique to the sticker;
determining that the HMD and a computing device are being used for the medical procedure; and
responsive to determining that the HMD and the computing device are being used for the medical procedure, transmitting the location information from the HMD to the computing device over a wireless network communication using the encryption key.
9. The method of claim 8 , further comprising:
determining additional registration information by processing an additional image showing the fiducial marker of the sticker, the additional image captured by an additional HMD; and
determining that the HMD and the additional HMD are being used for the medical procedure by comparing the registration information and the additional registration information.
10. The method of claim 8 , wherein determining position of the sticker relative to the HMD comprises:
determining a distance and angle in 3D between the sticker and the HMD.
11. The method of claim 10 , wherein size or orientation of the graphic is determined using the distance and angle.
12. The method of claim 8 , further comprising:
generating augmented reality content as the graphic, wherein the HMD includes a display configured to present the augmented reality content.
13. The method of claim 8 , wherein the imaging sensor is coupled to the HMD.
14. The method of claim 8 , wherein the fiducial marker is a QR code or a barcode.
15. The method of claim 8 , further comprising:
responsive to determining that the HMD and the computing device are being used for the medical procedure, synchronizing annotations during the medical procedure on the graphic presented by the HMD and one or more graphics presented by the computing device.
16. A non-transitory computer-readable storage medium storing instructions, the instructions when executed by one or more processors cause the one or more processors to perform steps including:
receiving an image of a sticker captured by an imaging sensor, the image showing a fiducial marker of the sticker;
determining registration information by processing the fiducial marker of the sticker in the image;
determining location information of a head-mounted device (HMD) worn by a user;
determining a position of the sticker relative to the HMD using the registration information and the location information;
providing a graphic for presentation by the HMD to the user based on the position of the sticker;
generating an encryption key using the registration information for a medical procedure, the encryption key unique to the sticker;
determining that the HMD and a computing device are being used for the medical procedure; and
responsive to determining that the HMD and the computing device are being used for the medical procedure, transmitting the location information from the HMD to the computing device over a wireless network communication using the encryption key.