IP Library › Granted Patent US 12,287,912
Granted Patent B2
US 12,287,912 · App. 18/164,740 · Granted Apr 29, 2025

External device communication with virtual reality equipment

Inventors: June-Ray Lin (Taipei, TW); Jia Mao (Shanghai, CN); Nuo Xu (Hangzhou, CN); Yuan Li (Ningbo, CN)
Assignee: International Business Machines Corporation
G06F3/011G06F3/0346
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,287,912
App. No.
18/164,740
Granted
Apr 29, 2025
Kind
B2
Abstract

Embodiments are related to providing external device communication and localization for virtual reality based equipment using radio-frequency identification (RFID). At least two receivers and a transmitter are used to recognize an external device and determine a location of the external device relative to the headset, based on tags coupled to the external device. A three-dimensional (3D) model is downloaded of the external device based on information received by the at least two receivers from the tags. A location of the external device is matched to the 3D model based on the tags. A virtual image is displayed of the external device corresponding to the location of the external device.

Claims (49)

1. A computer-implemented method comprising:

receiving, by a processor of a virtual reality (VR) device, non-optical signals comprising pointer data transmitted from an external device;

capturing an image of the external device;

using, by the processor, the pointer data to select and access a model of the external device, wherein the non-optical signals are received from fixed positions on the external device, the model of the external device comprising coordinates of the fixed positions on the external device;

determining, by the processor, a location of the external device, based on the non-optical signals transmitted from the external device;

validating a rotation of the external device based on the image captured of the external device and the fixed positions from which the non-optical signals are received from the external device; and

generating, by the processor, a virtual representation of the external device based at least in part on the location, the rotation, and the model.

2. The computer-implemented method of claim 1 further comprising determining a rotation of the external device based on transmission devices coupled to the external device, the transmission devices being configured to transmit the non-optical signals.

3. The computer-implemented method of claim 1 further comprising matching the virtual representation to the rotation of the external device.

4. The computer-implemented method of claim 1 further comprising determining a distance of the external device relative to the VR device, based on transmission devices coupled to the external device, the transmission devices being configured to transmit the non-optical signals.

5. The computer-implemented method of claim 1 , wherein:

the virtual representation is displayed according to the location, the rotation, and a distance of the external device relative to the VR device, the VR device comprising at least two receivers and a transmitter;

transmission devices are coupled to the external device; and

the location, the rotation, and the distance are based at least in part on a relationship between the at least two receivers, the transmitter, and the transmission devices.

6. The computer-implemented method of claim 1 , wherein the pointer data comprises a type of the external device and a uniform resource locator (URL) to access the model.

7. The computer-implemented method of claim 1 , wherein the external device comprises a keyboard having the fixed positions for the non-optical signals.

8. A system for a virtual reality (VR) device comprising:

a memory having computer readable instructions; and

a computer for executing the computer readable instructions, the computer readable instructions controlling the computer to perform operations comprising:

receiving, by the computer, non-optical signals comprising pointer data transmitted from an external device;

capturing an image of the external device;

using, by the computer, the pointer data to select and access a model of the external device, wherein the non-optical signals are received from fixed positions on the external device, the model of the external device comprising coordinates of the fixed positions on the external device;

determining, by the computer, a location of the external device, based on the non-optical signals transmitted from the external device;

validating a rotation of the external device based on the image captured of the external device and the fixed positions from which the non-optical signals are received from the external device; and

generating, by the computer, a virtual representation of the external device based at least in part on the location, the rotation, and the model.

9. The system of claim 8 , wherein the operations further comprise determining a rotation of the external device based on transmission devices coupled to the external device, the transmission devices being configured to transmit the non-optical signals.

10. The system of claim 8 , wherein the operations further comprise matching the virtual representation to the rotation of the external device.

11. The system of claim 8 , wherein the operations further comprise determining a distance of the external device relative to the VR device, based on transmission devices coupled to the external device, the transmission devices being configured to transmit the non-optical signals.

12. The system of claim 8 , wherein:

the virtual representation is displayed according to the location, the rotation, and a distance of the external device relative to the VR device, the VR device comprising at least two receivers and a transmitter;

transmission devices are coupled to the external device; and

the location, the rotation, and the distance are based at least in part on a relationship between the at least two receivers, the transmitter, and the transmission devices.

13. The system of claim 8 , wherein the pointer data comprises a type of the external device and a uniform resource locator (URL) to access the model.

14. The system of claim 8 , wherein the external device comprises a keyboard having the fixed positions for the non-optical signals.

15. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer of a virtual reality (VR) device to cause the computer to perform operations comprising:

receiving, by the computer, non-optical signals comprising pointer data transmitted from an external device;

capturing an image of the external device;

using, by the computer, the pointer data to select and access a model of the external device, wherein the non-optical signals are received from fixed positions on the external device, the model of the external device comprising coordinates of the fixed positions on the external device;

determining, by the computer, a location of the external device, based on the non-optical signals transmitted from the external device;

validating a rotation of the external device based on the image captured of the external device and the fixed positions from which the non-optical signals are received from the external device; and

generating, by the computer, a virtual representation of the external device based at least in part on the location, the rotation, and the model.

16. The computer program product of claim 15 further comprising determining a rotation of the external device based on transmission devices coupled to the external device, the transmission devices being configured to transmit the non-optical signals.

17. The computer program product of claim 15 further comprising matching the virtual representation to the rotation of the external device.

18. The computer program product of claim 15 further comprising determining a distance of the external device relative to the VR device, based on transmission devices coupled to the external device, the transmission devices being configured to transmit the non-optical signals.

19. The computer program product of claim 15 , wherein:

the virtual representation is displayed according to the location, the rotation, and a distance of the external device relative to the VR device, the VR device comprising at least two receivers and a transmitter;

transmission devices are coupled to the external device; and

the location, the rotation, and the distance are based at least in part on a relationship between the at least two receivers, the transmitter, and the transmission devices.

20. The computer program product of claim 15 , wherein the pointer data comprises a type of the external device and a uniform resource locator (URL) to access the model.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 6, 2023
From: LIN, JUNE-RAY; MAO, JIA; XU, NUO; LI, YUAN
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 062598/0638 →
Continuity (1)
Related Publication 20240264659A1 · Aug 8, 2024
References Cited (12)
US 7706603B2 · Najafi et al. · 2010 [cited by applicant]
US 10928888B2 · Harvey et al. · 2021 [cited by applicant]
US 20090215536A1 · Yee · 2009 [cited by examiner]
US 20150312426A1 · Schoner · 2015 [cited by examiner]
US 20170185830A1 · Srivastava et al. · 2017 [cited by applicant]
US 20180173323A1 · Harvey · 2018 [cited by examiner]
US 20220044016A1 · Pan et al. · 2022 [cited by applicant]
EP 3133468A1 · 2017 [cited by applicant]
“Tetrahedral volume formula”, The mathematics of free thought, Zhihu, 2017, 8 pages, English translation. [cited by applicant]
Li, et al., “IDCam: Precise Item Identification for AR Enhanced Object Interactions”, IEEE, 2019, 7 pages. [cited by applicant]
Xie, et al., “TaggedAR: An RFID-based Approach for Recognition of Multiple Tagged Objects in Augmented Reality Systems”, IEEE Transactions on Mobile Commuting, IEEE Xplore Digital Library, 2018, 15 pages. [cited by applicant]
Xie, et al., “Tell Me What I See: Recognize RFID Tagged Objects in Augmented Reality Systems”, UBICOMP '16, Heidelberg, Germany, 2016, 12 pages. [cited by applicant]