IP Library › Granted Patent US 10,290,151
Granted Patent B2
US 10,290,151 · App. 15/239,346 · Granted May 14, 2019

AR/VR device virtualisation

Inventor: Phillip Riscombe-Burton (London, GB)
Assignee: BlackBerry Limited
G06T19/006G02B27/017G06F3/011G06F3/016G06F3/048G06F3/0481G06F9/451
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,290,151
App. No.
15/239,346
Granted
May 14, 2019
Kind
B2
Abstract

Methods, apparatuses and computer-readable media for virtualizing a physical device using an augmented reality (AR) or virtual reality (VR) virtualized or partially virtualized device (an AR/VR device) are provided. In particular, first data of a user interaction with the physical device is received by the AR/VR device from the physical device. The first data of the user interaction with the physical device is converted into second data for representation of the user interaction with the physical device in a virtual representation of the physical device. The virtual representation of the physical device and the representation of the user interaction with the physical device are outputted by the AR/VR device.

Claims (60)

1. An augmented reality (AR) or virtual reality (VR) device (AR/VR device), the AR/VR device comprising:

an AR/VR display;

one or more physical hardware components for presenting a virtual representation of a to-be-virtualized device by the AR/VR display;

at least one processor; and

a non-transitory computer-readable storage medium coupled to the at least one processor and storing programming instructions for execution by the at least one processor, the programming instructions comprising:

an emulation layer;

one or more hardware emulation application program interfaces (APIs); and

one or more hardware glue logics; and

wherein the programming instructions instruct the at least one processor to:

receive a code of core functionalities of the to-be-virtualized device, wherein the core functionalities comprises a functionality related to a hardware component that does not physically exist on the to-be-virtualized device; and

emulate, by the AR/VR device, the functionality related to the hardware component that does not physically exist on the to-be-virtualized device, wherein the emulating comprises:

receiving, by the emulation layer, a first data of a user interaction with the virtual representation of the to-be-virtualized device, the virtual representation comprising one or more of an image or a video visible to the user, the user interaction with the virtual representation of the to-be-virtualized device comprising one or more of a touch or a gesture of the user relative to the virtual representation of the to-be-virtualized device, the first data comprising an emulated output of the hardware component that does not physically exist on the to-be-virtualized device in response to the user interaction with the virtual representation of the to-be-virtualized device;

passing, via a hardware emulation API corresponding to the functionality related to the hardware component that does not physically exist on the to-be-virtualized device, the emulated output to a hardware glue logic corresponding to the hardware component that does not physically exist on the to-be-virtualized device; and

converting, by the hardware glue logic corresponding to the hardware component that does not physically exist on the to-be-virtualized device, the emulated output into the functionality related to the hardware component that does not physically exist on the to-be-virtualized device.

2. The AR/VR device of claim 1 , wherein the one or more physical hardware components comprises one or more of a screen, a camera, a microphone, a speaker, a connectivity or communication interface.

3. The AR/VR device of claim 1 , wherein the to-be-virtualized device comprises a virtual device that does not physically exist.

4. The AR/VR device of claim 1 , wherein the code of core functionalities of the to-be-virtualized device comprise a code written specifically for the to-be-virtualized device or a code of an equivalent real-world counterpart of the to-be-virtualized device.

5. The AR/VR device of claim 1 , wherein:

the hardware component that does not physically exist on the to-be-virtualized device comprises one or more of a gyroscope, a compass, a virtualized electromagnet, a rumble motor, or an exterior motor; and

the hardware emulation API corresponding to the functionality related to the hardware component that does not physically exist on the to-be-virtualized device comprises a movement API.

6. The AR/VR device of claim 1 , wherein:

the hardware component that does not physically exist on the to-be-virtualized device comprises one or more of a Bluetooth, Wi-Fi, cellular, Near Filed Communications (NFC), Universal Serial Bus (USB), or Global Positioning System (GPS), hardware component; and

the hardware emulation API corresponding to the functionality related to the hardware component that does not physically exist on the to-be-virtualized device comprises a connectivity API.

7. The AR/VR device of claim 1 , further operable to pair with the to-be-virtualized device.

8. A method for virtualizing a to-be-virtualized device using an augmented reality (AR) or virtual reality (VR) device (an AR/VR device), the method comprising:

receiving, by the AR/VR device, a code of core functionalities of the to-be-virtualized device, wherein the core functionalities comprises a functionality related to a hardware component that does not physically exist on the to-be-virtualized device;

presenting, by an AR/VR display of the AR/VR device, a virtual representation of the to-be-virtualized device; and

emulating, by the AR/VR device, the functionality related to the hardware component that does not physically exist on the to-be-virtualized device, wherein the emulating comprises:

receiving, by the AR/VR device, a first data of a user interaction with the virtual representation of the to-be-virtualized device, the virtual representation comprising one or more of an image or a video visible to the user, the user interaction with the virtual representation of the to-be-virtualized device comprising one or more of a touch or a gesture of the user relative to the virtual representation of the to-be-virtualized device, the first data comprising an emulated output of the hardware component that does not physically exist on the to-be-virtualized device in response to the user interaction with the virtual representation of the to-be-virtualized device;

passing, via a hardware emulation application program interface (API) corresponding to the functionality related to the hardware component that does not physically exist on the to-be-virtualized device, the emulated output to a hardware glue logic corresponding to the hardware component that does not physically exist on the to-be-virtualized device; and

converting, by the hardware glue logic corresponding to the hardware component that does not physically exist on the to-be-virtualized device, the emulated output into the functionality related to the hardware component that does not physically exist on the to-be-virtualized device.

9. The method of claim 8 , further comprising pairing the to-be-virtualized device with the AR/VR device.

10. The method of claim 8 , wherein the to-be-virtualized device comprises a virtual device that does not physically exist.

11. The method of claim 8 , wherein the code of core functionalities of the to-be-virtualized device comprise a code written specifically for the to-be-virtualized device or a code of an equivalent real-world counterpart of the to-be-virtualized device.

12. The method of claim 8 , wherein:

the hardware component that does not physically exist on the to-be-virtualized device comprises one or more of a gyroscope, a compass, a virtualized electromagnet, a rumble motor, or an exterior motor; and

the hardware emulation API corresponding to the functionality related to the hardware component that does not physically exist on the to-be-virtualized device comprises a movement API.

13. The method of claim 8 , wherein:

the hardware component that does not physically exist on the to-be-virtualized device comprises one or more of a Bluetooth, Wi-Fi, cellular, Near Filed Communications (NFC), Universal Serial Bus (USB), or Global Positioning System (GPS), hardware component; and

the hardware emulation API corresponding to the functionality related to the hardware component that does not physically exist on the to-be-virtualized device comprises a connectivity API.

14. A non-transitory computer-readable medium stored in an augmented reality (AR) or virtual reality (VR) device (AR/VR device), the computer-readable medium storing instructions comprising:

an emulation layer;

one or more hardware emulation application program interfaces (APIs); and

one or more hardware glue logics;

wherein the instructions, when executed by a data processing apparatus, cause the AR/VR device to perform operations comprising:

receiving a code of core functionalities of a to-be-virtualized device, wherein the core functionalities comprises a functionality related to a hardware component that does not physically exist on the to-be-virtualized device;

presenting a virtual representation of the to-be-virtualized device by an AR/VR display; and

emulating the functionality related to the hardware component that does not physically exist on the to-be-virtualized device, wherein the emulating comprises:

receiving, by the emulation layer, a first data of a user interaction with the virtual representation of the to-be-virtualized device, the virtual representation comprising one or more of an image or a video visible to the user, the user interaction with the virtual representation of the to-be-virtualized device comprising one or more of a touch or a gesture of the user relative to the virtual representation of the to-be-virtualized device, the first data comprising an emulated output of the hardware component that does not physically exist on the to-be-virtualized device in response to the user interaction with the virtual representation of the to-be-virtualized device;

passing, via a hardware emulation API corresponding to the functionality related to the hardware component that does not physically exist on the to-be-virtualized device, the emulated output to a hardware glue logic corresponding to the hardware component that does not physically exist on the to-be-virtualized device; and

converting, by the hardware glue logic corresponding to the hardware component that does not physically exist on the to-be-virtualized device, the emulated output into the functionality related to the hardware component that does not physically exist on the to-be-virtualized device.

15. The non-transitory computer-readable medium of claim 14 , wherein to-be-virtualized device comprises a virtual device that does not physically exist.

16. The non-transitory computer-readable medium of claim 14 , wherein the code of core functionalities of the to-be-virtualized device comprise a code written specifically for the to-be-virtualized device or a code of an equivalent real-world counterpart of the to-be-virtualized device.

17. The non-transitory computer-readable medium of claim 14 , wherein:

the hardware component that does not physically exist on the to-be-virtualized device comprises one or more of a gyroscope, a compass, a virtualized electromagnet, a rumble motor, or an exterior motor; and

the hardware emulation API corresponding to the functionality related to the hardware component that does not physically exist on the to-be-virtualized device comprises a movement API.

18. The non-transitory computer-readable medium of claim 14 , wherein:

the hardware component that does not physically exist on the to-be-virtualized device comprises one or more of a Bluetooth, Wi-Fi, cellular, Near Filed Communications (NFC), Universal Serial Bus (USB), or Global Positioning System (GPS), hardware component; and

the hardware emulation API corresponding to the functionality related to the hardware component that does not physically exist on the to-be-virtualized device comprises a connectivity API.

19. The non-transitory computer-readable medium of claim 14 , wherein the operations further comprising pairing with the to-be-virtualized device.

Assignments (4)
NUNC PRO TUNC ASSIGNMENT Recorded Jun 19, 2023
From: BLACKBERRY LIMITED
To: MALIKIE INNOVATIONS LIMITED
Reel/Frame 064271/0199 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 16, 2023
From: BLACKBERRY LIMITED
To: MALIKIE INNOVATIONS LIMITED
Reel/Frame 064104/0103 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 28, 2016
From: BLACKBERRY UK LIMITED
To: BLACKBERRY LIMITED
Reel/Frame 040423/0680 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 27, 2016
From: RISCOMBE-BURTON, PHILLIP
To: BLACKBERRY UK LIMITED
Reel/Frame 039870/0215 →
Continuity (1)
Related Publication 20180053350A1 · Feb 22, 2018
Cited By (1)
US 12,724,277