IP Library Granted Patent US 10,890,992
Granted Patent B2
US 10,890,992 · App. 16/353,428 · Granted Jan 12, 2021

Synchronizing augmented or virtual reality (AR/VR) applications with companion device interfaces

Inventors: Joshua Reino Timonen (Portland, OR); Andrew Chalkley (Milwaukie, OR)
Assignee: eBay Inc.
G06F3/038H04L67/06H04W76/14G06F2203/0384
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,890,992
App. No.
16/353,428
Granted
Jan 12, 2021
Kind
B2
Abstract

An augmented reality or virtual reality (AR/VR) device pairs with a companion device to augment input interfaces associated with an AR/VR application at the AR/VR device. In implementations, an AR/VR device determines a portion of a markup file that corresponds to an AR/VR scene of a plurality of AR/VR scenes in an AR/VR environment, and communicates the portion of the markup file to the companion device to cause the companion device to configure a companion user interface associated with initiating an action as part of the AR/VR scene. In response to receiving user input via the companion user interface, the companion device communicates the action to the AR/VR device to initiate the action. The AR/VR device receives input data from the companion device, and initiates the action for the AR/VR scene.

Claims (45)

1. A service provider system comprising:

an augmented or virtual reality (AR/VR) hub system implemented using a processing system and computer-readable storage medium of a computing device, the AR/VR hub system including:

a request manger module to receive a request via a network from a companion device for companion data that corresponds to an augmented or virtual reality (AR/VR) application configured for execution by an AR/VR device separate from the companion device;

a companion markup location module to locate, from storage, the companion data from a plurality of companion data;

a data transmission module to form a communication for transmission via the network to the companion device that includes the companion data; and

an AR/VR environment manager module to synchronize execution of the companion data by the companion device with execution of the AR/VR application by the AR/VR device, the synchronization including:

determining a portion of a markup file that corresponds to an AR/VR scene of a plurality of AR/VR scenes of the AR/VR application that is rendered by the AR/VR device;

communicating the portion of the markup file to the companion device as the companion data;

detecting a change from the AR/VR scene to another AR/VR scene of the plurality of AR/VR scenes; and

communicating another portion of the markup file to the companion device that corresponds to the other AR/VR scene.

2. The service provider system as described in claim 1 , further comprising a request verification module to verify that an originator of the request is to receive the companion data and the data transmission module is configured to form the communication responsive to successful verification by the request verification module.

3. The service provider system as described in claim 1 , further comprising a pairing data manager module configured to provide pairing data corresponding to the AR/VR application to the companion device for pairing with the AR/VR device.

4. The service provider system as described in claim 3 , wherein the pairing data includes a QR code or a verification code.

5. The service provider system as described in claim 1 , wherein the AR/VR environment manager module is configured to synchronize the execution by communicating input data received from the companion device to initiate an action of the AR/VR device.

6. The service provider system as described in claim 5 , wherein the input data is generated based on user interaction with a companion user interface at the companion device, the companion user interface generated based on the companion markup data.

7. The service provider system as described in claim 1 , wherein the companion data is configured for execution within a browser by the companion device.

8. The service provider system as described in claim 1 , wherein the AR/VR environment manager module is configured to receive protected data from the companion device without exposing the protected data to the AR/VR device.

9. The service provider system as described in claim 8 , wherein the protected data is payment data.

10. The service provider system as described in claim 8 , wherein the protected data is user login information that is used to login the companion device as part of an AR/VR environment managed by the AR/VR environment manager module and then permits a login of the AR/VR device to the AR/VR environment.

11. A method implemented by at least one computing device, the method comprising:

receiving, by the at least one computing device, inputs as part of creating an augmented or virtual reality (AR/VR) application executable by an augmented reality (AR) device, the AR/VR application having a plurality of AR/VR scenes and corresponding actions as part of user interaction with respective AR/VR scenes of the plurality of AR/VR scenes;

generating, by the at least one computing device, a markup file specifying linkages between controls usable as part of user interaction with a companion device and the corresponding actions within the respective AR/VR scenes of the AR/VR application as executed by the AR/VR device;

storing, by the at least one computing device, the markup file as associated with the AR/VR application, the markup file configured to support communication of respective linkages to the companion device between the controls as executed by the companion device and corresponding actions for the respective AR/VR scenes output by the AR/VR device;

determining, by the at least one computing device, a portion of the markup file that corresponds to an AR/VR scene of the plurality of AR/VR scenes of the AR/VR application that is rendered by the AR/VR device;

communicating, by the at least one computing device, the portion of the markup file to the companion device as companion data;

detecting, by the at least one computing device, a change from the AR/VR scene to another AR/VR scene of the plurality of AR/VR scenes; and

communicating, by the at least one computing device, another portion of the markup file to the companion device that corresponds to the other AR/VR scene.

12. The method as described in claim 11 , wherein the markup file indicates the corresponding actions.

13. The method as described in claim 11 , wherein the markup file indicates the controls.

14. The method as described in claim 11 , wherein the markup file is configured for execution within a browser of the companion device.

15. The method as described in claim 11 , further comprising receiving, by the at least one computing device, protected data from the companion device without exposing the protected data to the AR/VR device.

16. A method as implemented by an augmented or virtual reality (AR/VR) device, the method comprising:

determining, by the AR/VR device, a portion of a markup file that corresponds to an AR/VR scene of a plurality of AR/VR scenes;

communicating, by the AR/VR device, the portion of the markup file to a companion device to cause the companion device to configure a companion user interface to initiate an action as part of the AR/VR scene;

receiving, by the AR/VR device, input data from the companion device to initiate the action, the input data generated based on user interaction with the companion user interface at the companion device;

initiating, by the AR/VR device, the action for the AR/VR scene;

detecting, by the AR/VR device, a change from the AR/VR scene to another AR/VR scene of the plurality of AR/VR scenes; and

communicating, by the AR/VR device, another portion of the markup file to the companion device that corresponds to the other AR/VR scene, the other portion configured to cause the companion device to configure the companion user interface to initiate another action as part of the other AR/VR scene.

17. The method as described in claim 16 , wherein the portion of the markup file identifies a control to be used to initiate the action in the companion user interface.

18. The method as described in claim 16 , wherein the portion of the markup file identifies the action that is used as a basis by the companion device to select a control as part of the companion user interface to initiate the action.

19. The method as described in claim 16 , further comprising:

capturing, by a digital camera of the AR/VR device, a digital image of a display device of the companion device, the digital image including pairing data obtained from a network address via a network from a service provider system by the companion device;

verifying, by the AR/VR device, the pairing data for pairing the companion device with the AR/VR device; and

responsive to the verifying, pairing the AR/VR device as communicatively coupled with the companion device.

20. The method as described in claim 16 , wherein the markup file is included as part of an AR/VR application that is executed to implement the plurality of AR/VR scenes as part of an AR/VR environment.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 14, 2019
From: TIMONEN, JOSHUA REINO; CHALKLEY, ANDREW
To: EBAY INC.
Reel/Frame 048600/0226 →
Continuity (1)
Related Publication 20200293120A1 · Sep 17, 2020
Cited By (3)
US 12,314,496 US 12,348,697 US 12,664,696