IP Library Granted Patent US 9,690,465
Granted Patent B2
US 9,690,465 · App. 13/723,365 · Granted Jun 27, 2017

Control of remote applications using companion device

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,690,465
App. No.
13/723,365
Granted
Jun 27, 2017
Kind
B2
Abstract

Embodiments of the invention provide underlying communication functionality to enable companion experiences. A companion experience allows the user to interact with content playing on a primary device through a companion device. An application on the companion device interacts with an application running on a base device (e.g., a game console, PC, or TV) to provide additional interface options on the companion that are related to a title or application playing on the base device.

Claims (39)

1. A method of using a companion device to control an application running on a primary device, the method comprising:

outputting for display to a screen on a display device a graphical user interface generated by an application running on the primary device;

receiving from the companion device, which has a touch screen display, a normalized touch data that describes a touch event comprising an object contacting the touch screen at a plurality of touch points on the touch screen display, the normalized touch data comprising a normalized x coordinate and a normalized y coordinate for each of the touch points, wherein the normalized x coordinate is calculated using the formula x n =x/w where x n is the normalized x coordinate, x is an x coordinate for a touch point, and w is the width of the touch screen display in pixels; wherein the normalized y coordinate is calculated using the formula y n =y/h where y n is the normalized y coordinate, y is an y coordinate for the touch point, and h is the height of the touch screen display in pixels, wherein the touch event is movement across the touch screen;

calculating a distance an object on the graphical user interface is to be moved using the normalized touch data as input and applying a dampening logic that adjusts a delta calculated between touch points using both a dampening coefficient and an acceleration component; and

outputting for display to the display device an updated graphical user interface generated by the application showing the object moved the distance.

2. The method of claim 1 , wherein the method further comprises:

rescaling the normalized touch data by multiplying the normalized x coordinate by a width of the updated graphical user interface and the normalized y coordinate by the height of the updated graphical user interface to generate scaled touch data; and

interpreting the scaled touch data to determine the distance derived from the normalized touch data.

3. The method of claim 1 , wherein the method further comprises:

rescaling the normalized touch data by multiplying the normalized x coordinate by a width of the updated graphical user interface and the normalized y coordinate by the height of the updated graphical user interface to generate scaled touch data;

generating an optimized touch data by multiplying the scaled touch data by a factor that is a ratio of the companion device's screen size to a template screen size, and

interpreting the optimized touch data to determine the distance derived from the normalized touch data.

4. The method of claim 1 , wherein the object is a cursor.

5. The method of claim 1 , wherein a scroll function is activated when the touch event indicates that two fingers are contacting the touch screen display and remain a threshold distance apart as they move.

6. The method of claim 1 , wherein the method further comprises receiving movement data from the companion device.

7. One or more computer-storage media comprising computer-executable instructions embodied thereon that when executed by a computing device performs a method for using a companion device to manipulate a graphical user interface generated by an application operating on a primary device, the method comprising:

outputting for display on a screen of a display device a graphical user interface (“GUI”) generated by the application running on the primary device, the GUI comprising a cursor having a present location;

receiving control information from the companion device, which has a touch screen display, the control information comprising a normalized touch data that describes a touch event comprising an object contacting the touch screen at a plurality of touch points on the touch screen display, the normalized touch data comprising a normalized x coordinate and a normalized y coordinate for each of the touch points, wherein the normalized x coordinate is calculated using the formula x n =x/w where x n is the normalized x coordinate, x is an x coordinate for a touch point, and w is the width of the touch screen display in pixels; wherein the normalized y coordinate is calculated using the formula y n =y/h where y n is the normalized y coordinate, y is an y coordinate for the touch point, and h is the height of the touch screen display in pixels, wherein the touch event is movement across the touch screen;

calculating a distance an object on the graphical user interface is to be moved using the normalized touch data as input and applying a dampening logic that adjusts a delta calculated between touch points using both a dampening coefficient and an acceleration component;

and

outputting for display an updated GUI showing the object moved the distance.

8. The media of claim 7 , wherein the GUI height is less than a height of the display device.

9. The media of claim 7 , wherein the movement data indicates the companion device is tilted up from the home position and the direction of scrolling is up.

10. The media of claim 7 , wherein the movement data indicates the companion device is twisted rightward from the home position and the direction of scrolling is to the right.

11. The media of claim 7 , wherein the home position is a position of the companion device when a scroll function is activated in response to the movement data.

12. The media of claim 7 , wherein the normalized touch data describes a finger moving down a side of the touch screen display.

13. A computing system to implement a method for using a companion device to manipulate a graphical user interface generated by an application operating on a primary device, the computing system comprising:

a processor; and

computer storage media having computer-executable instructions stored thereon which, when executed by the processor, configure the computing system to:

activate, at the companion device, a navigation function that enables touch input on a touch screen display that is integrated in the companion device to manipulate the application running on the primary device that is separate from the companion device and has a separate;

receive, from the touch screen display, touch data describing points contacted during a touch event comprising an object contacting the touch screen at a plurality of touch points;

convert the touch data to a normalized touch data; the normalized touch data comprising a normalized x coordinate and a normalized y coordinate for each of the touch points, wherein the normalized x coordinate is calculated using the formula x n =x/w where x n is the normalized x coordinate, x is an x coordinate for a touch point, and w is the width of the touch screen display in pixels; wherein the normalized y coordinate is calculated using the formula y n =y/h where y n is the normalized y coordinate, y is an y coordinate for the touch point, and h is the height of the touch screen display in pixels;

dynamically adjust a frame rate at which the normalized touch data is communicated to the primary device to match a frame rate of the primary device based on network latency characteristics; and

communicate the normalized touch data to the primary device that is associated with a display device having a primary screen.

14. The computing system of claim 13 , wherein the computing system is further configured to, subsequent to the touch event, detect an orientation change of the companion device and communicating, prior to a subsequent touch event, an updated screen area for the touch screen display to the primary device.

15. The computing system of claim 13 , wherein the computing system is further configured to receive a message from the application running on the primary device that a web page is loading and, in response, enable a stop button displayed on the touch screen display, the stop button associated with a function that instructs the application to stop loading the web page.

16. The computing system of claim 13 , wherein the computing system is further configured to:

receive a message from the application running on the primary device that the web page has just finished loading and, in response, enable a refresh button displayed on the touch screen display, the refresh button associated with a function that instructs the application to reload the web page; and

disable the stop button.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2014
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 034544/0541 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 16, 2013
From: TUCK, JASON R.; MEDAK, MATTHEW ALAN; HADDAD, MARC V.; TUDOR, ALEXANDER D.
To: MICROSOFT CORPORATION
Reel/Frame 030223/0148 →