IP Library › Granted Patent US 11,093,201
Granted Patent B2
US 11,093,201 · App. 16/599,383 · Granted Aug 17, 2021

Device manager that utilizes physical position of display devices

Inventors: Alexander James Faaborg (Mountain View, CA); Ariel Sachter-Zeltzer (Sunnlyvale, CA)
Assignee: GOOGLE LLC
G06F3/1423G06F3/02G06F3/033
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,093,201
App. No.
16/599,383
Granted
Aug 17, 2021
Kind
B2
Abstract

A system and method may provide for a computing system including multiple display devices. Data collected by sensors such as, for example, image sensors and positional sensors, of the display devices is leveraged to determine a position and/or an orientation of each of the display devices in the real world environment. This identification and utilization of the relative positioning of the display devices may be used to configure, and reconfigure, display settings, set display priorities, control movement of content, and respond to gesture based commands related to content management.

Claims (86)

1. A method, comprising:

detecting, by a processor of a computing device, a plurality of display devices in a physical environment, the plurality of display devices being operably coupled to the computing device in the physical environment;

receiving, by the processor from a plurality of sensors of the plurality of display devices, data related to the physical environment, including:

image data related to the physical environment from at least one image sensor of the plurality of display devices; and

orientation data related to the plurality of display devices from at least one position sensor of the plurality of display devices;

detecting, by the processor, a physical position of the plurality of display devices in the physical environment based on the data received from the plurality of sensors of the plurality of display devices; and

controlling, by the processor, at least one of:

content displayed on at least two of the plurality of display devices based on the detected physical positions of the plurality of display devices, or

an interaction of an auxiliary device with the plurality of display devices.

2. The method of claim 1 , wherein controlling the content displayed on at least two of the plurality of display devices includes:

detecting a user input; and

controlling the display of the content in response to the user input.

3. The method of claim 2 , wherein

detecting the user input includes detecting a controller input that moves the displayed content from a first display device to a second display device of the plurality of display devices; and

controlling the display of the content includes moving the displayed content from a display position on the first display device to a display position on the second display device of the plurality of display devices.

4. The method of claim 3 , wherein moving the displayed content from the first display device to the second display device includes:

adjusting a position and an orientation of the displayed content, from a first position and a first orientation on the first display device to a second position and a second orientation on the second display device, in response to the detected physical position of the first display device and the detected physical position of the second display device.

5. The method of claim 4 , wherein the physical orientation of the first display device is different from the physical orientation of the second display device.

6. The method of claim 4 , wherein the first display device is oriented substantially horizontally, and the second display device is oriented substantially vertically.

7. The method of claim 3 , wherein moving the displayed content from the first display device to the second display device includes:

detecting a physical discontinuity between the first display device and the second display device, based on the detected physical position of the first display device and the detected physical position of the second display device in the physical environment; and

adjusting the moving of the displayed content from the first display device to the second display device based on the detected physical discontinuity.

8. The method of claim 7 , wherein detecting the physical discontinuity includes at least one of:

detecting a physical gap between the detected physical position of the first display device and the detected physical position of the second display device; or

detecting a physical bezel area surrounding a display area of at least one of the first display device or the second display device.

9. The method of claim 7 , wherein adjusting the movement of the displayed content includes delaying the display of the content on the second display device to correspond to movement of the displayed content through the physical discontinuity.

10. The method of claim 1 , wherein detecting the physical position of the plurality of display devices in the physical environment includes:

receiving the orientation data from the position sensor of at least one of the plurality of display devices;

detecting a physical orientation of the at least one display device based on the received orientation data; and

detecting the physical position of the at least one display device relative to remaining display devices of the plurality of display devices based on the physical orientation and the image data related to the physical environment received from image sensors of the plurality of display devices.

11. The method of claim 1 , further comprising:

dynamically configuring display settings of the computing device based on the detected physical positions of the plurality of display devices,

wherein the content displayed on the at least two of the plurality of display devices is controlled based on the dynamically configured display settings.

12. The method of claim 11 , further comprising;

collecting movement data from a position sensor of at least one of the plurality of display devices;

detecting a movement of the at least one display device based on the movement data; and

dynamically configuring the display settings in response to detecting the movement of the at least one display device of the plurality of display devices.

13. The method of claim 12 , wherein the at least one display device of the plurality of display devices is a mobile display device.

14. The method of claim 11 , wherein dynamically configuring the display settings of the computing device includes at least one of dynamically setting an order of the plurality of display devices, dynamically setting a primary display device of the plurality of display devices, or dynamically setting relative positions of the plurality of the display devices.

15. The method of claim 14 , wherein detecting the physical position of each display device of the plurality of display devices includes:

intermittently receiving the image data from the image sensors of the plurality of display devices and orientation data from the position sensors of the plurality of display devices;

dynamically updating the physical position of the plurality of display devices based on updated image data and updated orientation data; and

dynamically updating the display settings of the computing device based on the updated physical position of the plurality of display devices.

16. The method of claim 1 , wherein controlling interaction of the auxiliary device with the plurality of display devices includes controlling interaction of a display auxiliary device or a non-display auxiliary device with the plurality of display devices.

17. The method of claim 16 , wherein the display auxiliary device includes at least one of a tablet computing device or a wearable display device.

18. The method of claim 17 , wherein the wearable display device includes at least one of an augmented reality head mounted display device or a wrist worn computing device.

19. The method of claim 17 , wherein the non-display auxiliary device includes at least one of a keyboard input device, a mouse input device, or a touchpad input device.

20. A method, comprising:

detecting, by a processor of a computing device, a plurality of display devices in a physical environment, the plurality of display devices being operably coupled to the computing device in the physical environment;

receiving, by the processor, data related to the physical environment from sensors of the plurality of display devices, including:

receiving image data related to the physical environment from image sensors of the plurality of display devices; and

receiving orientation data related to the plurality of display devices from position sensors of the plurality of display devices;

detecting, by the processor, a physical position of each display device of the plurality of display devices in the physical environment based on the data related to the physical environment; and

configuring display settings of the computing device without user intervention based on the detected physical positions of the plurality of display devices.

21. A computer program product tangibly embodied on a non-transitory computer-readable storage medium, comprising instructions that, when executed by a computing device, are configured to cause the computing device to:

detect a plurality of display devices in a physical environment;

receive data from the plurality of display devices, collected by a plurality of sensors of the plurality of display devices, including:

receive image data related to the physical environment from at least one image sensor of the plurality of display devices; and

receive orientation data related to the plurality of display devices from at least one position sensor of the plurality of display devices;

detect a physical position of each display device of the plurality of display devices in the physical environment based on data related to the physical environment, based on the data collected by the plurality of sensors of the plurality of display devices; and

control at least one of:

content displayed on at least two of the plurality of display devices based on the detected physical positions of the plurality of display devices, or

an interaction of an auxiliary device with the plurality of display devices.

22. The computer program product of claim 21 , wherein, in controlling the content displayed on the plurality of display devices, the instructions cause the computing device to:

detect a user input, including a controller input that moves the displayed content from a first display device to a second display device of the plurality of display devices; and

move displayed content from a display position on the first display device to a display position on the second display device of the plurality of display devices.

23. The computer program product of claim 22 , wherein, in moving the displayed content from the first display device to the second display device, the instructions cause the computing device to:

detect that a physical orientation of the first display device is different from a physical orientation of the second display device; and

adjust a position and an orientation of the displayed content, from a first position and a first orientation on the first display device to a second position and a second orientation on the second display device, in response to the detected physical orientation of the first display device and the detected physical orientation second display device.

24. The computer program product of claim 22 , wherein, in moving the displayed content from the first display device to the second display device, the instructions cause the computing device to:

detect a physical discontinuity between the first display device and the second display device, the detected physical discontinuity including at least one of:

a physical gap between the detected physical position of the first display device and the detected physical position of the second display device; or

a physical bezel area surrounding a display area of at least one of the first display device or the second display device; and

adjust the moving of the displayed content from the first display device to the second display device to correspond to movement of the displayed content through the detected physical discontinuity.

25. The computer program product of claim 21 , wherein, in detecting the physical position of the plurality of display devices in the physical environment, the instructions cause the computing device to, for each display device of the plurality of display devices:

receive orientation data from the position sensor of the display device;

detect a physical orientation of the display device based on the orientation data; and

detect a physical position of the display device relative to remaining display devices of the plurality of display devices based on the physical orientation and the image data related to the physical environment collected by the image sensors.

26. The computer program product of claim 25 , wherein the instructions also cause the computing device to:

receive movement data from a position sensor of at least one of the plurality of display devices;

detect a movement of the at least one display device based on the movement data; and

dynamically update display settings of the computing device in in response to the detected movement of the at least one display device of the plurality of display devices.

27. The computer program product of claim 26 , wherein, in detecting the physical position of each display device of the plurality of display devices, the instructions cause the computing device to:

periodically receive image data and orientation data from the plurality of sensors of the plurality of display devices;

dynamically update the physical position of the plurality of display devices based on updated image data and updated orientation data; and

dynamically update the display settings of the computing device based on the updated physical position of the plurality of display devices.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 22, 2019
From: FAABORG, ALEXANDER JAMES; SACHTER-ZELTZER, ARIEL
To: GOOGLE LLC
Reel/Frame 050783/0028 →
Continuity (2)
Provisional Application 62906586 · Sep 26, 2019
Related Publication 20210096802A1 · Apr 1, 2021