IP Library Granted Patent US 10,559,133
Granted Patent B2
US 10,559,133 · App. 15/890,800 · Granted Feb 11, 2020

Visual space management across information handling system and augmented reality

Inventors: Michiel Sebastiaan Emanuel Petrus Knoppert (Amsterdam, NL); Deeder M. Aurongzeb (Austin, TX)
Assignee: Dell Products L.P.
G06T19/006G02B27/017G06F3/011G06T19/20G06T2219/028G06T2219/2004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,559,133
App. No.
15/890,800
Granted
Feb 11, 2020
Kind
B2
Abstract

An information handling system coordinates presentation of a visual item with a flat panel display and head mounted display using tokens detected by one or more sensors, such as a depth camera and ultrasonic sensor. Upon detection of a token by a sensor, gestures detected by at least one sensor manipulate a three dimensional image presented at a location by the head mounted display. A visible surface determination of the view area sorts upright extents of the items presented by the head mounted display and flat panel display so the united view of an end user is uncluttered with image overlap.

Claims (60)

1. An information handling system comprising:

a processor operable to execute instructions that process information;

a memory interfaced with the processor and operable to store the instructions and information;

a flat panel display operable to present visual information as visual images;

a head mounted display operable to present visual information as three dimensional visual images focused at a location in front of the head mounted display;

one or more graphics processors interfaced with one or more of the flat panel display and head mounted display and operable to communicate the visual information to the flat panel display and head mounted display formatted for presentation as visual images;

a plurality of sensors disposed proximate the flat panel display and operable to sense end user hand gestures;

a token having one or more features identifiable by one or more of the plural sensors; and

an environment definition engine interfaced with the plural sensors and the one or more graphics processors, the environment definition engine applying gestures detected by the plural sensors to visual images of the flat panel display absent the token identification of the token by the one or more of the plural sensors and applying gestures detected by the plural sensors to the visual images of the head mounted display in the event of identification of the token.

2. The information handling system of claim 1 wherein:

the plural sensors comprise at least a depth camera and an ultrasonic sensor; and

the environment definition engine applies the ultrasonic sensor to manipulate the head mounted display visual images if the depth camera detects the token.

3. The information handling system of claim 2 wherein the token features include a predetermined geometric shape.

4. The information handling system of claim 2 wherein the token features include a predetermined placement on the flat panel display.

5. The information handling system of claim 1 further comprising a mixed reality engine interfaced with the environment definition engine, the mixed reality engine coordinating presentation of flat panel display and head mounted display visual images by sorting upright extents of the visual images in visibility order.

6. The information handling system of claim 5 wherein the token is associated with color selection and the gestures include at least a hand movement associate with color selection.

7. The information handling system of claim 6 wherein the mixed reality engine applies colors selected for presentation at the head mounted display visual image to the flat panel display visual image.

8. The information handling system of claim 7 wherein the token comprises a stylus.

9. A method for presenting visual information at an information handling system, the method comprising:

executing an application on the information handling system to generate the visual information;

presenting the visual information as a visual image at a flat panel display interfaced with the information handling system;

detecting a predetermined object proximate the flat panel display with at least one of plural sensors;

in response to the detecting, presenting a three dimensional visual image of the image at the flat panel display with a head mounted display interfaced with the information handling system, the three dimensional image presented at a location proximate the visual image presented at the flat panel display;

in response to the detecting, associating one or more of the plural sensors to interact with the location of the three dimensional visual image; and

altering the three dimensional visual image in response to hand movements detected at the location by the one or more of the plural sensors.

10. The method of claim 9 further comprising:

initiating the location at the visual image presented on the flat panel display;

detecting a gesture from the location upwards; and

generating the three dimensional visual image at the location and moving the three dimensional visual image upward with the gesture to have a location raised above the flat panel display.

11. The method of claim 10 further comprising:

detecting overlap of the three dimensional visual image and the flat panel display visual image; and

in response to the detecting, altering the visual image at the flat panel display to remove the overlap.

12. The method of claim 9 wherein:

the plural sensors comprise at least a depth camera and an ultrasonic sensor;

detecting a predetermined object comprises detecting a token on the flat panel display with the depth camera; and

associating one or more of the plural sensors to interact with a location of a three dimensional visual image further comprises associating the ultrasonic sensor to detect gestures as inputs at the three dimensional visual image location.

13. The method of claim 9 further comprising:

detecting a second predetermined object associated with color of the three dimensional visual image;

altering the color at the three dimensional visual image in response to the detecting; and

altering the flat panel display visual image to have the color in response to the detecting.

14. The method of claim 13 wherein the second predetermined object comprises a stylus.

15. The method of claim 9 further comprising:

detecting manipulation of the predetermined object; and

in response to the detecting manipulation, associating the one or more of the plural sensors with inputs to the flat panel display visual image.

16. The method of claim 15 further comprising:

detecting overlap of the three dimensional visual image and the flat panel display visual image; and

in response to the detecting, altering the three dimensional visual image to remove the overlap.

17. A method for presenting visual information at an information handling system, the method comprising:

interfacing a flat panel display and a head mounted display with an information handling system;

monitoring an area proximate the flat panel display with plural sensors;

detecting a first predetermined condition with the plural sensors;

in response to the first predetermined condition, rendering an item as a visual image at only the flat panel display;

detecting a second predetermined condition with the plural sensors;

in response to detecting the second predetermined condition, rendering the item as a visual image at both the flat panel display and the head mounted display; and

in response to detecting the second predetermined condition, assigning gestures detected by one of the plural sensors to manipulation of the visual image presented by the head mounted display.

18. The method of claim 17 wherein the plural sensors comprise a depth camera and an ultrasonic sensor, the one of the plural sensors being the ultrasonic sensor, the ultrasonic sensor detecting gestures at a location of the head mounted display presentation of the visual image as a three dimensional visual image.

19. The method of claim 18 comprising:

comparing the relative view of the three dimensional visual image and the flat panel display visual image; and

sorting the upright extents of the three dimensional visual image and the flat panel display visual image to present the three dimensional visual image without overlap of the flat panel display visual image.

20. The method of claim 18 wherein the gesture comprises altering the three dimensional visual image surface color, the method further comprising rendering the flat panel display visual image to have the surface color selected by the gesture.

Assignments (8)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (046366/0014) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC
Reel/Frame 060450/0306 →
RELEASE OF SECURITY INTEREST AT REEL 046286 FRAME 0653 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC
Reel/Frame 058298/0093 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
PATENT SECURITY AGREEMENT (NOTES) Recorded Jun 1, 2018
From: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 046366/0014 →
PATENT SECURITY AGREEMENT (CREDIT) Recorded Jun 1, 2018
From: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 046286/0653 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 7, 2018
From: KNOPPERT, MICHIEL SEBASTIAAN EMANUEL PETRUS; AURONGZEB, DEEDER M.
To: DELL PRODUCTS L.P.
Reel/Frame 044856/0195 →