IP Library Granted Patent US 10,671,161
Granted Patent B2
US 10,671,161 · App. 16/057,931 · Granted Jun 2, 2020

Multi display eye tracking

Inventor: Karthik Krishnakumar (Austin, TX)
Assignee: Dell Products L.P.
G06F3/013G06F3/041G06F3/1431G06T7/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,671,161
App. No.
16/057,931
Granted
Jun 2, 2020
Kind
B2
Abstract

Eye tracking across multiple displays is supported with re-definition of a gaze vector matrix that correlates a surface of a first display with detected eye gazes as inputs to an information handling system to include a surface of a second display. Eye gazes that correlate to the second display are mapped to the redefined gaze vector matrix, such as with a calibration performed by a user or an estimate derived from tracked eye gazes. The information handling system applies a map of display surface areas to the gaze vector matrix to sort reported eye gazes as at a location of the first display surface, a location of the second display surface or discarded as outside of either display surface.

Claims (58)

1. An information handling system comprising:

a processor operable to process information;

a memory interfaced with the processor and operable to store the information;

a graphics controller interfaced with the processor and memory and operable to present the information as visual images at first and second displays;

a first display interfaced with the graphics controller to present visual images;

a second display interfaced with the graphics controller to present visual images;

an eye tracking device coupled to the first display and configured to report eye gaze as a coordinate on the first display from a first set of coordinates defined over a surface of the first display;

an eye tracking module interfaced with the eye tracking device and the second display, the eye tracking module assigning a second set of coordinates to the eye tracking device, the second set of coordinates defined over at least the surface of the first display and extending over at least part of a surface of the second display, the eye tracking module converting gaze coordinates reported from the eye tracking device into position coordinates at one of the first or second displays;

a first configuration with the first set of coordinates mapped to only the first display surface; and

a second configuration with the second set of coordinates mapped to both the first display surface and the second display surface, the second set of coordinates including space between the first and second displays.

2. The information handling system of claim 1 wherein the second set of coordinates include space between the first display surface and the second display surface.

3. The information handling system of claim 1 wherein the gaze vector matrix further comprises a third configuration with the second set of coordinates mapped to both the first display surface aligned in a landscape orientation and the second display surface aligned in a portrait orientation, the second set of coordinates including space above the first display.

4. The information handling system of claim 1 further comprising a multiple display calibrator interfaced with the eye tracking module and operable to present visual markers at known positions on the second display to associate the eye tracker with the second set of coordinates.

5. The information handling system of claim 1 further comprising a multiple display calibrator interfaced with a camera integrated in the second display and operable to apply images captured by the camera to estimate the second set of coordinates.

6. The information handling system of claim 1 further comprising a multiple display calibrator interfaced with the eye tracking module and applying a size, resolution and orientation of each of the first and second displays to determine the second set of coordinates.

7. The information handling system of claim 6 wherein the multiple display calibrator is further operable to:

detect a change in orientation of the first or second display; and

in response to determine a second set of coordinates that includes the surface area of the first and second displays adjusted for the change in orientation.

8. A method for tracking eye gaze at first and second displays, the method comprising:

tracking eye gaze at the first display with a single eye tracking device coupled to the first display;

reporting gaze points from the first display that fall within a gaze vector matrix corresponding with the first display surface;

adjusting the gaze vector matrix to correspond with the first display surface and a second display surface of a second display, the second display surface spaced apart and separate from the first display surface;

reporting gaze points from the single eye tracking device that fall within the adjusted gaze vector; and

applying the gaze points as inputs to the first display that correspond to the first display surface and as inputs to the second display that correspond to the second display surface.

9. The method of claim 8 wherein:

adjusting the gaze vector matrix further comprises defining a perimeter that includes all of the surface of both the first display and the second display; and

applying the gaze points as inputs further comprises, discarding gaze points as inputs that fall outside of the first display surface and the second display surface.

10. The method of claim 8 further comprising:

presenting calibration marks at the second display; and

applying gaze points reported in response to the calibration marks to adjust the gaze vector matrix to include the second display surface.

11. The method of claim 10 further comprising:

detecting a change in orientation of the second display from a landscape orientation to a portrait orientation; and

in response to the detecting, adjusting the gaze vector matrix to define a perimeter that includes all of the surface of the second display in the portrait orientation based upon the gaze points reported in response to the calibration marks in the landscape orientation.

12. The method of claim 8 further comprising:

detecting an image of an end user viewing the first display with a camera integrated in the second display;

detecting a gaze point of the end user at the first display; and

applying the image and the gaze point to estimate a calibration of the second display for the adjusted gaze vector matrix.

13. The method of claim 8 further comprising:

detecting a touch input at the second display;

detecting a gaze point of the end user at the touch input; and

applying the gaze point to estimate a calibration of the second display for the adjusted gaze vector matrix.

14. The method of claim 8 further comprising applying a size, resolution and orientation of each of the first and second displays to determine the adjusted gaze vector matrix.

15. The method of claim 8 wherein adjusting the gaze vector matrix to correspond with the first display surface and a second display surface further comprises:

tracking gaze points outside of the first display surface; and

applying the gaze points outside of the first display surface to estimate the second display surface.

16. A system for tracking eye gaze at plural displays, the system comprising:

an eye tracking device coupled to a first display and configured to report eye gazes at the surface of the first display to an information handling system; and

non-transitory memory integrated in the information handling system storing instructions that when executed on a processing component:

configure the eye tracking device to report eye gazes outside of the surface of the first display including at least a portion of a display surface of a second display disposed spaced and separate from the first display;

map eye gazes reported by the eye tracking device that correspond to the first display as eye gaze inputs to the first display; and

map eye gazes reported by the eye tracking device that correspond to a second display surface area as eye gaze inputs to the second display.

17. The system of claim 16 wherein the instructions further discard eye gazes reported by the eye tracking device that fail to correspond to the first display surface area or the second display surface area.

18. The system of claim 16 wherein the instructions further:

track reported eye gazes outside of the first display surface; and

calibrate the map of eye gazes to the second display surface by analyzing eye gazes outside of the first display surface and input device inputs to the information handling system.

19. The system of claim 16 wherein the instructions further:

present calibration marks at the second display surface; and

calibrate the map of eye gazes to the second display based upon end user gazes at the calibration marks.

Assignments (5)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
SECURITY AGREEMENT Recorded Oct 1, 2021
From: DELL PRODUCTS, L.P.; EMC IP HOLDING COMPANY LLC
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
Reel/Frame 057682/0830 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 8, 2018
From: KRISHNAKUMAR, KARTHIK
To: DELL PRODUCTS L.P.
Reel/Frame 046581/0653 →
Continuity (1)
Related Publication 20200050265A1 · Feb 13, 2020