IP Library Granted Patent US 11,784,906
Granted Patent B2
US 11,784,906 · App. 16/893,327 · Granted Oct 10, 2023

System-latency-aware display device

Inventors: Joohwan Kim (San Jose, CA); Benjamin Boudaoud (Efland, NC); Josef B. Spjut (Durham, NC); Morgan S. McGuire (Waterloo, CA); Seth P. Schneider (San Jose, CA); Rouslan L. Dimitrov (Santa Clara, CA); Lars Nordskog (Corte Madera, CA); Cody J. Robson (San Jose, CA); Sau Yan Keith Li (San Jose, CA); Gerrit Ary Slavenburg (Hayward, CA); Tom J. Verbeure (Sunnyvale, CA)
Assignee: NVIDIA Corporation
H04L43/106G06F3/14G06T1/20G09G5/14H04L43/0852G09G2340/0407
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,784,906
App. No.
16/893,327
Granted
Oct 10, 2023
Kind
B2
Abstract

A display device for measuring the end-to-end latency of a computing system. The computing system includes an input device, a computing device, and the display device. The display device is directly connected with the input device and receives input data packets generated by the input device in response to received user input events. The display device passes the input packets to the computing device for graphics processing. The display device measures the end-to-end latency comprising the sum of three latencies. A first latency comprises an input delay of the input device. A second latency comprises an amount of time between generation of the input packet and a corresponding change in pixel values caused by the input event at the display device. A third latency comprises a display latency. The display device also displays latency information associated with the measured end-to-end latency.

Claims (37)

1. A computer-implemented method for determining a system latency, the method comprising:

detecting, by a display device, an input packet received from a user input device coupled to the display device that has been generated by the user input device in response to an input event received by the user input device;

generating, by the display device, a first timestamp in response to detecting the input packet;

detecting, by the display device, a change in one or more pixel values received from a computing device that is separate from the display device, wherein the one or more pixel values are associated with one or more target pixel locations within a display screen of the display device, and the change is caused by the input event;

generating, by the display device, a second timestamp in response to detecting the change in the one or more pixel values; and

determining, by the display device, the system latency based on the first timestamp and the second timestamp.

2. The computer-implemented method of claim 1 , further comprising displaying information associated with the system latency to a user via a user interface.

3. The computer-implemented method of claim 1 , wherein detecting the change in the one or more pixel values comprises detecting that the one or more pixel values have changed by at least a threshold amount.

4. The computer-implemented method of claim 1 , further comprising determining an input latency associated with the user input device, wherein the system latency is further based on the input latency.

5. The computer-implemented method of claim 4 , wherein the input latency comprises an amount of time between when the input event occurs and when the input packet is generated in response to the input event.

6. The computer-implemented method of claim 4 , wherein the input latency comprises an estimated input latency associated with the user input device.

7. The computer-implemented method of claim 1 , further comprising determining a display latency associated with the display device, wherein the system latency is further based on the display latency.

8. The computer-implemented method of claim 7 , wherein the display latency comprises an amount of time between when the change in the one or more pixel values of the one or more target pixel locations is detected and when a corresponding photon event occurs on the display screen of the display device.

9. The computer-implemented method of claim 7 , wherein the display latency comprises an estimated display latency based on a screen resolution and a refresh rate of the display device.

10. The computer-implemented method of claim 1 , wherein the system latency is further based on an input latency and a display latency.

11. One or more non-transitory computer-readable media including instructions that, when executed by one or more processors, cause the one or more processors to determine a system latency by performing the steps of:

detecting, by a display device, an input packet received from a user input device coupled to the display device that has been generated by the user input device in response to an input event received by the user input device;

generating, by the display device, a first timestamp in response to detecting the input packet;

detecting, by the display device, a change in one or more pixel values received from a computing device that is separate from the display device, wherein the one or more pixel values are associated with one or more target pixel locations within a display screen of the display device, and the change is caused by the input event;

generating, by the display device, a second timestamp in response to detecting the change in the one or more pixel values; and

determining, by the display device, the system latency based on the first timestamp and the second timestamp.

12. The one or more non-transitory computer-readable media of claim 11 , further comprising displaying information associated with the system latency to a user via a user interface.

13. The one or more non-transitory computer-readable media of claim 11 , wherein detecting the change in the one or more pixel values comprises detecting that the one or more pixel values have changed by at least a threshold amount.

14. The one or more non-transitory computer-readable media of claim 11 , further comprising determining an input latency associated with the user input device, wherein the system latency is further based on the input latency.

15. The one or more non-transitory computer-readable media of claim 11 , further comprising determining a display latency associated with the display device, wherein the system latency is further based on the display latency.

16. A display device, comprising:

a display screen; and

a latency engine that determines a system latency by performing the steps of:

detecting, by a display device, an input packet received from a user input device coupled to the display device that has been generated by the user input device in response to an input event received by the user input device;

generating, by the display device, a first timestamp in response to detecting the input packet;

detecting, by the display device, a change in one or more pixel values received from a computing device that is separate from the display device, wherein the one or more pixel values are associated with one or more target pixel locations within a display screen of the display device, and the change is caused by the input event;

generating, by the display device, a second timestamp in response to detecting the change in the one or more pixel values; and

determining, by the display device, the system latency based on the first timestamp and the second timestamp.

17. The display device of claim 16 , wherein the latency engine performs the additional steps of determining an input latency associated with the user input device, wherein the user input device is directly connected to the display device.

18. The display device of claim 17 , wherein the input latency comprises an amount of time between when the input event occurs and when the input packet is generated in response to the input event.

19. The display device of claim 16 , wherein the latency engine performs the additional steps of determining a display latency associated with the display device, wherein the system latency is further based on the display latency.

20. The display device of claim 19 , wherein the display latency comprises an estimated display latency based on a screen resolution and a refresh rate of the display device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 18, 2020
From: KIM, JOOHWAN; BOUDAOUD, BENJAMIN; SPJUT, JOSEF B.; MCGUIRE, MORGAN S.; SCHNEIDER, SETH P.; DIMITROV, ROUSLAN L.; NORDSKOG, LARS; ROBSON, CODY J.; LI, SAU YAN KEITH; SLAVENBURG, GERRIT ARY; VERBEURE, TOM J.
To: NVIDIA CORPORATION
Reel/Frame 053536/0025 →
Continuity (3)
Provisional Application 63029343 · May 22, 2020
Provisional Application 62970096 · Feb 4, 2020
Related Publication 20210243101A1 · Aug 5, 2021