IP Library Granted Patent US 12,229,871
Granted Patent B2
US 12,229,871 · App. 17/481,656 · Granted Feb 18, 2025

Motion biased foveated renderer

Inventors: Prasoonkumar Surti (Folsom, CA); Karthik Vaidyanathan (San Fancisco, CA); Atsuo Kuwahara (Portland, OR); Hugues Labbe (Granite Bay, CA); Sameer K P (Bangalore, IN); Jonathan Kennedy (Bristol, GB); Joydeep Ray (Folsom, CA); Travis T. Schluessler (Berthoud, CO); John H. Feit (Folsom, CA); Nikos Kaburlasos (Folsom, CA); Jacek Kwiatkowski (Santa Clara, CA); Tomer Bar-On (Petah Tikva, IL); Carsten Benthin (Voelklingen, DE); Adam T. Lake (Portland, OR); Vasanth Ranganathan (El Dorado Hills, CA); Abhishek R. Appu (El Dorado Hills, CA)
Assignee: Intel Corporation
G06T15/08G02B27/017G06T15/005G06T15/10G06T15/60G06V20/49G06V40/19H04N13/239H04N13/344H04N23/67H04N25/702G06T2200/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,229,871
App. No.
17/481,656
Granted
Feb 18, 2025
Kind
B2
Abstract

An embodiment of an electronic processing system may include an application processor, persistent storage media communicatively coupled to the application processor, a graphics subsystem communicatively coupled to the application processor, a sense engine communicatively coupled to the graphics subsystem to provide sensed information, a focus engine communicatively coupled to the sense engine and the graphics subsystem to provide focus information, a motion engine communicatively coupled to the sense engine, the focus engine, and the graphics subsystem to provide motion information, and a motion biased foveated renderer communicatively coupled to the motion engine, the focus engine, the sense engine to adjust one or more parameters of the graphics subsystem based on one or more of the sense information, the focus information, and the motion information. Other embodiments are disclosed and claimed.

Claims (45)

1. At least one non-transitory computer readable storage medium comprising a set of instructions, execution of which by a graphics apparatus causes the graphics apparatus to:

associate a first image portion with a focus area of an image based on eye-tracking information;

cause the first image portion to be rendered at a first rate;

cause a second image portion to be rendered at a second rate, wherein the second image portion is a region outside of the first image portion, and wherein the second rate is different from the first rate;

associate a third image portion with a relocation of the focus area of the image based on a change in the eye-tracking information; and

cause an adjustment to a size of the focus area, wherein the size of the focus area is to be adjusted based on at least one of a workload amount for the graphics apparatus or scene content.

2. The at least one non-transitory computer readable storage medium of claim 1 , wherein before adjustment the focus area is a first shape, and after adjustment the focus area is a second shape, and wherein the first shape is one of a circular shape or an elliptical shape.

3. The at least one non-transitory computer readable storage medium of claim 2 , wherein the second shape is an elliptical shape.

4. The at least one non-transitory computer readable storage medium of claim 1 , wherein the first rate is higher than the second rate.

5. The at least one non-transitory computer readable storage medium of claim 1 , comprising further instructions, execution of which by the graphics apparatus causes the graphics apparatus to cause the third image portion to be rendered at the first rate.

6. The at least one non-transitory computer readable storage medium of claim 1 , wherein the adjustment to the size of the focus area results in an increase in the size of the focus area.

7. A graphics apparatus comprising:

a cache memory; and

logic communicatively coupled to the cache memory, the logic implemented at least partly in one or more of configurable hardware logic or fixed-functionality hardware logic, the logic to:

associate a first image portion with a focus area of an image based on eye-tracking information;

cause the first image portion to be rendered at a first rate;

cause a second image portion to be rendered at a second rate, wherein the second image portion is a region outside of the first image portion, and wherein the second rate is different from the first rate;

associate a third image portion with a relocation of the focus area of the image based on a change in the eye-tracking information; and

cause an adjustment to a size of the focus area, wherein the size of the focus area is to be adjusted based on at least one of a workload amount for the graphics apparatus or scene content.

8. The graphics apparatus of claim 7 , wherein before adjustment the focus area is a first shape, and after adjustment the focus area is a second shape, and wherein the first shape is one of a circular shape or an elliptical shape.

9. The graphics apparatus of claim 8 , wherein the second shape is an elliptical shape.

10. The graphics apparatus of claim 7 , wherein the first rate is higher than the second rate.

11. The graphics apparatus of claim 7 , wherein the logic is to cause the third image portion to be rendered at the first rate.

12. The graphics apparatus of claim 7 , wherein the adjustment to the size of the focus area results in an increase in the size of the focus area.

13. A method comprising:

associating a first image portion with a focus area of an image based on eye-tracking information;

rendering the first image portion at a first rate;

rendering a second image portion at a second rate, wherein the second image portion is a region outside of the first image portion, and wherein the second rate is different from the first rate;

associating a third image portion with a relocation of the focus area of the image based on a change in the eye-tracking information; and

adjusting a size of the focus area, wherein the size of the focus area is adjusted based on at least one of a workload amount for the graphics apparatus or scene content.

14. The method of claim 13 , wherein before adjusting the size of the focus area, the focus area is a first shape, and after adjusting the size of the focus area, the focus area is a second shape, and wherein the first shape is one of a circular shape or an elliptical shape.

15. The method of claim 14 , wherein the second shape is an elliptical shape.

16. The method of claim 13 , wherein the first rate is higher than the second rate.

17. The method of claim 13 , further comprising rendering the third image portion at the first rate.

18. The method of claim 13 , wherein adjusting the size of the focus area results in an increase in the size of the focus area.

19. A graphics apparatus comprising:

means for associating a first image portion with a focus area of an image based on eye-tracking information;

means for rendering the first image portion at a first rate;

means for rendering a second image portion at a second rate, wherein the second image portion is a region outside of the first image portion, and wherein the second rate is different from the first rate;

means for associating a third image portion with a relocation of the focus area of the image based on a change in the eye-tracking information; and

means for adjusting a size of the focus area, wherein the size of the focus area is adjusted based on at least one of a workload amount for the graphics apparatus or scene content.

20. The graphics apparatus of claim 19 , wherein before adjusting the size of the focus area, the focus area is a first shape, and after adjusting the size of the focus area, the focus area is a second shape, wherein the first shape is one of a circular shape or an elliptical shape, and wherein the second shape is an elliptical shape.

21. The graphics apparatus of claim 19 , wherein the first rate is higher than the second rate.

22. The graphics apparatus of claim 19 , further comprising means for rendering the third image portion at the first rate.

23. The graphics apparatus of claim 19 , wherein adjusting the size of the focus area results in an increase in the size of the focus area.

Continuity (4)
Continuation 16662636 · Oct 24, 2019
Continuation 16173722 · Oct 29, 2018
Continuation 15477019 · Apr 1, 2017
Related Publication 20220005259A1 · Jan 6, 2022
References Cited (138)
US 5610658A · Uchida et al. · 1997 [cited by applicant]
US 6760024B1 · Lokovic et al. · 2004 [cited by applicant]
US 6798443B1 · Maguire, Jr. · 2004 [cited by applicant]
US 7020207B1 · Tavares et al. · 2006 [cited by applicant]
US 7742623B1 · Moon et al. · 2010 [cited by applicant]
US 9129443B2 · Gruen et al. · 2015 [cited by applicant]
US 9165399B2 · Uralsky et al. · 2015 [cited by applicant]
US 9177413B2 · Tatarinov et al. · 2015 [cited by applicant]
US 9241146B2 · Neill · 2016 [cited by applicant]
US 9262797B2 · Minkin et al. · 2016 [cited by applicant]
US 9292753B2 · Ranjan · 2016 [cited by examiner]
US 9342857B2 · Kubisch et al. · 2016 [cited by applicant]
US 9355483B2 · Lum et al. · 2016 [cited by applicant]
US 9437040B2 · Lum et al. · 2016 [cited by applicant]
US 9829710B1 · Newell et al. · 2017 [cited by applicant]
US 9916090B1 · Can · 2018 [cited by examiner]
US 10387741B2 · Zink et al. · 2019 [cited by applicant]
US 10585472B2 · Tokubo et al. · 2020 [cited by applicant]
US 10682038B1 · Zhang · 2020 [cited by examiner]
US 20020064314A1 · Comaniciu et al. · 2002 [cited by applicant]
US 20040012600A1 · Deering et al. · 2004 [cited by applicant]
US 20040207642A1 · Crisu et al. · 2004 [cited by applicant]
US 20070024639A1 · Hastings · 2007 [cited by applicant]
US 20070040832A1 · Tan et al. · 2007 [cited by applicant]
US 20070103182A1 · Nawata · 2007 [cited by examiner]
US 20070147690A1 · Ishiwata et al. · 2007 [cited by applicant]
US 20080175519A1 · Nagumo et al. · 2008 [cited by applicant]
US 20090109342A1 · Heng et al. · 2009 [cited by applicant]
US 20090135190A1 · Bakalash et al. · 2009 [cited by applicant]
US 20090167763A1 · Waechter et al. · 2009 [cited by applicant]
US 20090189974A1 · Deering · 2009 [cited by examiner]
US 20090324085A1 · Leung · 2009 [cited by examiner]
US 20100290672A1 · Nishino et al. · 2010 [cited by applicant]
US 20110058014A1 · Yamashita et al. · 2011 [cited by applicant]
US 20110098056A1 · Rhoads · 2011 [cited by examiner]
US 20110212761A1 · Paulsen · 2011 [cited by examiner]
US 20110228975A1 · Hennessey et al. · 2011 [cited by applicant]
US 20110285747A1 · Kilgard · 2011 [cited by applicant]
US 20120016349A1 · Brownell · 2012 [cited by examiner]
US 20130050432A1 · Perez et al. · 2013 [cited by applicant]
US 20130113803A1 · Bakedash et al. · 2013 [cited by applicant]
US 20130141445A1 · Engh-halstvedt et al. · 2013 [cited by applicant]
US 20130187917A1 · Foo et al. · 2013 [cited by applicant]
US 20140118351A1 · Uralsky et al. · 2014 [cited by applicant]
US 20140125650A1 · Neill · 2014 [cited by applicant]
US 20140168035A1 · Luebke et al. · 2014 [cited by applicant]
US 20140168242A1 · Kubisch et al. · 2014 [cited by applicant]
US 20140168783A1 · Luebke et al. · 2014 [cited by applicant]
US 20140218390A1 · Rouet et al. · 2014 [cited by applicant]
US 20140253555A1 · Lum et al. · 2014 [cited by applicant]
US 20140267238A1 · Lum et al. · 2014 [cited by applicant]
US 20140267315A1 · Minkin et al. · 2014 [cited by applicant]
US 20140292771A1 · Kubisch et al. · 2014 [cited by applicant]
US 20140308988A1 · Thorn · 2014 [cited by examiner]
US 20140338724A1 · Tan · 2014 [cited by examiner]
US 20140347359A1 · Gruen et al. · 2014 [cited by applicant]
US 20140354675A1 · Lottes · 2014 [cited by applicant]
US 20150002508A1 · Tatarinov et al. · 2015 [cited by applicant]
US 20150009306A1 · Moore · 2015 [cited by applicant]
US 20150022537A1 · Lum et al. · 2015 [cited by applicant]
US 20150042553A1 · Mecham · 2015 [cited by applicant]
US 20150049004A1 · Deering · 2015 [cited by examiner]
US 20150049104A1 · Lum et al. · 2015 [cited by applicant]
US 20150062142A1 · Goel et al. · 2015 [cited by applicant]
US 20150091902A1 · Pediredla et al. · 2015 [cited by applicant]
US 20150130915A1 · More et al. · 2015 [cited by applicant]
US 20150138065A1 · Alfierri · 2015 [cited by applicant]
US 20150138228A1 · Lum et al. · 2015 [cited by applicant]
US 20150170345A1 · Vaidyanathan et al. · 2015 [cited by applicant]
US 20150170408A1 · He et al. · 2015 [cited by applicant]
US 20150170409A1 · He et al. · 2015 [cited by applicant]
US 20150178983A1 · Akenine-Moller et al. · 2015 [cited by applicant]
US 20150187129A1 · Sloan · 2015 [cited by applicant]
US 20150194128A1 · Hicok · 2015 [cited by applicant]
US 20150264299A1 · Leech et al. · 2015 [cited by applicant]
US 20150273179A1 · Krueger · 2015 [cited by applicant]
US 20150278599A1 · Zhang et al. · 2015 [cited by applicant]
US 20150287240A1 · Janczak et al. · 2015 [cited by applicant]
US 20150294498A1 · Mei et al. · 2015 [cited by applicant]
US 20150317827A1 · Crassin et al. · 2015 [cited by applicant]
US 20160048999A1 · Patney et al. · 2016 [cited by applicant]
US 20160049000A1 · Patney et al. · 2016 [cited by applicant]
US 20160071242A1 · Uralsky et al. · 2016 [cited by applicant]
US 20160071246A1 · Uralsky et al. · 2016 [cited by applicant]
US 20160127715A1 · Shotton et al. · 2016 [cited by applicant]
US 20170068119A1 · Antaki et al. · 2017 [cited by applicant]
US 20170161940A1 · Liktor et al. · 2017 [cited by applicant]
US 20170169602A1 · Blackmon et al. · 2017 [cited by applicant]
US 20170200252A1 · Nguyen et al. · 2017 [cited by applicant]
US 20170213388A1 · Margolis et al. · 2017 [cited by applicant]
US 20170221184A1 · Grossman et al. · 2017 [cited by applicant]
US 20170223395A1 · Elliot et al. · 2017 [cited by applicant]
US 20170227765A1 · Mammou et al. · 2017 [cited by applicant]
US 20170263046A1 · Patney · 2017 [cited by examiner]
US 20170287112A1 · Stafford et al. · 2017 [cited by applicant]
US 20170352324A1 · Spence et al. · 2017 [cited by applicant]
US 20180047203A1 · Grossman et al. · 2018 [cited by applicant]
US 20180157915A1 · Sherry et al. · 2018 [cited by applicant]
US 20180173983A1 · Zink et al. · 2018 [cited by applicant]
US 20180205926A1 · Mogalapalli et al. · 2018 [cited by applicant]
US 20180288423A1 · Vembar · 2018 [cited by examiner]
US 20180357780A1 · Young et al. · 2018 [cited by applicant]
US 20190295503A1 · Ouderkirk et al. · 2019 [cited by applicant]
US 20200003673A1 · Van der Lee · 2020 [cited by examiner]
US 20200111195A1 · Machos et al. · 2020 [cited by applicant]
WO 2017025487A1 · 2017 [cited by applicant]
Nicholas Wilt, “The CUDA Handbook: A Comprehensive Guide to GPU Programming”, 522 pages, Jun. 2013, Addison-Wesley, USA. [cited by applicant]
Shane Cook, “CUDA Programming: A Developer's Guide to Parallel Computing with GPUs”, 591 pages, 2013, Elsevier, USA. [cited by applicant]
Vaidyanathan et al., “Coarse Pixel Shading,” The Eurographics Association, 2014, 10 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 15/477,019, mailed Apr. 5, 2018, 36 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 15/477,019, mailed Jul. 31, 2018, 9 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/173,722, mailed Apr. 18, 2019, 17 pages. [cited by applicant]
Advisory Action for U.S. Appl. No. 16/173,722, mailed Jan. 15, 2020, 5 pages. [cited by applicant]
Office Action for U.S. Appl. No. 16/173,722, mailed Feb. 14, 2020, 22 pages. [cited by applicant]
Office Action for U.S. Appl. No. 16/173,722, mailed May 29, 2020, 25 pages. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 16/173,722, mailed Aug. 21, 2020, 8 pages. [cited by applicant]
Office Action for U.S. Appl. No. 16/662,636, mailed Jan. 8, 2020, 11 pages. [cited by applicant]
Office Action for U.S. Appl. No. 16/662,636, mailed May 15, 2020, 14 pages. [cited by applicant]
Advisory Action for U.S. Appl. No. 16/662,636, mailed Aug. 6, 2020, 3 pages. [cited by applicant]
Office Action for U.S. Appl. No. 16/662,636, mailed Oct. 6, 2020, 14 pages. [cited by applicant]
Office Action for U.S. Appl. No. 16/662,636, mailed Apr. 13, 2021, 14 pages. [cited by applicant]
Guenter et al., “Foveated 3D Graphics,” ACM Transactions on Graphics, vol. 31, No. 6, Article 164, Nov. 2012. [cited by applicant]
Guenter et al., “Foveated 3D Graphics,” YouTube video and screen shots, uploaded Aug. 2016, available at URL: https://www.youtube.com/watch?v=hKSlewfW_5Q. [cited by applicant]
U.S. Appl. No. 17/484,925 entitled “Graphics Anti-Aliasing Resolve With Stencil Mask,” filed Sep. 24, 2021, 121 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 17/484,925, mailed Nov. 30, 2021, 16 pages. [cited by applicant]
Amendment and Response dated Feb. 24, 2022 (from U.S. Appl. No. 17/484,925). [cited by applicant]
Final Office Action for U.S. Appl. No. 17/484,925, mailed Mar. 8, 2022, 15 pages. [cited by applicant]
Amendment and Response dated May 9, 2022 (from U.S. Appl. No. 17/484,925). [cited by applicant]
Advisory Action for U.S. Appl. No. 17/484,925, mailed May 20, 2022, 3 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 17/484,925, mailed Jul. 5, 2022, 15 pages. [cited by applicant]
Corrected Notice of Allowance for U.S. Appl. No. 16/662,636, mailed Apr. 15, 2022, 2 pages. [cited by applicant]
Response to Final Office Action in U.S. Appl. No. 17/484,925, filed Jan. 9, 2023, 12 pages. [cited by applicant]
Advisory Action in U.S. Appl. No. 17/484,925 mailed Feb. 1, 2023, 3 pages. [cited by applicant]
Notice of Allowance in U.S. Appl. No. 17/484,925 mailed Feb. 16, 2023, 8 pages. [cited by applicant]
Guenter et al., “Foveated 3D Graphics,” Aug. 18, 2016; YouTube (Screenshots); https://www.youtube.com/watch?vhKSlewfW_5Q; 99. 13 pages (Year 2016). [cited by applicant]
Guenter et al., “Supplement to Foveated 3D Graphics: User Study Details,” Nov. 2012, ACM Transactions on Graphics, vol. 31, No. 6; pp. 1-4 (Year 2012). [cited by applicant]
Final Office Action of Nov. 8, 2022 for U.S. Appl. No. 17/484,925. [cited by applicant]
Response to Non-Final Office Action in U.S. Appl. No. 17/484,925, filed Oct. 5, 2022. [cited by applicant]