IP Library › Granted Patent US 12,322,023
Granted Patent B2
US 12,322,023 · App. 17/989,671 · Granted Jun 3, 2025

Graphics processing method, system and storage medium for achieving increased hidden surface removal efficiency

Inventors: Tord Kvestad Øygard (Kirkenes, NO); Toni Viki Brkic (Staffanstorp, SE)
Assignee: Arm Limited
G06T15/005G06T15/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,322,023
App. No.
17/989,671
Granted
Jun 3, 2025
Kind
B2
Abstract

When performing tile-based rendering a first, pre-pass operation in which primitives in a sequence of primitives for a tile are processed to determine visibility information for the sequence of primitives, the visibility information being usable to determine whether or not fragments for a primitive in the sequence of primitives should subsequently be processed further for the render output, is performed. Thereafter a second, main pass operation is performed in which the further processing of fragments for primitives that were processed during the first, pre-pass operation is controlled based on the determined visibility information for the sequence of primitives, such that for fragments for which the visibility information indicates that the fragments should not be processed further for the render output some or all of the processing during the second, main pass is omitted.

Claims (33)

1. A method of compiling a fragment shader for a graphics processor to generate a render output, the graphics processor including an execution unit that is operable to execute fragment shaders to process fragments to generate fragment shaded data for the render output, wherein a fragment shader may in addition to generating the fragment shaded data include one or more fragment shader side effects, wherein the graphics processor is configured to:

render primitives in a sequence of primitives for a render output by:

performing a first, pre-pass operation in which primitives in the sequence of primitives are processed into respective sets of one or more fragments, each fragment associated with a respective set of one or more sampling positions within the render output, and the resulting fragments are processed to determine visibility information for the sequence of primitives, wherein the first, pre-pass operation is operable to execute a first fragment shader routine for a fragment, if required, to determine the visibility information for the sequence of primitives; and

thereafter performing a second, main pass operation in which primitives that were processed by the first, pre-pass operation are processed again into their respective sets of one or more fragments for processing for respective sampling positions within the render output, wherein the fragment processing during the second, main pass is controlled based on the determined visibility information, and wherein for fragments that the visibility information indicates should be processed further to generate rendered output data, a second fragment shader routine is executed to generate the rendered output data for the sampling position or positions to which the fragments relate;

the method comprising:

including in a fragment shader for execution by the execution unit of the graphics processor a sequence of instructions that when executed by the first fragment shader routine and/or by the second fragment shader routine controls the execution of one or more fragment shader side effects for a fragment for a primitive such that for any fragments for which both the first fragment shader routine and the second fragment shader routine are executed, the one or more fragment shader side effects associated with the fragment shader are only executed once.

2. The method of claim 1 , wherein the fragment shader includes one or more non-cullable fragment shader side effects, and wherein the sequence of instructions that controls the execution of one or more fragment shader side effects for a fragment for a primitive checks whether the current fragment shader invocation is the first invocation of a fragment shader including the one or more non-cullable fragment shader side effects, and causes the one or more non-cullable fragment shader side effects to be executed on the first invocation of the fragment shader, whether that be by the first fragment shader routine during the first, pre-pass operation or by the second fragment shader routine during the second, main pass operation in the case that the first fragment shader routine was not executed.

3. The method of claim 2 , wherein the sequence of instructions that checks whether the current fragment shader invocation is the first invocation of a fragment shader including the one or more non-cullable fragment shader side effects checks whether a corresponding descriptor indicating that the current invocation is the first invocation has been set.

4. The method of claim 1 , wherein the fragment shader includes one or more potentially cullable fragment shader side effects, and wherein such potentially cullable fragment shader side effects are executed only by the second fragment shader routine, if at all, such that when the determined visibility information indicates that a fragment for a primitive should not be processed further such that the fragment is culled during the second, main pass without executing the second fragment shader routine, the one or more potentially cullable fragment shader side effects are also culled for the fragment.

5. The method of claim 4 , wherein the sequence of instructions that controls the execution of one or more fragment shader side effects for a fragment for a primitive checks whether the graphics processor is performing a respective instance of the first, pre-pass operation, and when the graphics processor is performing a respective instance of the first, pre-pass operation, the sequence of instructions causes the fragment shader to omit the one or more cullable fragment shader side effects.

6. The method of claim 4 , wherein the first fragment shader routine is a partial version of the second fragment shader routine, wherein the first fragment shader routine executes only up to a certain discard point in the second fragment shader routine, and wherein the one or more cullable fragment shader side effects are included in the fragment shader after the discard instruction such that they can only be executed as part of the second fragment shader routine in the second, main pass operation.

7. A method of compiling a fragment shader for a graphics processor to generate a render output, the graphics processor including an execution unit that is operable to execute fragment shaders to process fragments to generate fragment shaded data for the render output, wherein the graphics processor is configured to:

render primitives in a sequence of primitives for a render output by:

performing a first, pre-pass operation in which primitives in the sequence of primitives are processed into respective sets of one or more fragments, each fragment associated with a respective set of one or more sampling positions within the render output, and the resulting fragments are processed to determine visibility information for the sequence of primitives, wherein the first, pre-pass operation is operable to execute a first fragment shader routine for a fragment, if required, to determine the visibility information for the sequence of primitives; and

thereafter performing a second, main pass operation in which primitives that were processed by the first, pre-pass operation are processed again into their respective sets of one or more fragments for processing for respective sampling positions within the render output, wherein the fragment processing during the second, main pass is controlled based on the determined visibility information, and wherein for fragments that the visibility information indicates should be processed further to generate rendered output data, a second fragment shader routine is executed to generate the rendered output data for the sampling position or positions to which the fragments relate;

the method comprising:

for a fragment shader for execution by the execution unit of the graphics processor including the first and second fragment shader routines, and a discard point, wherein the first fragment shader routine executes only up to the discard point in the fragment shader, whereas the second fragment shader routine executes beyond the discard point in the fragment shader:

identifying one or more sequences of instructions corresponding to processing operations that should desirably be executed in the second, main pass operation only, and including the identified one or more sequences of instructions corresponding to the processing operations that should desirably be executed in the second, main pass operation only in the fragment shader after the discard point.

8. A graphics processor configured to generate a render output, the graphics processor comprising a rendering circuit including an execution unit that is operable to execute fragment shaders to process fragments to generate fragment shaded data for the render output, wherein a fragment shader may in addition to generating the fragment shaded data include one or more fragment shader side effects, the graphics processor configured to:

for a sequence of primitives to be processed for a render output:

cause the rendering circuit to:

perform a first, pre-pass operation in which primitives in the sequence of primitives are processed into respective sets of one or more fragments, each fragment associated with a respective set of one or more sampling positions within the render output, and the resulting fragments are processed to determine “visibility” information for the sequence of primitives, wherein the first, pre-pass operation is operable to execute a first fragment shader routine for a fragment, if required, to determine the visibility information for the sequence of primitives; and

thereafter perform a second, main pass operation in which primitives that were processed by the first, pre-pass operation are processed again into their respective sets of one or more fragments for processing for respective sampling positions within the render output, wherein the fragment processing during the second, main pass is controlled based on the determined visibility information, and wherein for fragments that the visibility information indicates should be processed further to generate rendered output data, a second fragment shader routine is executed to generate the rendered output data for the sampling position or positions to which the fragments relate;

the graphic processor further configured to cause the executionunit to:

execute as part of the first fragment shader routine in the first, pre-pass operation and/or as part of the second fragment shader routine in the second, main pass operation a sequence of instructions that controls the execution of the one or more fragment shader side effects for a fragment for a primitive in the sequence of primitives such that the one or more fragment shader side effects are only executed once for the fragment, if at all.

9. The graphics processor of claim 8 , wherein for a fragment for which both the first fragment shader routine and the second fragment shader routine are executed, any non-cullable fragment shader side effects are executed only by the first fragment shader routine in the first, pre-pass operation whereas any cullable fragment shader side effects are executed only by the second fragment shader routine in the second, main pass operation.

10. The graphics processor of claim 8 , configured to determine that a primitive is associated with a fragment shader side effect that is not compatible with being processed by the pre-pass operation, and omit some or all of the first, pre-pass and second, main pass operations for the primitive such that neither the first fragment shader routine nor the second fragment shader routine is executed, and instead subsequently processing the primitive associated with the incompatible fragment shader side effect by a third, fallback operation.

11. A non-transitory computer readable storage medium comprising computer software code which when executing on at least one processor performs a method of compiling a fragment shader for a graphics processor to generate a render output, the graphics processor including an execution unit that is operable to execute fragment shaders to process fragments to generate fragment shaded data for the render output, wherein a fragment shader may in addition to generating the fragment shaded data include one or more fragment shader side effects, wherein the graphics processor is configured to:

render primitives in a sequence of primitives for a render output by:

performing a first, pre-pass operation in which primitives in the sequence of primitives are processed into respective sets of one or more fragments, each fragment associated with a respective set of one or more sampling positions within the render output, and the resulting fragments are processed to determine visibility information for the sequence of primitives, wherein the first, pre-pass operation is operable to execute a first fragment shader routine for a fragment, if required, to determine the visibility information for the sequence of primitives; and

thereafter performing a second, main pass operation in which primitives that were processed by the first, pre-pass operation are processed again into their respective sets of one or more fragments for processing for respective sampling positions within the render output, wherein the fragment processing during the second, main pass is controlled based on the determined visibility information, and wherein for fragments that the visibility information indicates should be processed further to generate rendered output data, a second fragment shader routine is executed to generate the rendered output data for the sampling position or positions to which the fragments relate;

the method comprising:

including in a fragment shader for execution by the execution unit of the graphics processor a sequence of instructions that when executed by the first fragment shader routine and/or by the second fragment shader routine controls the execution of one or more fragment shader side effects for a fragment for a primitive such that for any fragments for which both the first fragment shader routine and the second fragment shader routine are executed, the one or more fragment shader side effects associated with the fragment shader are only executed once.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 6, 2023
From: ØYGARD, TORD KVESTAD; BRKIC, TONI VIKI
To: ARM LIMITED
Reel/Frame 062597/0978 →
Continuity (1)
Related Publication 20240169639A1 · May 23, 2024
References Cited (69)
US 6092097A · Suzuoka · 2000 [cited by examiner]
US 6246415B1 · Grossman et al. · 2001 [cited by applicant]
US 6480205B1 · Greene et al. · 2002 [cited by applicant]
US 6535209B1 · Abdalla et al. · 2003 [cited by applicant]
US 7023437B1 · Voorhies · 2006 [cited by examiner]
US 7375727B1 · Greene et al. · 2008 [cited by applicant]
US 8325184B2 · Jiao · 2012 [cited by examiner]
US 8698820B2 · Cox et al. · 2014 [cited by applicant]
US 9626795B2 · Sathe · 2017 [cited by examiner]
US 9767595B2 · Nystad · 2017 [cited by examiner]
US 10074210B1 · Spencer · 2018 [cited by examiner]
US 10096147B2 · Goel · 2018 [cited by examiner]
US 11030783B1 · Engh-Halstvedt et al. · 2021 [cited by applicant]
US 12154224B2 · Achrenius · 2024 [cited by examiner]
US 20020196252A1 · Liao et al. · 2002 [cited by applicant]
US 20070257905A1 · French et al. · 2007 [cited by applicant]
US 20140218390A1 · Rouet · 2014 [cited by examiner]
US 20140267256A1 · Heggelund · 2014 [cited by examiner]
US 20140354661A1 · Balci et al. · 2014 [cited by applicant]
US 20150062154A1 · Ellis · 2015 [cited by examiner]
US 20160005140A1 · Engh-Halstvedt et al. · 2016 [cited by applicant]
US 20160035129A1 · Bolz et al. · 2016 [cited by applicant]
US 20160098856A1 · Broadhurst et al. · 2016 [cited by applicant]
US 20160217608A1 · Howson · 2016 [cited by examiner]
US 20170161940A1 · Liktor · 2017 [cited by examiner]
US 20170178386A1 · Redshaw et al. · 2017 [cited by applicant]
US 20180025463A1 · Kazakov et al. · 2018 [cited by applicant]
US 20180082467A1 · Andersson · 2018 [cited by examiner]
US 20180082469A1 · Andersson · 2018 [cited by examiner]
US 20180197323A1 · Howson · 2018 [cited by examiner]
US 20180268604A1 · Story · 2018 [cited by applicant]
US 20180349315A1 · Heggelund · 2018 [cited by examiner]
US 20190088009A1 · Forey et al. · 2019 [cited by applicant]
US 20190311535A1 · Lacey · 2019 [cited by applicant]
US 20190362533A1 · Fuller · 2019 [cited by applicant]
US 20200134894A1 · Uhrenholt · 2020 [cited by applicant]
US 20200312020A1 · Zhong · 2020 [cited by examiner]
US 20200402297A1 · Brigg et al. · 2020 [cited by applicant]
US 20210065437A1 · Brkic · 2021 [cited by examiner]
US 20210110510A1 · Brigg et al. · 2021 [cited by applicant]
US 20210407191A1 · Yang · 2021 [cited by examiner]
US 20220092801A1 · Achrenius et al. · 2022 [cited by applicant]
US 20220207827A1 · Acharya et al. · 2022 [cited by applicant]
US 20220237730A1 · Brkic · 2022 [cited by applicant]
US 20240087078A1 · Ashkar · 2024 [cited by examiner]
US 20240169639A1 · Øygard · 2024 [cited by examiner]
WO 2018052592A1 · 2018 [cited by applicant]
Combined Search and Examination Report under Sections 17 and 18(3) dated Jun. 3, 2024, GB Patent Application No. GB 2317492.3, pp. 1-7. [cited by applicant]
Combined Search and Examination Report under Sections 17 and 18(3) dated May 24, 2023, GB Patent Application No. GB 2217223.3, pp. 1-6. [cited by applicant]
Combined Search and Examination Report under Sections 17 and 18(3) dated May 30, 2023, GB Patent Application No. GB 2217253.0, pp. 1-6. [cited by applicant]
Search Report under Section 17 dated May 19, 2023, GB Patent Application No. GB 2217247.2, pp. 1-4. [cited by applicant]
Combined Search and Examination Report under Sections 17 and 18(3) dated May 19, 2023, GB Patent Application No. GB 2217238.1, pp. 1-7. [cited by applicant]
Search Report under Section 17 dated Jun. 19, 2023, GB Patent Application No. GB 2217246.4, pp. 1-4. [cited by applicant]
Øygard, T.K., et al., “Graphics Processors”, U.S. Appl. No. 17/989,548, filed Nov. 17, 2022. [cited by applicant]
Øygard, T.K., et al., “Graphics Processors”, U.S. Appl. No. 18/509,426, filed Nov. 15, 2023. [cited by applicant]
Øygard, T.K., et al., “Graphics Processors”, U.S. Appl. No. 18/509,432, filed Nov. 15, 2023. [cited by applicant]
Øygard, T.K., et al., “Graphics Processors”, U.S. Appl. No. 18/509,436, filed Nov. 15, 2023. [cited by applicant]
Øygard, T.K., et al., “Graphics Processors”, U.S. Appl. No. 18/509,439, filed Nov. 15, 2023. [cited by applicant]
Kjøll, P.K., et al., “Graphics Processors”, U.S. Appl. No. 18/509,679, filed Nov. 15, 2023. [cited by applicant]
Ruud, O.M., et al., “Graphics Processors”, U.S. Appl. No. 18/509,441, filed Nov. 15, 2023. [cited by applicant]
Ruud, O.M., et al., “Graphics Processors”, U.S. Appl. No. 18/509,448, filed Nov. 15, 2023. [cited by applicant]
Ruud, O.M., et al., “Graphics Processors”, U.S. Appl. No. 18/509,676, filed Nov. 15, 2023. [cited by applicant]
Ruud, O.M., et al., “Graphics Processors”, U.S. Appl. No. 18/509,687, filed Nov. 15, 2023. [cited by applicant]
PCT International Search Report and Written Opinion, dated Feb. 19, 2024, PCT International Application No. PCT/GB2023/052995, pp. 1-17. [cited by applicant]
Extended European Search Report dated Jul. 5, 2024, European Patent Application No. 23210210.3, pp. 1-10. [cited by applicant]
Response to Non-Final Office Action dated Sep. 19, 2024, U.S. Appl. No. 17/989,548, 14 pages. [cited by applicant]
Non-Final Office Action dated Aug. 2, 2024, U.S. Appl. No. 17/989,548, 37 pages. [cited by applicant]
Notice of Allowance dated Nov. 22, 2024, U.S. Appl. No. 17/989,548, 25 pages. [cited by applicant]
Examination Report dated Apr. 1, 2025, GB Patent Application No. GB2217238.1, 4 pages. [cited by applicant]
Cited By (2)
US 12,626,462 US 12,639,881