IP Library Granted Patent US 11,967,012
Granted Patent B2
US 11,967,012 · App. 17/849,322 · Granted Apr 23, 2024

Efficient convolution operations

Inventors: Rostam King (Hertfordshire, GB); William Thomas (Hertfordshire, GB)
Assignee: Imagination Technologies Limited
G06T15/005G06T15/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,967,012
App. No.
17/849,322
Granted
Apr 23, 2024
Kind
B2
Abstract

A method of operation of a texturing/shading unit in a GPU pipeline is used for efficient convolution operations. The method uses texture hardware to collectively fetch all the texels required to calculate properties for a group of output pixels without any duplication. The method then bypasses bilinear filter hardware in the texture hardware and passes the fetched and unfiltered texel data from the texture hardware unit to shader hardware in the texturing/shading unit. The shader hardware uses the fetched texel data to perform a plurality of convolution operations to calculate the properties of each of the output pixel.

Claims (45)

1. A method of operation of a texturing/shading unit in a GPU pipeline, the method comprising:

collectively fetching, by texture hardware in the texturing/shading unit, all texels required to calculate properties for a group of output pixels;

for all the texels required to calculate properties for the group of output pixels, bypassing bilinear filter hardware in the texture hardware and passing the fetched and unfiltered texel data from the texture hardware unit to shader hardware in the texturing/shading unit; and

performing a plurality of convolution operations in the shader hardware using the texel data to calculate the properties of each output pixel in the group of output pixels.

2. The method according to claim 1 , wherein each texel required to calculate properties for the group of output pixels is fetched only once.

3. The method according to claim 1 , further comprising, prior to performing the plurality of convolution operations:

broadcasting the texel data to registers allocated to instances corresponding to each of the output pixels.

4. The method according to claim 3 , wherein all the fetched texel data for a group of output pixels is broadcast to registers allocated to each of the instances.

5. The method according to claim 3 , wherein performing a plurality of convolution operations using the texel data to calculate the properties for the group of output pixels comprises, for each output pixel:

accessing texel data stored in registers allocated to corresponding instance; and

performing a plurality of convolution operations using the accessed texel data and a set of weights to calculate the properties for the output pixel, wherein the weight for any texel that is outside a kernel of the convolution for the output pixel is set to zero.

6. The method according to claim 3 , wherein the fetched texel data is selectively broadcast to registers allocated to instances corresponding to each of the output pixels such that registers for an instance only receive texel data for use in performing convolution operations for that instance.

7. The method according to claim 3 , wherein performing a plurality of convolution operations using the texel data to calculate the properties for the group of output pixels comprises, for each output pixel:

accessing texel data stored in registers allocated to corresponding instance; and

performing a plurality of convolution operations using the accessed texel data to calculate the properties for the output pixel.

8. The method according to claim 1 , wherein the fetched texel data is stored in shared registers for the group of output pixels.

9. The method according to claim 8 , wherein performing a plurality of convolution operations using the texel data to calculate the properties for the group of output pixels comprises, for each output pixel:

accessing a subset of the fetched texel data for the group of output pixels from the shared registers, the subset corresponding to a kernel of the convolution for the output pixel with zeros; and

performing a convolution operation in the shader hardware using the accessed texel data to calculate the properties for the output pixel.

10. The method according to claim 8 , wherein performing a plurality of convolution operations using the texel data to calculate the properties for the group of output pixels comprises, for each output pixel:

accessing a weight for each texel in the fetched texel data, wherein the weight for any texel that is outside a kernel of the convolution for the output pixel is set to zero; and

performing a convolution operation in the shader hardware using the fetched texel data and the accessed weights to calculate the properties for the output pixel.

11. The method according to claim 10 , wherein accessing a weight for each texel in the fetched texel data comprises selecting a subset of a weight array, the subset dependent upon the output pixel and wherein in the selected subset the weight for any texel that is outside a kernel of the convolution for the output pixel is set to zero.

12. The method according to claim 10 , wherein accessing a weight for each texel in the fetched texel data comprises accessing one of a plurality of pre-defined weight arrays, each of the weight arrays corresponding to a different one of the plurality of output pixels and in which a different subset of the weights are set to zero.

13. The method according to claim 1 , wherein collectively fetching all texels required to calculate properties for a group of output pixels comprises:

executing a plurality of fetch operations for different sample positions, each sample position being offset from a texel centre and defining a different, non-overlapping patch of adjacent texels to be fetched.

14. The method according to claim 1 , wherein collectively fetching all texels required to calculate properties for a group of output pixels comprises:

executing a plurality of gather operations for different sample positions, each sample position being offset by integer values from a centre of one of the output pixels and defining a different, non-overlapping patch of adjacent texels to be fetched.

15. A texturing/shading unit for use in a GPU pipeline, the texturing/shading unit comprising:

texture hardware comprising a fetch unit and bilinear filter hardware; and

shader hardware,

wherein the texture hardware is arranged to fetch, in the fetch unit, all texels required to calculate properties for a group of output pixels,

for all the texels required to calculate properties for the group of output pixels, bypass the bilinear filter hardware and output the fetched and unfiltered texel data to shader hardware, and

the shader hardware is arranged to perform a plurality of convolution operations using the texel data to calculate the properties of each output pixel in the group of output pixels.

16. The texturing/shading unit according to claim 15 , wherein each texel required to calculate properties for the group of output pixels is fetched only once.

17. The texturing/shading unit according to claim 15 , wherein the shader hardware is arranged, prior to performing the plurality of convolution operations, to broadcast the texel data to registers allocated to instances corresponding to each of the output pixels.

18. The texturing/shading unit according to claim 17 , wherein all the fetched texel data for a group of output pixels is broadcast to registers allocated to each of the instances.

19. The texturing/shading unit according to claim 17 , wherein the shader hardware is arranged to perform the plurality of convolution operations using the texel data to calculate the properties for the group of output pixels comprises, for each output pixel, by:

accessing texel data stored in registers allocated to corresponding instance; and

performing a plurality of convolution operations using the accessed texel data and a set of weights to calculate the properties for the output pixel, wherein the weight for any texel that is outside a kernel of the convolution for the output pixel is set to zero.

20. A non-transitory computer readable storage medium having stored thereon a computer readable dataset description of an integrated circuit that, when processed in an integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture a texturing/shading unit for use in a GPU pipeline, the texturing/shading unit comprising:

texture hardware comprising a fetch unit and a bilinear filter hardware; and shader hardware,

wherein the texture hardware is arranged to fetch, in the fetch unit, all texels required to calculate properties for a group of output pixels,

for all the texels required to calculate properties for the group of output pixels, bypass the bilinear filter hardware and output the fetched and unfiltered texel data to shader hardware, and

the shader hardware is arranged to perform a plurality of convolution operations using the texel data to calculate the properties of each output pixel in the group of output pixels.

Assignments (2)
SECURITY INTEREST Recorded Jul 31, 2024
From: IMAGINATION TECHNOLOGIES LIMITED
To: FORTRESS INVESTMENT GROUP (UK) LTD
Reel/Frame 068221/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 15, 2022
From: KING, ROSTAM; THOMAS, WILLIAM
To: IMAGINATION TECHNOLOGIES LIMITED
Reel/Frame 061767/0389 →