IP Library Granted Patent US 10,339,443
Granted Patent B1
US 10,339,443 · App. 15/441,963 · Granted Jul 2, 2019

Systems and methods for processing convolutional neural network operations using textures

Inventor: Tom Medioni (Paris, FR)
Assignee: GoPro, Inc.
G06N3/06G06N3/08G06T7/40G06T15/04G06T15/80H03H17/0202G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,339,443
App. No.
15/441,963
Granted
Jul 2, 2019
Kind
B1
Abstract

Convolutional neural network information may define a convolutional neural network including layers. The layers may define operations on an input to the convolutional neural network. The layers in the convolutional neural network information may be formatted as shaders. Input information defining the input to the convolutional neural network may be accessed. The input information may be formatted as an array of textures. The shaders may be applied to the textures to effectuate processing the input to the convolutional neural network through the layers of the convolutional neural network. One or more results may be obtained from applying the shaders to the array of textures.

Claims (38)

1. A system for processing convolutional neural network operations using textures, the system comprising:

physical storage media storing convolutional neural network information, the convolutional neural network information defining a convolutional neural network, the convolutional neural network including layers, the layers defining operations on an input to the convolutional neural network, wherein the layers in the convolutional neural network information are formatted as shaders; and

one or more physical processors configured by machine-readable instructions to:

access the input to the convolutional neural network;

format the input as an array of textures;

store the array of textures as input information;

access the input information, the input information defining the input to the convolutional neural network, the input information formatted as the array of textures, the textures characterized by a width, a height, and a number of channels;

apply the shaders to the array of textures, wherein applying the shaders to the array of textures effectuates processing the input to the convolutional neural network through the layers of the convolutional neural network; and

obtain one or more results from applying the shaders to the array of textures.

2. The system of claim 1 , wherein the one or more physical processors are further configured by machine-readable instructions to:

access layer information, the layer information defining one or more of the layers of the convolutional neural network;

format one or more of the operations defined by the one or more of the layers as one or more of the shaders; and

add the shaders to the convolutional neural network information.

3. The system of claim 1 , wherein the number of channels of the textures is four.

4. The system of claim 3 , wherein individual shaders output information for four channels.

5. The system of claim 1 , further comprising a graphics processing unit, wherein the shaders are applied to the array of textures by the graphics processing unit.

6. The system of claim 5 , further comprising a graphics processing unit memory, wherein one or more inputs to the shaders and one or more outputs of the shaders are stored in the graphics processing unit memory.

7. The system of claim 5 , wherein the graphics processing unit is included in a mobile device.

8. The system of claim 1 , wherein the layers of the convolutional neural network include one or more convolution layers, the convolution layers processed using binary convolution.

9. The system of claim 1 , wherein the array of textures are stored using an int32 structure, the int32 structure including four int8 structures, an individual int8 structure storing a four by two patch of a single channel of an individual texture.

10. A method for processing convolutional neural network operations using textures, the method comprising:

accessing the input to a convolutional neural network;

formatting the input as an array of textures;

storing the array of textures as input information

accessing the input information, the input information defining the input to the convolutional neural network, the convolutional neural network defined by convolutional neural network information, the convolutional neural network including layers, the layers defining operations on the input to the convolutional neural network, wherein the layers in the convolutional neural network information are formatted as shaders and the input information is formatted as the array of textures, the textures characterized by a width, a height, and a number of channels;

applying the shaders to the array of textures, wherein applying the shaders to the array of textures effectuates processing the input to the convolutional neural network through the layers of the convolutional neural network; and

obtaining one or more results from applying the shaders to the array of textures.

11. The method of claim 10 , further comprising:

accessing layer information, the layer information defining one or more of the layers of the convolutional neural network;

formatting one or more of the operations defined by the one or more of the layers as one or more of the shaders; and

adding the shaders to the convolutional neural network information.

12. The method of claim 10 , wherein the number of channels of the textures is four.

13. The method of claim 12 , wherein individual shaders output information for four channels.

14. The method of claim 10 , wherein the shaders are applied to the array of textures by a graphics processing unit.

15. The method of claim 14 , wherein one or more inputs to the shaders and one or more outputs of the shaders are stored in a graphics processing unit memory.

16. The method of claim 14 , wherein the graphics processing unit is included in a mobile device.

17. The method of claim 10 , wherein the layers of the convolutional neural network include one or more convolution layers, the convolution layers processed using binary convolution.

18. The method of claim 10 , wherein the array of textures are stored using an int32 structure, the int32 structure including four int8 structures, an individual int8 structure storing a four by two patch of a single channel of an individual texture.

Assignments (7)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY INTEREST Recorded Oct 19, 2020
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054113/0594 →
CORRECTIVE ASSIGNMENT TO CORRECT THE SCHEDULE TO REMOVE APPLICATION 15387383 AND REPLACE WITH 15385383 PREVIOUSLY RECORDED ON REEL 042665 FRAME 0065. ASSIGNOR(S) HEREBY CONFIRMS THE SECURITY INTEREST. Recorded Oct 23, 2019
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 050808/0824 →
SECURITY INTEREST Recorded Jun 1, 2017
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 042665/0065 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2017
From: MEDIONI, TOM
To: GOPRO, INC.
Reel/Frame 041372/0486 →
Cited By (2)
US 12,271,990 US 12,614,403