IP Library › Granted Patent US 12664432
Granted Patent B1
US 12664432 · App. 17/950,018 · Granted Jun 23, 2026

Removing neural network layers

Inventors: Slawomir Kierat (Warsaw, PL); Mateusz Sieniawski (Warsaw, PL); Piotr Karpinski (Warsaw, PL); Pawel Morkisz (San Jose, CA); Szymon Migacz (Santa Clara, CA); Linnan Wang (Pleasanton, CA); Chen-Han Yu (Mountain House, CA); Satish Salian (Santa Clara, CA); Ashwath Aithal (Fremont, CA); Alexandru Fit-Florea (Los Altos Hills, CA)
Assignee: NVIDIA Corporation
G06N3/082
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12664432
App. No.
17/950,018
Granted
Jun 23, 2026
Kind
B1
Abstract

Apparatuses, systems, and techniques to determine whether to remove one or more neural network layers. In at least one embodiment, one or more neural network layers are determined to be removed based on, for example, a neural architecture search (NAS).

Claims (41)

1 . A processor, comprising:

one or more circuits to use a neural architecture search (NAS) to;

obtain one or more performance metrics for potential variations of a neural network including a first variation with a layer replaced by a skip connection block and a second variation with a block implementing a neural network operation for the layer;

modify at least one performance metric of the one or more performance metrics by a first value for the first variation and by a second value for the second variation, where the first value is different than the second value; and

determine whether to remove one or more neural network layers based, at least in part, on a comparison between the at least one performance metric modified by the first value and the at least one performance metric modified by the second value.

2 . The processor of claim 1 , wherein the one or more circuits are to determine whether to remove the one or more neural network layers as a result of calculating the one or more neural network layers from a set of candidate neural network layers.

3 . The processor of claim 1 , wherein the one or more circuits are to determine whether to remove the one or more neural network layers by at least scaling an output of the one or more neural network layers.

4 . The processor of claim 1 , wherein the one or more circuits are to remove the one or more neural network layers by at least selecting a skip connection operation associated with the one or more neural network layers.

5 . The processor of claim 1 , wherein the one or more circuits are to generate one or more neural networks based, at least in part, on the determination of whether to remove the one or more neural network layers.

6 . The processor of claim 1 , wherein the one or more circuits are to compare the at least one performance metric when the one or more neural network layers is removed and the at least one performance metric when the one or more neural network layers is not removed with a threshold value to determine whether to remove the one or more neural network layers.

7 . The processor of claim 1 , wherein the one or more circuits are to use the NAS in connection with one or more graphics processing units (GPUs).

8 . A system, comprising:

one or more computers having one or more processors to:

obtain one or more performance metrics for potential variations of a neural network including a first variation with a layer replaced by a skip connection block and a second variation with a block implementing a neural network operation for the layer;

modify at least one performance metric of the one or more performance metrics by a first value for the first variation and by a second value for the second variation, where the first value is different than the second value; and

determine whether to remove one or more neural network layers based, at least in part, on a comparison between the at least one performance metric modified by the first value and the at least one performance metric modified by the second value.

9 . The system of claim 8 , wherein the one or more processors are further to determine whether to remove the one or more neural network layers as a result of reducing a set of candidate neural network layers to the one or more neural network layers.

10 . The system of claim 8 , wherein the one or more processors are further to use the NAS to:

calculate a first value of the at least one performance metric when the one or more neural network layers is removed based, at least in part, on a training dataset;

calculate a second value of the at least one performance metric when the one or more neural network layers is not removed based, at least in part, on the training dataset; and

determine whether to remove the one or more neural network layers through at least a comparison between the first value and the second value, and a threshold value.

11 . The system of claim 8 , wherein the one or more processors are further to remove the one or more neural network layers as a result of calculating a value of the at least one performance metric when the one or more neural network layers is not removed that is below a threshold.

12 . The system of claim 8 , wherein the one or more processors are further to scale data that bypasses the one or more neural network layers to determine whether to remove the one or more neural network layers.

13 . The system of claim 8 , wherein the one or more processors are further to, as a result of determining to use the one or more neural network layers, output one or more neural networks comprising the one or more neural network layers.

14 . The system of claim 8 , wherein the one or more processors are further to perform the NAS in connection with one or more parallel processing units (PPUs).

15 . A method, comprising:

using a neural architecture search (NAS) to:

obtain one or more performance metrics for potential variations of a neural network including a first variation with a layer replaced by a skip connection block and a second variation with a block implementing a neural network operation for the layer;

modify at least one performance metric of the one or more performance metrics by a first value for the first variation and by a second value for the second variation, where the first value is different than the second value; and

determine whether to remove one or more neural network layers based, at least in part, on a comparison between the at least one performance metric modified by the first value and the at least one performance metric modified by the second value.

16 . The method of claim 15 , further comprising:

calculating one or more values of the at least one performance metric;

comparing the one or more values with a threshold value; and

determining whether to remove the one or more neural network layers based, at least in part, on the comparison of the one or more values with the threshold value.

17 . The method of claim 15 , further comprising:

multiplying an output of the one or more neural network layers by a first value;

multiplying data that bypasses the one or more neural network layers by a second value; and

calculating one or more weight values based, at least in part, on the output and the data to determine whether to remove the one or more neural network layers.

18 . The method of claim 15 , wherein the NAS is based, at least in part, on one or more latency constraints.

19 . The method of claim 15 , further comprising, as a result of determining to remove the one or more neural network layers, outputting one or more neural networks without the one or more neural network layers.

20 . A non-transitory computer readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least perform the method of claim 15 .