IP Library › Granted Patent US 12675696
Granted Patent B2
US 12675696 · App. 17/850,807 · Granted Jul 7, 2026

Sparsity for neural network models based on sparsity attributes

Inventors: Ningxin Zheng (Shanghai, CN); Quanlu Zhang (Beijing, CN); Yuqing Yang (Shanghai, CN); Lingxiao Ma (Beijing, CN); Fan Yang (Beijing, CN); Yang Wang (Beijing, CN); Mao Yang (Beijing, CN); Lidong Zhou (Beijing, CN)
G06N3/082G06N3/084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12675696
App. No.
17/850,807
Granted
Jul 7, 2026
Kind
B2
Abstract

Embodiments of the present disclosure include systems and methods for providing sparsity for neural network models based on sparsity attributes. A first neural network model definition is received. The first neural network model definition specifies a neural network model comprising a set of tensors and a set of sparsity attribute values for elements of a tensor in the set of tensors. The set of sparsity attribute values for the tensor are propagated to elements of a subset of the set of tensors to form a second neural network model definition. The neural network model is generated based on the second neural network model definition.

Claims (34)

1 . A non-transitory machine-readable medium storing a program executable by at least one processing unit of a device, the program comprising sets of instructions for:

receiving a first neural network model definition, the first neural network model definition specifying a neural network model comprising a set of tensors and a set of sparsity attribute values for elements of a tensor in the set of tensors;

performing structural propagation of the sparsity attribute values as metadata of the first neural network model definition by applying a predefined set of operator-specific structural propagation rules that, based on data-flow-graph connectivity of the first neural network model definition, determine pruned or quantized tensor elements and assign corresponding sparsity attributes to elements of a subset of the set of tensors;

constructing a second neural network model definition that encodes the propagated sparsity attribute values; and

generating the neural network model based on the second neural network model definition, wherein generating the neural network model comprises transforming an operator in the set of operators operating on second and third tensors in the set of tensors into a plurality of sub-operators.

2 . The non-transitory machine-readable medium of claim 1 , wherein the set of sparsity attribute values for elements of the tensor in the set of tensors is a first set of sparsity attribute values for elements of a first tensor in the set of tensors, wherein the neural network model further comprises a set of operators configured to operate on the set of tensors.

3 . The non-transitory machine-readable medium of claim 2 , wherein transforming the operator into the plurality of sub-operators comprises transforming a second set of sparsity attribute values for elements of the second tensor into a plurality of sets of sparsity attribute values for elements of a plurality of tensors operated on by the plurality of sub-operators, wherein each set of sparsity attribute values in the plurality of sets of sparsity attribute values comprises at least one sparsity attribute value in the second set of sparsity attribute values.

4 . The non-transitory machine-readable medium of claim 2 , wherein generating the executable neural network model further comprises generating a plurality of code for the plurality of sub-operators to implement the operator, identifying a set of dead code in the plurality of code, and eliminating the set of dead code from the plurality of code.

5 . The non-transitory machine-readable medium of claim 1 , wherein the set of sparsity attribute values comprises a sparsity attribute value indicating to prune an element in the tensor.

6 . The non-transitory machine-readable medium of claim 1 , wherein the set of sparsity attribute values comprises a sparsity attribute value indicating to quantize an element in the tensor.

7 . The non-transitory machine-readable medium of claim 1 , wherein propagating the set of sparsity attribute values comprises propagating the set of sparsity attribute values forwards through the neural network model.

8 . The non-transitory machine-readable medium of claim 1 , wherein propagating the set of sparsity attribute values comprises propagating the set of sparsity attribute values backwards through the neural network model.

9 . The non-transitory machine-readable medium of claim 1 , wherein propagating the set of sparsity attribute values for the tensor to the elements of the subset of the set of tensors is based on a set of propagation rules.

10 . A method comprising:

receiving a first neural network model definition, the first neural network model definition specifying a neural network model comprising a set of tensors and a set of sparsity attribute values for elements of a tensor in the set of tensors;

performing structural propagation of the sparsity attribute values as metadata of the first neural network model definition by applying a predefined set of operator-specific structural propagation rules that, based on data-flow-graph connectivity of the first neural network model definition, determine pruned or quantized tensor elements and assign corresponding sparsity attributes to elements of a subset of the set of tensors;

constructing a second neural network model definition that encodes the propagated sparsity attribute values; and

generating the neural network model based on the second neural network model definition, wherein generating the neural network model comprises transforming an operator in the set of operators operating on second and third tensors in the set of tensors into a plurality of sub-operators.

11 . The method of claim 10 , wherein the set of sparsity attribute values for elements of the tensor in the set of tensors is a first set of sparsity attribute values for elements of a first tensor in the set of tensors, wherein the neural network model further comprises a set of operators configured to operate on the set of tensors.

12 . The method of claim 11 , wherein transforming the operator into the plurality of sub-operators comprises transforming a second set of sparsity attribute values for elements of the second tensor into a plurality of sets of sparsity attribute values for elements of a plurality of tensors operated on by the plurality of sub-operators, wherein each set of sparsity attribute values in the plurality of sets of sparsity attribute values comprises at least one sparsity attribute value in the second set of sparsity attribute values.

13 . The method of claim 11 , wherein generating the executable neural network model further comprises generating a plurality of code for the plurality of sub-operators to implement the operator, identifying a set of dead code in the plurality of code, and eliminating the set of dead code from the plurality of code.

14 . The method of claim 10 , wherein the set of sparsity attribute values comprises a sparsity attribute value indicating to prune an element in the tensor.

15 . The method of claim 10 , wherein the set of sparsity attribute values comprises a sparsity attribute value indicating to quantize an element in the tensor.

16 . The method of claim 10 , wherein propagating the set of sparsity attribute values comprises propagating the set of sparsity attribute values forwards through the neural network model.

17 . The method of claim 10 , wherein propagating the set of sparsity attribute values comprises propagating the set of sparsity attribute values backwards through the neural network model.

18 . The method of claim 10 , wherein propagating the set of sparsity attribute values for the tensor to the elements of the subset of the set of tensors is based on a set of propagation rules.

19 . A system comprising:

a set of processing units; and

a non-transitory machine-readable medium storing instructions that when executed by at least one processing unit in the set of processing units cause the at least one processing unit to:

receive a first neural network model definition, the first neural network model definition specifying a neural network model comprising a set of tensors and a set of sparsity attribute values for elements of a tensor in the set of tensors;

perform structural propagation of the sparsity attribute values as metadata of the first neural network model definition by applying a predefined set of operator-specific structural propagation rules that, based on data-flow-graph connectivity of the first neural network model definition, determine pruned or quantized tensor elements and assign corresponding sparsity attributes to elements of a subset of the set of tensors;

construct a second neural network model definition that encodes the propagated sparsity attribute values and

generate the neural network model based on the second neural network model definition, wherein generating the neural network model comprises transforming an operator in the set of operators operating on second and third tensors in the set of tensors into a plurality of sub-operators.

20 . The system of claim 19 , wherein the set of sparsity attribute values for elements of the tensor in the set of tensors is a first set of sparsity attribute values for elements of a first tensor in the set of tensors, wherein the neural network model further comprises a set of operators configured to operate on the set of tensors.