IP Library Granted Patent US 10,318,317
Granted Patent B2
US 10,318,317 · App. 15/975,930 · Granted Jun 11, 2019

Processing core with operation suppression based on contribution estimate

Inventors: Ljubisa Bajic (Toronto, CA); Milos Trajkovic (Toronto, CA); Ivan Hamer (Toronto, CA); Syed Gilani (Markham, CA)
Assignee: Tenstorrent Inc.
G06F9/4494G06F7/50G06F7/523G06F15/76G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,318,317
App. No.
15/975,930
Granted
Jun 11, 2019
Kind
B2
Abstract

Processing cores with the ability to suppress operations based on a contribution estimate for those operation for purposes of increasing the overall performance of the core are disclosed. Associated methods that can be conducted by such processing cores are also disclosed. One such method includes generating a reference value for a composite computation. A complete execution of the composite computation generates a precise output and requires execution of a set of component computations. The method also includes generating a component computation approximation. The method also includes evaluating the component computation approximation with the reference value. The method also includes executing a partial execution of the composite computation using the component computation approximation to produce an estimated output. The method also includes suppressing the component computation, while executing the partial execution, based on the evaluation of the component computation approximation with the reference value.

Claims (82)

1. A computer-implemented method, wherein each step is executed by a processing core, comprising:

generating a reference value for a composite computation, wherein a complete execution of the composite computation generates a precise output and requires execution of a set of component computations;

generating a component computation approximation for a component computation in the set of component computations;

evaluating the component computation approximation with the reference value;

executing a partial execution of the composite computation to produce an estimated output, wherein the partial execution of the composite computation uses the component computation approximation; and

suppressing the component computation, while executing the partial execution, based on the evaluation of the component computation approximation with the reference value.

2. The computer-implemented method of claim 1 , further comprising:

generating a second component computation approximation for a second component computation in the set of component computations;

evaluating the second component computation approximation; and

executing the second component computation, while executing the partial execution, conditionally based on the evaluation of the second component computation approximation.

3. The computer-implemented method of claim 1 , wherein:

the composite computation is a sum-of-products computation;

the component computation is a product computation;

the precise output requires execution of the set of component computations at a first resolution; and

the component computation approximation is generated using a lower resolution execution of the product computation.

4. The computer-implemented method of claim 3 , wherein generating the reference value includes:

generating a set of component computations approximations for a subset of the set of component computations; and

summing the set of component computation approximations to generate a sum.

5. The computer-implemented method of claim 1 , wherein:

the composite computation is a multi-product computation between a set of input data and a first filter segment from a first filter;

the component computation is a product computation conducted with: (i) a filter entry from the first filter segment; and (ii) an entry from the set of input data; and

the component computation approximation is generated using a lower resolution execution of the product computation.

6. The computer-implemented method of claim 5 , wherein generating the reference value includes:

generating a second component computation approximation using: (i) a filter entry from a second filter segment from the first filter; and (ii) a second entry from the set of input data.

7. The computer-implemented method of claim 6 , wherein:

generating the reference value includes selecting a maximum exponent of the second component computation approximation;

evaluating the component computation approximation with the reference value includes comparing a maximum exponent of the component computation approximation with an exponent of the second component computation approximation; and

suppressing the component computation when the maximum exponent of the component computation approximation is smaller than the maximum exponent of the second component computation approximation.

8. The computer-implemented method of claim 1 , further comprising:

comparing the component computation approximation to the reference value;

wherein the generating the reference value step uses a programmable tolerance.

9. The computer-implemented method of claim 1 , wherein:

the method consumes a first amount of computational resources;

the complete execution of the composite computation consumes a second amount of computational resources; and

the first amount is less than the second amount.

10. The computer-implemented method of claim 1 , wherein:

the composite computation is a sum-of-products computation; and

the set of component computations includes a set of product computations.

11. The computer-implemented method of claim 10 , wherein:

the component computation is a multiplication computation with a first floating point input and a second floating point input; and

the generating of the component computation approximation step includes adding an exponent of the first floating point input with an exponent of the second floating point input to generate an exponent sum.

12. The computer-implemented method of claim 11 , wherein:

the exponent sum is generated by an adder; and

the adder has less than 9-bit precision.

13. The computer-implemented method of claim 1 , wherein generating the reference value for the composite computation comprises:

generating a set of component computations approximations; and

selecting a largest component computation approximation in the set of component computation approximations;

wherein the reference value is a fraction of the largest component computation approximation.

14. The computer-implemented method of claim 1 , wherein generating the reference value for the composite computation comprises:

generating a set of component computations approximations; and

summing the set of component computation approximations to generate a sum;

wherein the reference value is a fraction of the sum.

15. The computer-implemented method of claim 10 , wherein generating the reference value for the composite computation comprises:

continuously updating the reference value while executing the partial execution to be equivalent to a current accumulated value of the sum-of-products computation less a scaling percentage of the current accumulated value; and

wherein the scaling percentage increases as the partial execution is conducted.

16. The computer-implemented method of claim 1 , wherein:

a set of operands for the composite computation are located in a register file;

the suppressing of the component computation step is conducted by a suppression logic block; and

the suppressing of the component computation step includes:

inhibiting a first subset of the set of operands from being passed through the suppression logic block; and

transmitting a second subset of the set of operands through the suppression logic block.

17. A computer-implemented method, wherein each step is executed by a processing core, comprising:

storing a reference value for a composite computation in a memory, wherein a complete execution of the composite computation generates a precise output and requires execution of a set of component computations;

generating a component computation approximation for a component computation in the set of component computations;

evaluating the component computation approximation with the reference value;

executing a partial execution of the composite computation to produce an estimated output, wherein the partial execution of the composite computation uses the component computation approximation; and

suppressing the component computation, while executing the partial execution, based on the evaluation of the component computation approximation with the reference value.

18. The computer-implemented method of claim 17 , wherein:

the composite computation is a sum-of-products computation;

the component computation is a product computation;

the precise output requires execution of the set of component computations at a first resolution;

the component computation approximation is a lower resolution product computation output generated using a lower resolution execution of the product computation; and

the partial execution of the composite computation uses the component computation approximation by summing the lower resolution product computation output in the sum-of-products computation.

19. A computer-implemented method, wherein each step is executed by a processing core, comprising:

storing at least one reference value for a composite computation, wherein a complete execution of the composite computation includes a set of component computations;

generating a set of component computation approximations for the set of component computations;

executing a partial execution of the composite computation to produce an estimated output, wherein the partial execution of the composite computation uses the set of component computation approximations; and

alternatively suppressing and executing the component computations in the set of component computations, while executing the partial execution, based on the at least one reference value and the set of component computation approximations.

20. The computer-implemented method of claim 19 , wherein:

the composite computation is a sum-of-products computation;

the set of component computations is a set of product computations; and

when a component computation is suppressed a corresponding component computation approximation is applied to execute the partial execution.

Assignments (3)
CHANGE OF NAME Recorded Feb 23, 2025
From: TENSTORRENT INC.
To: TENSTORRENT AI INC.
Reel/Frame 070298/0922 →
CHANGE OF NAME Recorded Feb 23, 2025
From: TENSTORRENT AI INC.
To: TENSTORRENT AI ULC
Reel/Frame 070298/0944 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 14, 2018
From: BAJIC, LJUBISA; TRAJKOVIC, MILOS; HAMER, IVAN; GILANI, SYED
To: TENSTORRENT INC.
Reel/Frame 045794/0253 →
Continuity (2)
Provisional Application 62505775 · May 12, 2017
Related Publication 20180329723A1 · Nov 15, 2018
Cited By (1)
US 12,399,713