IP Library Granted Patent US 10,585,679
Granted Patent B2
US 10,585,679 · App. 16/416,749 · Granted Mar 10, 2020

Processing core with operation suppression based on contribution estimate

Inventors: Ljubisa Bajic (Toronto, CA); Milos Trajkovic (Toronto, CA); Ivan Hamer (Toronto, CA); Syed Gilani (Markham, CA)
Assignee: Tenstorrent Inc.
G06F9/4494G06F7/50G06F7/523G06F15/76G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,585,679
App. No.
16/416,749
Granted
Mar 10, 2020
Kind
B2
Abstract

Processing cores with the ability to suppress operations based on a contribution estimate for those operation for purposes of increasing the overall performance of the core are disclosed. Associated methods that can be conducted by such processing cores are also disclosed. One such method includes generating a reference value for a composite computation. A complete execution of the composite computation generates a precise output and requires execution of a set of component computations. The method also includes generating a component computation approximation. The method also includes evaluating the component computation approximation with the reference value. The method also includes executing a partial execution of the composite computation using the component computation approximation to produce an estimated output. The method also includes suppressing the component computation, while executing the partial execution, based on the evaluation of the component computation approximation with the reference value.

Claims (95)

1. A computer-implemented method, wherein each step is executed by a processing core, comprising:

generating a reference value for a composite computation, wherein a complete execution of the composite computation generates a precise output and requires execution of a set of component computations;

generating a component computation approximation for a component computation in the set of component computations;

evaluating the component computation approximation with the reference value;

executing a partial execution of the composite computation to produce an estimated output; and

suppressing the component computation, while executing the partial execution, based on the evaluation of the component computation approximation with the reference value.

2. The computer-implemented method of claim 1 , further comprising:

generating a second component computation approximation for a second component computation in the set of component computations;

evaluating the second component computation approximation; and

executing the second component computation, while executing the partial execution, conditionally based on the evaluation of the second component computation approximation.

3. The computer-implemented method of claim 1 , wherein:

the composite computation is a sum-of-products computation;

the component computation is a product computation;

the precise output requires execution of the set of component computations at a first resolution; and

the component computation approximation is generated using a lower resolution execution of the product computation.

4. The computer-implemented method of claim 3 , wherein generating the reference value includes:

generating a set of component computation approximations for a subset of the set of component computations; and

summing the set of component computation approximations to generate a sum.

5. The computer-implemented method of claim 1 , wherein:

the composite computation is a multi-product computation between a set of input data and a first filter segment from a first filter; and

the component computation is a product computation conducted with: (i) a filter entry from the first filter segment; and (ii) an entry from the set of input data.

6. The computer-implemented method of claim 5 , wherein generating the reference value includes:

generating a second component computation approximation using the set of input data.

7. The computer-implemented method of claim 6 , wherein:

generating the reference value includes selecting a maximum exponent of the second component computation approximation;

evaluating the component computation approximation with the reference value includes comparing a maximum exponent of the component computation approximation with the maximum exponent of the second component computation approximation; and

suppressing the component computation when the maximum exponent of the component computation approximation is smaller than the maximum exponent of the second component computation approximation.

8. The computer-implemented method of claim 1 , further comprising:

comparing the component computation approximation to the reference value; and

wherein the generating the reference value step uses a programmable tolerance.

9. The computer-implemented method of claim 1 , wherein:

the method consumes a first amount of computational resources;

the complete execution of the composite computation consumes a second amount of computational resources; and

the first amount is less than the second amount.

10. The computer-implemented method of claim 1 , wherein:

the composite computation is a sum-of-products computation; and

the set of component computations includes a set of product computations.

11. The computer-implemented method of claim 10 , wherein:

the component computation is a multiplication computation with a first floating point input and a second floating point input; and

the generating of the component computation approximation step includes adding an exponent of the first floating point input with an exponent of the second floating point input to generate an exponent sum.

12. The computer-implemented method of claim 11 , wherein:

the exponent sum is generated by an adder; and

the adder has less than 9-bit precision.

13. The computer-implemented method of claim 10 , wherein generating the reference value for the composite computation comprises:

continuously updating the reference value while executing the partial execution using a current accumulated value of the sum-of-products computation and a scaling percentage of the current accumulated value; and

wherein the scaling percentage increases as the partial execution is conducted.

14. The computer-implemented method of claim 1 , wherein generating the reference value for the composite computation comprises:

generating a set of component computation approximations;

selecting a largest component computation approximation in the set of component computation approximations; and

wherein the reference value is a fraction of the largest component computation approximation.

15. The computer-implemented method of claim 1 , wherein generating the reference value for the composite computation comprises:

generating a set of component computation approximations;

summing the set of component computation approximations to generate a sum; and

wherein the reference value is a fraction of the sum.

16. The computer-implemented method of claim 1 , wherein:

a set of operands for the composite computation are located in a register file;

the suppressing of the component computation step is conducted by a suppression logic block; and

the suppressing of the component computation step includes:

inhibiting a first subset of the set of operands from being passed through the suppression logic block; and

transmitting a second subset of the set of operands through the suppression logic block.

17. A computer-implemented method, wherein each step is executed by a processing core, comprising:

storing a reference value for a composite computation in a memory, wherein a complete execution of the composite computation generates a precise output and requires execution of a set of component computations;

generating a component computation approximation for a component computation in the set of component computations;

evaluating the component computation approximation with the reference value;

executing a partial execution of the composite computation to produce an estimated output; and

suppressing the component computation, while executing the partial execution, based on the evaluation of the component computation approximation with the reference value.

18. The computer-implemented method of claim 17 , wherein:

the composite computation is a sum-of-products computation;

the component computation is a product computation;

the precise output requires execution of the set of component computations at a first resolution;

the component computation approximation is a lower resolution product computation output generated using a lower resolution execution of the product computation; and

the partial execution of the composite computation uses the component computation approximation by summing the lower resolution product computation output in the sum-of-products computation.

19. The computer-implemented method of claim 17 , further comprising:

the composite computation is a sum-of-products computation; and

generating the reference value for the composite computation comprises:

continuously updating the reference value while executing the partial execution using a current accumulated value of the sum-of-products computation and a scaling percentage of the current accumulated value; and

wherein the scaling percentage increases as the partial execution is conducted.

20. The computer-implemented method of claim 17 , wherein:

the memory is a ROM memory.

21. The computer-implemented method of claim 17 , wherein:

generating the component computation approximation for the component computation in the set of component computations comprises evaluating a set of operands to the component computation to determine if any of the operands in the set of operands are highly influential to the component computation; and

suppressing the component computation comprises obtaining a replacement value from memory.

22. A computer-implemented method, wherein each step is executed by a processing core, comprising:

storing at least one reference value for a composite computation, wherein a complete execution of the composite computation includes a set of component computations;

generating a set of component computation approximations for the set of component computations;

executing a partial execution of the composite computation to produce an estimated output, wherein the partial execution of the composite computation uses the set of component computation approximations; and

alternatively suppressing and executing the component computations in the set of component computations, while executing the partial execution, based on the at least one reference value and the set of component computation approximations.

23. The computer-implemented method of claim 22 , wherein:

the composite computation is a sum-of-products computation;

the set of component computations is a set of product computations; and

when a component computation is suppressed a corresponding component computation approximation is applied to execute the partial execution.

24. The computer-implemented method of claim 22 , wherein:

generating the set of component computation approximations includes evaluating a set of exponents of a set of operands used in the set of component computations.

25. The computer-implemented method of claim 22 , wherein:

generating the set of component computation approximations includes evaluating a number of leading zeros of a set of operands used in the set of component computations.

Assignments (3)
CHANGE OF NAME Recorded Feb 23, 2025
From: TENSTORRENT INC.
To: TENSTORRENT AI INC.
Reel/Frame 070298/0922 →
CHANGE OF NAME Recorded Feb 23, 2025
From: TENSTORRENT AI INC.
To: TENSTORRENT AI ULC
Reel/Frame 070298/0944 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 20, 2019
From: BAJIC, LJUBISA; TRAJKOVIC, MILOS; HAMER, IVAN; GILANI, SYED
To: TENSTORRENT INC.
Reel/Frame 049230/0929 →
Continuity (3)
Continuation 15975930 · May 10, 2018
Provisional Application 62505775 · May 12, 2017
Related Publication 20190272183A1 · Sep 5, 2019