IP Library Granted Patent US 12,333,270
Granted Patent B2
US 12,333,270 · App. 17/737,925 · Granted Jun 17, 2025

Calculating a floating-point function using multiple lookup tables

Inventors: Mingran Wang (San Jose, CA); Xiaoyan Li (San Jose, CA); Yongning Sheng (San Jose, CA)
Assignee: SambaNova Systems, Inc.
G06F7/483G06F16/9017G06F17/17
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,333,270
App. No.
17/737,925
Granted
Jun 17, 2025
Kind
B2
Abstract

A computation unit includes input lines to provide a floating-point value, a first lookup table, a second lookup table, a range detector, and an output stage. The input lines include exponent lines and mantissa lines. The first lookup table has a first address input coupled to a first subset of the input lines to provide a first output. The second lookup table has a second address input coupled to a second subset of the input lines to provide a second output. The range detector is coupled to at least some of the input lines and indicates whether the floating-point value provided on the input lines is within a specified range on a range output. The output stage is operatively coupled to the first output, the second output and the range output, to generate a function output based on the first output, the second output, and the range output.

Claims (57)

1. A computation unit, comprising:

input lines to provide a floating-point value, the input lines including exponent lines and mantissa lines;

a first lookup table having a first address input coupled to a first subset of the input lines and providing a first output, the first subset of the input lines including a most significant bit of the exponent lines;

a second lookup table having a second address input coupled to a second subset of the input lines and providing a second output, the second subset of the input lines excluding the most significant bit of the exponent lines;

a range detector, coupled to at least some of the input lines, to indicate, on a range output, whether the floating-point value provided on the input lines is within a specified range; and

an output stage, operatively coupled to the first output, the second output and the range output, to generate a function output based on the first output, the second output, and the range output.

2. The computation unit of claim 1 , wherein the second subset of the input lines includes a subset of the exponent lines and a subset of the mantissa lines, and the second subset of the input lines has more of the mantissa lines than the first subset of the input lines.

3. The computation unit of claim 1 , the output stage to select the function output to be either the first output or the second output dependent upon the range output.

4. The computation unit of claim 1 , further comprising:

a third lookup table having a third address input coupled to the first subset of the input lines and providing a third output operatively coupled to the output stage;

an arithmetic circuitry to generate a fourth output based on the input lines, the first output, and the third output; and

a selector to select the function output to be either the fourth output or the second output dependent upon the range output.

5. The computation unit of claim 4 , the arithmetic circuitry to multiply the floating-point value provided on the input lines by a first value provided on the first output and add a second value provided on the third output to generate the fourth output.

6. The computation unit of claim 1 , further comprising a third lookup table having a third address input coupled to a third subset of the input lines and providing a third output operatively coupled to the output stage;

the range detector to further indicate, on the range output, whether the floating-point value provided on the input lines is within a second specified range; and

the output stage to select the function output to be either the first output, the second output, or the third output, dependent upon the range output.

7. The computation unit of claim 6 , wherein the second subset of the input lines and the third subset of the input lines are identical.

8. The computation unit of claim 1 , wherein the first output and the second output each have a number of lines equal to that of the input lines.

9. The computation unit of claim 8 , wherein the number of lines in the first output, the second output, and the input lines are each 32, the first subset of the input lines and the second subset of the input lines have 16 lines each, and the first lookup table and the second lookup table each have 2 16 (32768) entries.

10. The computation unit of claim 1 , wherein both the first subset of the input lines and the second subset of the input lines include a least significant line of the exponent lines and a most significant line of the mantissa lines.

11. The computation unit of claim 1 , the output stage comprising an interpolator.

12. The computation unit of claim 1 , comprising a reconfigurable data processor (RDP) that includes at least one configurable memory unit storing the first lookup table and the second lookup table, and at least one configurable computing unit implementing the range detector and the output stage.

13. A method to calculate a function of a floating-point value using a circuit, the method comprising:

receiving the floating-point value on input lines of the circuit, including exponent lines and mantissa lines;

using a first subset of the input lines, including a most significant bit of the exponent lines, as a first address input into a first lookup table of the circuit to retrieve a first value;

using a second subset of the input lines, excluding the most significant bit of the exponent lines, as a second address input into a second lookup table of the circuit to retrieve a second value;

determining, by the circuit, whether the floating-point value received on the input lines is within a specified range to generate a range output; and

generating a function output of the circuit based on the first value, the second value, and the range output.

14. The method of claim 13 , wherein the second subset of the input lines includes a subset of the exponent lines and a subset of the mantissa lines, and the second subset of the input lines has more of the mantissa lines than the first subset of the input lines.

15. The method of claim 13 , further comprising generating the function output by selecting either the first value or the second value dependent upon the range output.

16. The method of claim 13 , further comprising:

using the first subset of the input lines as a third address input into a third lookup table to retrieve a third value;

generating the function output at least in part by calculating a mathematical function using the floating-point value on the input lines, the first value, and the third value.

17. The method of claim 16 , further comprising generating the function output by selecting either the calculated mathematical function or the second value dependent upon the range output.

18. The method of claim 16 , wherein the mathematical function multiplies the floating-point value on the input lines by the first value and adds the third value.

19. The method of claim 13 , further comprising:

using a third subset of the input lines as a third address input into a third lookup table to retrieve a third value;

determining whether the floating-point value received on the input lines is within a second specified range to generate the range output; and

selecting the function output to be either the first value, the second value, or the third value, dependent upon the range output.

20. The method of claim 19 , wherein the second subset of the input lines and the third subset of the input lines are identical.

21. A computer program product comprising a machine readable, non-transitory data storage medium having stored thereon, one or more configuration files for a reconfigurable data processor (RDP), that when loaded onto the RDP cause one or more configurable units in the RDP to:

receive a floating-point value on input lines including exponent lines and mantissa lines;

use a first subset of the input lines, including a most significant bit of the exponent lines, as a first address input into a first lookup table to retrieve a first value;

use a second subset of the input lines, excluding the most significant bit of the exponent lines, as a second address input into a second lookup table to retrieve a second value;

determine whether the floating-point value received on the input lines is within a specified range to generate a range output; and

generate a function output based on the first value, the second value, and the range output.

22. The computer program product of claim 21 , wherein the one or more configuration files further cause the one or more configurable units in the RDP to generate the function output by selecting either the first value or the second value dependent upon the range output.

23. The computer program product of claim 21 , wherein the one or more configuration files further cause the one or more configurable units in the RDP to:

use the first subset of the input lines as a third address input into a third lookup table to retrieve a third value;

generate the function output at least in part by calculating a mathematical function using the floating-point value on the input lines, the first value, and the third value.

24. The computer program product of claim 23 , wherein the one or more configuration files further cause the one or more configurable units in the RDP to generate the function output by selecting either the calculated mathematical function or the second value dependent upon the range output.

25. The computer program product of claim 23 , wherein the mathematical function multiplies the floating-point value on the input lines by the first value and adds the third value.

26. The computer program product of claim 21 , wherein the one or more configuration files further cause the one or more configurable units in the RDP to:

use a third subset of the input lines as a third address input into a third lookup table to retrieve a third value;

determine whether the floating-point value received on the input lines is within a second specified range to generate the range output; and

select the function output to be either the first value, the second value, or the third value, dependent upon the range output.

27. The computer program product of claim 26 , wherein the second subset of the input lines and the third subset of the input lines are identical.

Assignments (2)
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Apr 18, 2025
From: SAMBANOVA SYSTEMS, INC.
To: SILICON VALLEY BANK, A DIVISION OF FIRST-CITIZENS BANK & TRUST COMPANY, AS AGENT
Reel/Frame 070892/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 6, 2022
From: WANG, MINGRAN; LI, XIAOYAN; SHENG, YONGNING
To: SAMBANOVA SYSTEMS, INC.
Reel/Frame 059842/0272 →
Continuity (2)
Continuation 16590058 · Oct 1, 2019
Related Publication 20220261220A1 · Aug 18, 2022
References Cited (136)
US 5365465A · Larson · 1994 [cited by applicant]
US 5506797A · Koshiba · 1996 [cited by applicant]
US 6256653B1 · Juffa et al. · 2001 [cited by applicant]
US 7015921B1 · Trivedi et al. · 2006 [cited by applicant]
US 7472149B2 · Endo · 2008 [cited by applicant]
US 8184317B2 · Okamoto · 2012 [cited by applicant]
US 9411756B2 · Nogueira et al. · 2016 [cited by applicant]
US 9743073B2 · Huang · 2017 [cited by examiner]
US 10698853B1 · Grohoski et al. · 2020 [cited by applicant]
US 11106430B1 · Kadkol et al. · 2021 [cited by applicant]
US 20030068097A1 · Wilson et al. · 2003 [cited by applicant]
US 20050160129A1 · Endo · 2005 [cited by applicant]
US 20120030203A1 · Chiang et al. · 2012 [cited by applicant]
US 20130151576A1 · Lutz et al. · 2013 [cited by applicant]
US 20130262539A1 · Wegener · 2013 [cited by applicant]
US 20130339564A1 · Nogueira et al. · 2013 [cited by applicant]
US 20140040334A1 · Burgess et al. · 2014 [cited by applicant]
US 20140222883A1 · Pineiro et al. · 2014 [cited by applicant]
US 20170054449A1 · Mani et al. · 2017 [cited by applicant]
US 20170244982A1 · Fuldseth et al. · 2017 [cited by applicant]
US 20170322774A1 · Zhang · 2017 [cited by applicant]
US 20170322805A1 · Zohar et al. · 2017 [cited by applicant]
US 20180157465A1 · Bittner et al. · 2018 [cited by applicant]
US 20180174022A1 · Young · 2018 [cited by applicant]
US 20180220144A1 · Su et al. · 2018 [cited by applicant]
US 20180329681A1 · Zhang et al. · 2018 [cited by applicant]
US 20190042244A1 · Henry et al. · 2019 [cited by applicant]
US 20190042922A1 · Pillai et al. · 2019 [cited by applicant]
US 20190042924A1 · Pasca et al. · 2019 [cited by applicant]
US 20190114139A1 · Zhang · 2019 [cited by applicant]
US 20190114555A1 · Akerib · 2019 [cited by applicant]
US 20190147323A1 · Li et al. · 2019 [cited by applicant]
US 20190205734A1 · Guntoro · 2019 [cited by applicant]
US 20190279075A1 · Liu et al. · 2019 [cited by applicant]
US 20190341052A1 · Allibhai · 2019 [cited by applicant]
US 20200134475A1 · Hill et al. · 2020 [cited by applicant]
US 20200159544A1 · Shah et al. · 2020 [cited by applicant]
US 20200159692A1 · Shah et al. · 2020 [cited by applicant]
US 20200241844A1 · Koeplinger et al. · 2020 [cited by applicant]
US 20200272882A1 · Lo · 2020 [cited by applicant]
US 20200356523A1 · Prabhakar et al. · 2020 [cited by applicant]
US 20200371805A1 · Lutz · 2020 [cited by applicant]
US 20210011770A1 · Prabhakar et al. · 2021 [cited by applicant]
US 20210034982A1 · Sather et al. · 2021 [cited by applicant]
US 20210042259A1 · Koeplinger et al. · 2021 [cited by applicant]
US 20210064341A1 · Kuo et al. · 2021 [cited by applicant]
US 20210064372A1 · Sun et al. · 2021 [cited by applicant]
US 20210064568A1 · Wang et al. · 2021 [cited by applicant]
US 20210072955A1 · Mellempudi et al. · 2021 [cited by applicant]
US 20210081691A1 · Chen et al. · 2021 [cited by applicant]
US 20210081769A1 · Chen et al. · 2021 [cited by applicant]
US 20210096816A1 · Wang et al. · 2021 [cited by applicant]
US 20210149634A1 · Wang et al. · 2021 [cited by applicant]
US 20210157550A1 · Wang et al. · 2021 [cited by applicant]
US 20210182021A1 · Wang et al. · 2021 [cited by applicant]
US 20210216873A1 · Liu · 2021 [cited by applicant]
CN 109871235A · 2019 [cited by applicant]
CN 110471643A · 2019 [cited by applicant]
TW I318337B · 2009 [cited by applicant]
TW I557641B · 2016 [cited by applicant]
TW 201935329A · 2019 [cited by applicant]
WO 2008036944A1 · 2008 [cited by applicant]
WO 2010142987A1 · 2010 [cited by applicant]
WO 2018100920A1 · 2018 [cited by applicant]
WO 2021046274A1 · 2021 [cited by applicant]
WO 2021067318A1 · 2021 [cited by applicant]
WO 2021108328A1 · 2021 [cited by applicant]
WO 2021126530A1 · 2021 [cited by applicant]
Anonymous, Activation Function, Wikipedia, Retrieved on Dec. 3, 2021, 5 pages. Retrieved from [ URL: https://en.wikipedia.org/wiki/Activation_function ]. [cited by applicant]
Anonymous: “Rounding to 0.5”, Arduino Forum, Aug. 28, 2017, 4 pages. [cited by applicant]
Basterretxea et al., “Approximation of sigmoid function and the derivative for hardware implementation of artificial neurons,” IEE Proceedings—Circuits, Devices and Systems, vol. 151, Issue 1, Feb. 5, 2004, 7 pages. [cited by applicant]
Bendersky “The Softmax function and its derivative,” https://eli.thegreenplace.net/2016/the-softmax-function-and-its-derivative, Oct. 18, 2016, 11 pages. [cited by applicant]
Cook, Comparing bfloat16 range and precision to other 16-bit numbers, dated Nov. 15, 2018, 8 pages. Retrieved on Dec. 3, 2021. Retrieved from the internet [URL: www.johndcook.com/blog/2018/11/15/bfloat16 ]. [cited by applicant]
Eppler et al. , “High speed neural network chip for trigger purposes in high energy physics,” IEEE, Proc. of the conference on design, automation and test in Europe, Feb. 1998, 8 pages. [cited by applicant]
Gomar et al. “Precise digital implementations of hyperbolic tanh and sigmoid function,” 2016 50th Asilomar Conference on Signals, Systems and Computers, Nov. 6-9, 2016, 4 pages. [cited by applicant]
Intel BLOAT16—Hardware Numerics Definition White Paper, Rev. 1.0, Nov. 2018, 7 pages. [cited by applicant]
Ioffe, et al., “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift,” Cornell University, available at https://arxiv.org/abs/1502.03167, Mar. 2, 2015, 11 pages. [cited by applicant]
Iqbal, “Reconfigurable Processor Artchitecture for High Speed Applications,” IEEE Int'l Advance Computing Conference (IACC 2009), Patiala, India, Mar. 6-7, 2009, 6 pages. [cited by applicant]
Koeplinger et. al., Spatial: A Language and Compiler for Application Accelerators, Proceedings of the 39th ACM SIGPLAN Conference On Programming Language Design And Embodiment (PLDI), Proceedings of the 43rd Internation… [cited by applicant]
Lin et al., “A Digital Circuit Design of Hyperbolic Tangent Sigmoid Function for Neural Networks,” 2018 IEEE Int'l Symp. on Circuits and Systems, May 18-21, 2018, 4 pages. [cited by applicant]
M. Emani et al., “Accelerating Scientific Applications With SambaNova Reconfigurable Dataflow Architecture,” in Computing in Science & Engineering, vol. 23, No. 2, pp. 114-119, Mar. 1-Apr. 2021, doi: 10.1109/MCSE.2021.3… [cited by applicant]
MISB ST 1201.4, “Floating Point to Integer Mapping,” Feb. 28, 2019, pp. 1-21. [cited by applicant]
Olukotun, “Designing Computer Sytems for Software 2.0,” ISCA 2018 keynote, Jun. 2018, 49 pages. [cited by applicant]
Pasupuleti-etal, Low Complex High Accuracy Computation Approximations to Enable On device RNN Applications, May 26, 2019, 5 pages. [cited by applicant]
PCT/US2020/053344—International Search Report and Written Opinion dated Dec. 21, 2020, 11 pages. [cited by applicant]
PCT/US2020/061060—International Search Report and Written Opinion dated Mar. 4, 2021, 14 pages. [cited by applicant]
PCT/US2020/061862—International Search Report and Written Opinion dated Mar. 10, 2021, 11 pages. [cited by applicant]
PCT/US2020/062905—International Search Report and Written Opinion dated Mar. 22, 2021, 14 pages. [cited by applicant]
PCT/US2020/49285—International Search Report and Written Opinion dated Nov. 18, 2020, 13 pages. [cited by applicant]
Petersen, “Softmax with cross-entropy,” https://mattpetersen.github.io/softmax-with-cross-entropy, Jun. 25, 2017, 19 pages. [cited by applicant]
Podobas et al, A Survey on Coarse-Grained Reconfigurable Architectures From a Performance Perspective, IEEEAccess, vol. 2020.3012084, Jul. 27, 2020, 25 pages. [cited by applicant]
Prabhakar et. al., “Plasticine: A Reconfigurable Architecture for Parallel Patterns”, ISCA '17, Jun. 24-28, 2017, Toronto, ON, Canada. [cited by applicant]
Turkson et al. “Artificial neural network applications in the calibration of spark-ignition engines: An overview,” Engineering Science and Technology, an International Journal, vol. 19, Issue 3, Sep. 2016, 1346-1359. [cited by applicant]
TW 109129892—First Office Action dated Aug. 12, 2021, 17 pages. [cited by applicant]
TW 109129892—Response to First Office Action dated Oct. 21, 2021, 17 pages. [cited by applicant]
TW 109129892 Notice of Allowance, dated Nov. 1, 2021, 3 pages. [cited by applicant]
TW 109134270—First Office Action dated Sep. 30, 2021, 11 pages. [cited by applicant]
U.S. Appl. No. 16/590,058—Office Action dated Jun. 1, 2021, 9 pages. [cited by applicant]
U.S. Appl. No. 16/590,058 Notice of Allowance, dated Nov. 3, 2021, 15 pages. [cited by applicant]
U.S. Appl. No. 16/688,069—Notice of Allowance dated Sep. 22, 2021, 26 pages. [cited by applicant]
U.S. Appl. No. 16/688,069—Response to Non-Final Office Action dated Jun. 3, 2021, filed Sep. 3, 2021, 9 pages. [cited by applicant]
U.S. Appl. No. 16/695,138—Non-Final Office Action dated Apr. 22, 2021, 11 pages. [cited by applicant]
U.S. Appl. No. 16/695,138—Response to Non-Final Office Action dated Apr. 22, 2021 filed Jul. 22, 2022, 8 pages. [cited by applicant]
U.S. Appl. No. 16/695,138 Final Office Action, dated Oct. 21, 2021, 12 pages. [cited by applicant]
U.S. Appl. No. 16/718,094 Notice of Allowance, dated Jun. 14, 2021, 15 pages. [cited by applicant]
U.S. Appl. No. 16/688,069—Response to Non-Final Office Action dated Jan. 29, 2021, filed Mar. 31, 2021, 20 pages. [cited by applicant]
US Office Action in U.S. Appl. No. 16/688,069 dated Jun. 3, 2021, 9 pages. [cited by applicant]
US Office Action in U.S. Appl. No. 16/688,069 dated Jan. 29, 2021, 18 pages. [cited by applicant]
U.S. Appl. No. 16/590,058—Response to Non-Final Office Action dated Jun. 1, 2021 as filed Aug. 18, 2021, 20 pages. [cited by applicant]
Wikipedia “Batch normalization,” [retrieved -on Feb. 25, 2021] Retrieved from the Internet: <URL: https://en.wikipedia.org/w/index.php?title=Batch normalization&oldid=908065963>, 10 pages. [cited by applicant]
Wikipedia “Floor and ceiling functions,” downloaded Aug. 12, 2019, 5 pages. [cited by applicant]
Wikipedia, bfloat16 floating-point format, 5 pages, retreived on Dec. 3, 2021. Retrieved from the internet [URL: https://en.wikipedia.org/wiki/Bfloat16_floating-point_format ]. [cited by applicant]
U.S. Appl. No. 16/688,069—Response to Non-Final Office Action dated Jan. 29, 2021, filed Mar. 30, 2021, 20 pages. [cited by applicant]
Kumar etal, Low Complex & High Accuracy Computation Approximations to Enable On-Device RNN Applications, May 26, 2019, 5 pages. [cited by applicant]
Kumar et al, “Low Complex & High Accuracy Computation Approximations to Enable On-Device RNN Applications”, 2019 IEEE International Symposium On Circuits and Systems (ISCAS), IEE, May 26, 2019 (May 26, 2019), pp. 1-5. [cited by applicant]
TW 109134270—First Office Action dated Sep. 30, 2021, 6 pages. [cited by applicant]
U.S. Appl. No. 16/695,138—Response to Final Office Action dated Oct. 21, 2021, filed Jan. 13, 2022, 10 pages. [cited by applicant]
TW 109140964 First Office Action, dated Jan. 12, 2022, 10 pages. [cited by applicant]
U.S. Appl. No. 16/688,069 Notice of Allowance, dated Jan. 14, 2022, 22 pages. [cited by applicant]
TW 109140965 First Office Action, dated Dec. 21, 2021, 8 pages. [cited by applicant]
TW 109134270—Response to First Office Action dated Sep. 30, 2021, filed Dec. 29, 2021, 13 pages. [cited by applicant]
TW 109139904 First Office Action, dated Nov. 22, 2021, 15 pages. [cited by applicant]
U.S. Appl. No. 16/695,138 Notice of Allowance, dated Feb. 10, 2022, 17 pages. [cited by applicant]
U.S. Appl. No. 16/560,057 Notice of Allowance, dated Feb. 9, 2022, 46 pages. [cited by applicant]
U.S. Appl. No. 16/590,058 Notice of Allowance, dated Feb. 18, 2022, 10 pages. [cited by applicant]
TW 109139904—Response to First Office Action dated Nov. 22, 2021, filed Feb. 23, 2022, 58 pages. [cited by applicant]
TW 109140965—Response to First Office Action dated Dec. 21, 2021, filed Mar. 18, 2022, 108 pages. [cited by applicant]
PCT/US2020/049285—International Preliminary Report on Patentability, dated Mar. 17, 2022, 8 pages. [cited by applicant]
TW 109140964—Response to First Office Action dated Jan. 12, 2022, filed Apr. 13, 2022, 67 pages. [cited by applicant]
TW 109134270—Notice of Allowance, dated Jan. 27, 2022, 3 pages. [cited by applicant]
TW 109140965—Notice of Allowance with translation dated Apr. 20, 2022, 3 pages. [cited by applicant]
TW 109139904—Notice of Allowance with translation dated Aug. 17, 2022, 3 pages. [cited by applicant]
PCT/US2020/053344—International Preliminary Report on Patentability, dated Apr. 14, 2022, 8 pages. [cited by applicant]
PCT/US2020/061862—International Preliminary Report on Patentability, dated Jun. 9, 2022, 8 pages. [cited by applicant]
PCT/US2020/061060—International Preliminary Report on Patentability, dated Jun. 2, 2022, 8 pages. [cited by applicant]
PCT/US2020/062905—International Preliminary Report on Patentability, dated Jun. 30, 2022, 8 pages. [cited by applicant]