IP Library Granted Patent US 12,260,321
Granted Patent B2
US 12,260,321 · App. 17/385,316 · Granted Mar 25, 2025

Data feature augmentation system and method for low-precision neural network

Inventors: Fu-Cheng Tsai (Tainan, TW); Yi-Ching Kuo (Tainan, TW); Chih-Sheng Lin (Tainan, TW); Shyh-Shyuan Sheu (Zhubei, TW); Tay-Jyi Lin (Kaohsiung, TW); Shih-Chieh Chang (Hsinchu, TW)
Assignee: INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE
G06N3/063G06N3/04H03M1/1245G11C27/02H03M1/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,260,321
App. No.
17/385,316
Granted
Mar 25, 2025
Kind
B2
Abstract

A data feature augmentation system and method for a low-precision neural network are provided. The data feature augmentation system includes a first time difference unit. The first time difference unit includes a first sample-and-hold circuit and a subtractor. The first sample-and-hold circuit is used for receiving an input signal and obtaining a first signal according to the input signal. The first signal is related to a first leakage rate of the first sample-and-hold circuit and the first signal is the signal generated by delaying the input signal by one time unit. The subtractor is used for performing subtraction on the input signal and the first signal to obtain a time difference signal. The input signal and the time difference signal are inputted to the low-precision neural network.

Claims (37)

1. A data feature augmentation system for a low-precision neural network, comprising:

a first time difference unit, comprising:

a first sample-and-hold circuit, for receiving an input signal and obtaining a first signal according to the input signal, wherein the first signal is related to a first leakage rate of the first sample-and-hold circuit and the first signal is the signal generated by delaying the input signal by one time unit; and

a subtractor, for performing subtraction on the input signal and the first signal to obtain a time difference signal;

wherein the input signal and the time difference signal are inputted to the low-precision neural network.

2. The data feature augmentation system according to claim 1 , wherein the input signal, the first signal and the time difference signal are analog signals, and the data feature augmentation system further comprises:

a first analog-to-digital converter; and

a second analog-to-digital converter;

wherein the first analog-to-digital converter and the second analog-to-digital converter are low-resolution analog-to-digital converters, and before the input signal and the time difference signal are inputted to the low-precision neural network, the input signal and the time difference signal are respectively inputted to the first analog-to-digital converter and the second analog-to-digital converter.

3. The data feature augmentation system according to claim 1 , wherein the low-precision neural network is a binarized neural network or a short-wordlength neural network.

4. The data feature augmentation system according to claim 1 , wherein the first time difference unit further comprises:

a second sample-and-hold circuit, for receiving the input signal and obtaining a second signal according to the input signal, wherein the second signal is related to a second leakage rate of the second sample-and-hold circuit and the second signal is the signal generated by delaying the input signal by two time units;

wherein the subtractor is further used for subtracting the first signal and the second signal from the input signal to obtain the time difference signal.

5. The data feature augmentation system according to claim 4 , wherein the first sample-and-hold circuit and the second sample-and-hold circuit are connected in series.

6. The data feature augmentation system according to claim 4 , wherein the first sample-and-hold circuit and the second sample-and-hold circuit are connected in parallel.

7. The data feature augmentation system according to claim 4 , wherein the first leakage rate and the second leakage rate are adjustable.

8. The data feature augmentation system according to claim 1 , wherein the first time difference unit further comprises:

a second sample-and-hold circuit, for receiving the input signal and obtaining a second signal according to the input signal, wherein the second signal is related to a second leakage rate of the second sample-and-hold circuit and the second signal is the signal generated by delaying the input signal by two time units;

wherein the subtractor is further used for subtracting the second signal from the input signal to obtain the time difference signal.

9. The data feature augmentation system according to claim 8 , wherein the first sample-and-hold circuit and the second sample-and-hold circuit are connected in series.

10. The data feature augmentation system according to claim 8 , wherein the first sample-and-hold circuit and the second sample-and-hold circuit are connected in parallel.

11. The data feature augmentation system according to claim 1 , further comprising:

a second time difference unit, serially connected to the first time difference unit for obtaining a second time difference signal according to the time difference signal.

12. A data feature augmentation method for a low-precision neural network, comprising:

receiving an input signal and obtaining a first signal according to the input signal, wherein the first signal is related to a first leakage rate of a first sample-and-hold circuit and the first signal is the signal generated by delaying the input signal by one time unit; and

performing subtraction on the input signal and the first signal to obtain a time difference signal;

wherein the input signal and the time difference signal are inputted to the low-precision neural network.

13. The data feature augmentation method according to claim 12 , wherein the input signal, the first signal and the time difference signal are analog signals and the data feature augmentation method further comprises:

before the input signal and the time difference signal are inputted to the low-precision neural network, respectively inputting the input signal and the time difference signal to a first analog-to-digital converter and a second analog-to-digital converter, wherein the first analog-to-digital converter and the second analog-to-digital converter are low-resolution analog-to-digital converters.

14. The data feature augmentation method according to claim 12 , wherein the low-precision neural network is a binarized neural network or a short-wordlength neural network.

15. The data feature augmentation method according to claim 12 , further comprising:

receiving the input signal and obtaining a second signal according to the input signal, wherein the second signal is related to a second leakage rate of a second sample-and-hold circuit and the second signal is the signal generated by delaying the input signal by two time units; and

subtracting the first signal and the second signal from the input signal to obtain the time difference signal.

16. The data feature augmentation method according to claim 15 , wherein the first leakage rate and the second leakage rate are adjustable.

17. The data feature augmentation method according to claim 15 , further comprising:

receiving the input signal and obtaining a second signal according to the input signal, wherein the second signal is related to a second leakage rate of the second sample-and-hold circuit and the second signal is the signal generated by delaying the input signal by two time units; and

subtracting the second signal from the input signal to obtain the time difference signal.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 27, 2021
From: TSAI, FU-CHENG; KUO, YI-CHING; LIN, CHIH-SHENG; SHEU, SHYH-SHYUAN; LIN, TAY-JYI; CHANG, SHIH-CHIEH
To: INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE
Reel/Frame 056992/0436 →
Priority Claims (1)
TW 110111849 · Mar 31, 2021 · national
Continuity (1)
Related Publication 20220318605A1 · Oct 6, 2022
References Cited (23)
US 4398179A · Kaneko · 1983 [cited by examiner]
US 8423360B2 · Koshiba et al. · 2013 [cited by applicant]
US 9721559B2 · Cui et al. · 2017 [cited by applicant]
US 10546217B2 · Albright et al. · 2020 [cited by applicant]
US 11101810B1 · Veldhoven · 2021 [cited by examiner]
US 20150042496A1 · Straeussnigg · 2015 [cited by examiner]
US 20150161994A1 · Tang et al. · 2015 [cited by applicant]
US 20170110142A1 · Fan et al. · 2017 [cited by applicant]
US 20170200092A1 · Kisilev · 2017 [cited by applicant]
US 20190043482A1 · Li et al. · 2019 [cited by applicant]
CN 107729953A · 2018 [cited by applicant]
CN 108694442A · 2018 [cited by applicant]
CN 110335584A · 2019 [cited by applicant]
CN 110827837A · 2020 [cited by applicant]
TW 201833819A · 2018 [cited by applicant]
TW 201843998A · 2018 [cited by applicant]
TW 202025172A · 2020 [cited by applicant]
Hubara et al., “Binarized Neural Networks,” 30th Conference on Neural Information Processing Systems Barcelona, Spain, 2016, pp. 1-9. [cited by applicant]
Jiang et al., “XNOR-SRAM: In-Memory Computing SRAM Macro for Binary/Ternary Deep Neural Networks,” Symposium on VLSI Technology Digest of Technical Papers, 2018, pp. 173-174. [cited by applicant]
Kim et al., “In-Memory Batch-Normalization for Resistive Memory based Binary Neural Network Hardware,” ASPDAC, Association for Computing Machinery, 2019, pp. 1-6. [cited by applicant]
Rastegari et al., “XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks,” XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks, 2016, pp. 1-17. [cited by applicant]
Shafiee et al., “ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars,” ACM/IEEE 43rd Annual International Symposium on Computer Architecture, 2016, pp. 14-26. [cited by applicant]
Taiwanese Office Action and Search Report for Taiwanese Application No. 110111849, dated Dec. 7, 2021. [cited by applicant]