IP Library › Granted Patent US 12,367,385
Granted Patent B2
US 12,367,385 · App. 17/384,306 · Granted Jul 22, 2025

Unsupervised learning of memristor crossbar neuromorphic processing systems

Inventors: Md Shahanur Alam (Dayton, OH); Chris Yakopcic (Dayton, OH); Tarek M. Taha (Centerville, OH)
Assignee: UNIVERSITY OF DAYTON
G06N3/065G06N3/088G11C13/0002
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,367,385
App. No.
17/384,306
Granted
Jul 22, 2025
Kind
B2
Abstract

An analog neuromorphic circuit is disclosed having a first and a second memristor crossbar configuration implemented into an autoencoder. The first memristor crossbar configuration includes resistive memories that provide resistance values to each corresponding input voltage applied to the first memristor crossbar configuration to generate first output voltages that are compressed from the input voltages. The second memristor crossbar includes resistive memories that provide resistance values to each corresponding first output voltage applied to the second memristor crossbar configuration to generate second output voltages that are decompressed from the first output voltages. A controller compares the second output voltages to the input voltages to determine if the second output voltages are within a threshold of the input voltages. The controller generates an alert when the second output voltages exceed the threshold from the input voltages thereby indicating that input data associated with the input voltages has not been previously identified.

Claims (56)

1. An analog neuromorphic circuit that implements a plurality of resistive memories, comprising:

a first memristor crossbar configuration that includes a first plurality of resistive memories that is configured to provide a first plurality of resistance values to each corresponding input voltage from a plurality of input voltages applied to the first memristor crossbar configuration to generate a first plurality of output voltages, wherein the first plurality of output voltages is compressed from the plurality of input voltages;

a second memristor crossbar configuration that includes a second plurality of resistive memories that is configured to provide a second plurality of resistance values to each corresponding output voltage from the first plurality of output voltages generated from the first memristor crossbar configuration and applied to the second memristor crossbar configuration to generate a second plurality of output voltages, wherein the second plurality of output voltages is decompressed from the first plurality of output voltages; and

a controller configured to:

compare the second plurality of output voltages to the plurality of input voltages to determine if the second plurality of output voltages is within a threshold of the plurality of input voltages, and

generate an alert when the second plurality of output voltages exceeds the threshold from the plurality of input voltages thereby indicating that input data associated with the plurality of input voltages has not been previously input into the analog neuromorphic circuit.

2. The analog neuromorphic circuit of claim 1 , wherein each resistance value from the first plurality of resistance values is mapped to a corresponding weighted value that is a non-binary value included in a first weighted matrix and each resistance value from the second plurality of resistance values is mapped to a corresponding weighted from a corresponding weighted value that is a non-binary value included in a second weighted matrix.

3. The analog neuromorphic circuit of claim 2 , wherein the controller is further configured to:

compare the second plurality of output voltages to the plurality of input voltages after the first weighted matrix is applied to the first memristor crossbar configuration in compressing the plurality of input voltages and the second weighted matrix is applied to the second memristor crossbar configuration in decompressing the second plurality of output voltages from the first plurality of output voltages, wherein the first weighted matrix is identical to the second weighted matrix; and

generate the alert when the second plurality of output voltages exceeds the threshold from the plurality of input voltages after the first weighted matrix is applied to the first memristor crossbar configuration and the second weighted matrix is applied to the second memristor crossbar configuration thereby indicating that the application of the first weighted matrix and the second weighted matrix that are identical failed to generate the second plurality of output voltages that are within the threshold of the plurality of input voltages.

4. The analog neuromorphic circuit of claim 1 , wherein each first output voltage value associated with the first plurality of output voltages is generated from a dot product operation conducted by the first memristor crossbar configuration and each second output voltage value associated with the second plurality of output voltages is generated from a dot product operation conducted by the second memristor crossbar configuration.

5. The analog neuromorphic circuit of claim 1 , wherein the first memristor crossbar configuration that includes the first plurality of resistive memories that is further configured to:

provide the first plurality of resistance values to each corresponding output voltage from the first plurality of output voltages that is compressed from the plurality of input voltages that is applied to the first memristor crossbar configuration to generate the second plurality of output voltages that is decompressed from the first plurality of output voltages, wherein the compression of the plurality of input voltages and the decompression of the first plurality of output voltages to generate the second plurality of output voltages is conducted by a single memristor crossbar configuration.

6. The analog neuromorphic circuit of claim 1 , further comprising:

a second analog neuromorphic circuit that includes a third plurality of resistive memories that is configured to provide a third plurality of resistance values to each corresponding input voltage from the plurality of input voltages when the second plurality of output voltages exceeds the threshold from the plurality of input voltages to train the second analog neuromorphic circuit to identify the input data associated with the plurality of input voltages.

7. The analog neuromorphic circuit of claim 6 , wherein the controller is further configured to:

provide the plurality of input voltages to the second analog neuromorphic circuit when the second plurality of output voltages exceeds the threshold from the plurality of input voltages thereby indicating that the plurality of input voltages has not been previously identified and the training of the second analog neuromorphic circuit to identify the input data associated with the plurality of input voltages is required.

8. The analog neuromorphic circuit of claim 7 , wherein the plurality of input voltages is converted to an input eigenvector, the first plurality of output voltages is converted to a first output eigenvector, and the second plurality of output voltages is converted to a second output eigenvector.

9. The analog neuromorphic circuit of claim 8 , wherein the controller is further configured to:

compare the second output eigenvector to the input eigenvector to determine if a difference between the second output eigenvector and the input eigenvector deviates beyond an error threshold, wherein the deviation beyond the error threshold by the second output eigenvector and the input eigenvector is indicative that the input data associated with the input voltages has not been previously identified; and

generate the alert when the difference between the second output eigenvector and the first input eigenvector exceeds the error threshold.

10. The analog neuromorphic circuit of claim 9 , wherein the controller is further configured to:

square each difference between each output voltage value that corresponds to each second output voltage from the second plurality of output voltages and each corresponding input voltage value that corresponds to each input voltage to generate a square rooted difference between each output voltage value and each corresponding input voltage value;

square root a sum of each corresponding square rooted difference between each output voltage value and each corresponding input voltage to generate a root squared error for the analog neuromorphic circuit;

compare the root squared error to a standard deviation as the error threshold, wherein the standard deviation is previously determined as the error threshold that when the root squared error exceeds the standard deviation is indicative that the input data associated with the input voltages has not been previously identified; and

provide the plurality of input voltages to the second analog neuromorphic circuit when the root squared error exceeds the standard deviation of the error threshold thereby indicating that the plurality of input voltages has not been previously identified and the training of the second analog neuromorphic circuit to identify the input data associated with the plurality of input voltages is required.

11. A method for implementing a plurality of resistive memories into an autoencoder to compress and then decompress input data to determine if the input data has been previously identified, comprising:

providing a first plurality of resistance values by a first plurality of resistive memories to each corresponding input voltage from a plurality of input voltages applied to a first memristor crossbar configuration generating a first plurality of output voltages, wherein the first plurality of output voltages is compressed from the plurality of input voltages;

providing a second plurality of resistance values by a second plurality of resistive memories to each corresponding output voltage from the first plurality of output voltages generated from the first memristor crossbar configuration and applied to the second memristor crossbar configuration to generate a second plurality of output voltages, wherein the second plurality of output voltages is decompressed from the first plurality of output voltages;

comparing the second plurality of output voltages to the plurality of input voltages to determine if the second plurality of output voltages is within a threshold of the plurality of input voltages; and

generate an alert when the second plurality of output voltages exceeds the threshold from the plurality of input voltages thereby indicating that input data associated with the plurality of input voltages has not been previously input into the analog neuromorphic circuit.

12. The method of claim 11 , further comprising:

mapping each resistance value from the first plurality of resistance values to a corresponding weighted value that is a non-binary value included in a first weighted matrix; and

mapping each resistance value from the second plurality of resistance values to a corresponding weighted value that is a non-binary value included in a second weighted matrix.

13. The method of claim 12 , further comprising:

comparing the second plurality of output voltages to the plurality of input voltages after the first weighted matrix is applied to the first memristor crossbar configuration in compressing the plurality of input voltages and the second weighted matrix is applied to the second memristor crossbar configuration in decompressing the second plurality of output voltages from the first plurality of output voltages, wherein the first weighted matrix is identical to the second weighted matrix; and

generating the alert when the second plurality of output voltages exceeds the threshold from the plurality of input voltages after the first weighted matrix is applied to the first memristor crossbar configuration and the second weighted matrix is applied to the second memristor crossbar configuration thereby indicating that the application of the first weighted matrix and the second weighted matrix that are identical failed to generate the second plurality of output voltages that are within the threshold of the plurality of input voltages.

14. The method of claim 11 , further comprising:

generating each first output voltage value associated with the first plurality of output voltages from a dot product operation conducted by the first memristor crossbar configuration; and

generating each second output voltage value associated with the second plurality of output voltages from a dot product operation conducted by the second memristor crossbar configuration.

15. The method of claim 11 , further comprising:

providing the first plurality of resistance values to each corresponding output voltage from the first plurality of output voltages that is compressed from the plurality of input voltages that is applied to the first memristor crossbar configuration to generate the second plurality of output voltages that is decompressed from the first plurality of output voltages, wherein the compression of the plurality of input voltages and the decompression of the first plurality of output voltages to generate the second plurality of output voltages is conducted by a single memristor crossbar configuration.

16. The method of claim 11 , further comprising:

providing a third plurality of resistance values by a third plurality of resistive memories to each corresponding input voltage from the plurality of input voltages applied to a second analog neuromorphic circuit when the second plurality of output voltages exceeds the threshold from the plurality of input voltages to train the second analog neuromorphic circuit to identify the input data associated with the plurality of input voltages.

17. The method of claim 16 , further comprising:

providing the plurality of input voltages to the second analog neuromorphic circuit when the second plurality of output voltages exceeds the threshold from the plurality of input voltages thereby indicating that the plurality of input voltages has not been previously identified and the training of the second analog neuromorphic circuit to identify the input data associated with the plurality of input voltages is required.

18. The method of claim 17 , further comprising:

converting the plurality of input voltages into an input eigenvector, the first plurality of output voltages to a first output eigenvector, and the second plurality of output voltages to a second output eigenvector.

19. The method of claim 18 , further comprising:

comparing the second output eigenvector to the input eigenvector to determine if a difference between the second output eigenvector and the input eigenvector deviates from an error threshold, wherein the deviation beyond the error threshold by the second output eigenvector and the input eigenvector is indicative that the input data associated with the input voltages has not been previously identified; and

generating the alert when the difference between the second output eigenvector and the first input eigenvector exceeds the error threshold.

20. The method of claim 19 , further comprising:

squaring each difference between each output voltage value that corresponds to each second output voltage from the second plurality of output voltages and each corresponding input voltage value that corresponds to each input voltage to generate a square rooted difference between each output voltage value and each corresponding input voltage value;

square rooting a sum of each square rooted difference between each output voltage value and each corresponding input voltage to generate a root squared error for the analog neuromorphic circuit;

comparing the root squared error to a standard deviation as the error threshold, wherein the standard deviation is previously determined as the error threshold that when the root squared error exceeds the standard deviation is indicative that the input data associated with the input voltages has not been previously identified; and

providing the plurality of input voltages to the second analog neuromorphic circuit when the root squared error exceeds the standard deviation of the error threshold thereby indicating that the plurality of input voltages has not been previously identified and the training of the second analog neuromorphic circuit to identify the input data associated with the plurality of input voltages is required.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 18, 2022
From: ALAM, MD SHAHANUR; YAKOPCIC, CHRIS; TAHA, TAREK M.
To: UNIVERSITY OF DAYTON
Reel/Frame 059951/0883 →
Continuity (2)
Provisional Application 63055572 · Jul 23, 2020
Related Publication 20220027718A1 · Jan 27, 2022
References Cited (33)
US 20160371582A1 · Eleftheriou · 2016 [cited by examiner]
US 20170011290A1 · Taha · 2017 [cited by examiner]
US 20180018559A1 · Yakopcic · 2018 [cited by examiner]
US 20180068217A1 · Eleftheriou · 2018 [cited by examiner]
US 20190034201A1 · Muralimanohar · 2019 [cited by examiner]
US 20190311267A1 · Qin · 2019 [cited by examiner]
US 20200117986A1 · Burr · 2020 [cited by examiner]
US 20210342678A1 · Mostafa · 2021 [cited by examiner]
US 20210374514A1 · Tsai · 2021 [cited by examiner]
Md Zahangir Alom, Tarek M. Taha, ‘Network intrusion detection for cyber security using unsupervised deep learning approaches’, National Aerospace and Electronic Conference (NAECON), Jun. 2017. [cited by applicant]
P. K. Prajapati and M. Dixit, “Un-Supervised MRI Segmentation using Self Organised Maps,” International Conference on Computational Intelligence and Communication Networks, pp. 471-474, India, Dec. 2016. [cited by applicant]
Raghavendra Chalapathy, Sanjay Chawla, “Deep Learning for Anomaly Detection: A Survey”, arXiv:1901.03407v2 [cs.LG], Jan. 2019. [cited by applicant]
L. O. Chua, “Memristor—The Missing Circuit Element,” IEEE Transactions on Circuit Theory, vol. 18, No. 5, pp. 507-519 (1971). [cited by applicant]
T. M. Taha, R. Hasan, C. Yakopcic, and M. R. McLean, “Exploring the Design Space of Specialized Multicore Neural Processors,” IEEE/INNS International Joint Conference on Neural Networks (IJCNN), pp. 1-8, Dallas, TX, Aug… [cited by applicant]
R. Hasan, T. M. Taha, and C. Yakopcic, “On-chip Training of Memristor Crossbar Based Multi-layer Neural Networks,” Microelectronics Journal, vol. 66, No. 8, pp. 31-40, Aug. 2017. [cited by applicant]
Yakopcic and T. M. Taha, “Analysis and Design of Memristor Crossbar Based Neuromorphic Intrusion Detection Hardware,” IEEE/INNS International Joint Conference on Neural Networks (IJCNN), pp. 1-7, Rio de Janeiro, Brazil,… [cited by applicant]
Raqibul Hasan and Tarek M. Taha, “Enabling Back Propagation Training of Memristor Crossbar Neuromorphic Processors”, International Joint Conference on Neural Networks (IJCNN), Beijing, Jul. 2014. [cited by applicant]
B. R. Fernando, R. Hasan and M. Tarek Taha, “Low Power Memristor Crossbar Based Winner Takes All Circuit,” 2018 IJCNN, Rio de Janeiro, 2018, pp. 1-6. [cited by applicant]
V. Nikulin, ‘Threshold-based clustering with merging and regularization in application to network intrusion detection’, Computational Statistics & Data Analysis vol. 51, pp. 1184-1196, Nov. 2006. [cited by applicant]
A. Laszka, W. Abbas, S. S. Sastry, Yevgeniy Vorobeychik, Xenofon Koutsoukos, ‘Optimal Thresholds for Intrusion Detection Systems’, Proceeding of the Symposium and Bootcamp on the Science of Security, Pittsburgh, Pennsyl… [cited by applicant]
Setareh Roshana, Yoan Michel, Anton Akusokd, Amaury Lendasse, “Adaptive and online network intrusion detection system using clustering and Extreme Learning Machines”, Journal of the Franklin Institute vol. 355, pp. 1752… [cited by applicant]
Subutai Ahmada, Alexander Lavina, Scott Purdya, Zuha Agha, “Unsupervised real-time anomaly detection for streaming data”, vol. 262, pp. 134-147, Neurocomputing, 2017. [cited by applicant]
Dromard J., Roudière G., Owezarski P. (2015) Unsupervised Network Anomaly Detection in Real-Time on Big Data. In: Morzy T., Valduriez P., Bellatreche L. (eds) New Trends in Databases and Information Systems. ADBIS 2015.… [cited by applicant]
G. Kathareios, A. Anghel, Akos M, R. Clauberg, M. Gusat, “Catch It If You Can: Real-Time Network Anomaly Detection With Low False Alarm Rates”, IEEE International Conference on Machine Learning and Applications, Cancun,… [cited by applicant]
Cosmin Radu Popa, ‘Synthesis of Computational Structures for Analog Signal Processing’, Springer Publisher, 2011. [cited by applicant]
Pavlo Tymoshchuk, Serhii Shatnyi, ‘Hardware Implementation Design of Parallelized Fuzzy Adaptive Resonance Theory Neural Network,’ 2019 IEEE XVth International Conference on the Perspective Technologies and Methods in M… [cited by applicant]
Leonardo Enzo Brito da Silva, Islam Elnabarawy, Donald C. Wunsch II, ‘A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications,’ arXiv:1905.11437v1 [cs.NE] May 4, 2019. [cited by applicant]
Lakhmi C. Jain, Beatrice Lazzerini, Ugur Halici (Eds.), ‘Innovations in ART Neural Networks’, Springer Publisher, ISBN 978-3-7908-1857-4, 2000. [cited by applicant]
C. Yakopcic, T. M. Taha, G. Subramanyam, and R. E. Pino, “Memristor SPICE Model and Crossbar Simulation Based on Devices with Nanosecond Switching Time,” IEEE International Joint Conference on Neural Networks (IJCNN), A… [cited by applicant]
Mohamed Elshamy, Hassan Mostafa, Yehya H. Ghallab, and Mohamed Sameh Said, ‘A Novel Nondestructive Read/Write Circuit for Memristor-Based Memory Arrays,’ IEEE Transaction on VLSI system, vol. 23, Issue: 11, 2648-2656, 2… [cited by applicant]
Ngamwitthayanon N., Wattanapongsakorn N., Coit D.W. (2009) Investigation of Fuzzy Adaptive Resonance Theory in Network Anomaly Intrusion Detection. In: Yu W., He H., Zhang N. (eds) Advances in Neural Networks—ISNN 2009.… [cited by applicant]
Sandra Ackerman, ‘Discovering the Brain,’ Chapter 8, p. 123, National Academy Press, Washington, D.C.1992. [cited by applicant]
Chenping Hou, Zhi-Hua Zhou, ‘One-Pass Learning with Incremental and Decremental Features,’ IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, Issue: 11, Nov. 2018. [cited by applicant]