IP Library Granted Patent US 11,567,770
Granted Patent B2
US 11,567,770 · App. 15/943,926 · Granted Jan 31, 2023

Human-machine-interface system comprising a convolutional neural network hardware accelerator

Inventors: Jose de Jesus Pineda de Gyvez (Eindhoven, NL); Hamed Fatemi (Eindhoven, NL); Gonzalo Moro Pérez (Eindhoven, NL); Hendrik Corporaal (Rotselaar, BE)
Assignee: NXP B.V.
G06F9/30112G06F3/0488G06F9/30G06F9/3001G06F9/462G06F17/153G06K9/6256G06N3/02G06N3/063G06N3/0635G06V40/20G06N3/0454
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,567,770
App. No.
15/943,926
Granted
Jan 31, 2023
Kind
B2
Abstract

A human-machine-interface system comprising: register-file-memory, configured to store input-data; a first-processing-element-slice, a second-processing-element-slice, and a controller. Each of the processing-slices comprise: a register configured to store register-data; and a processing-element configured to apply an arithmetic and logic operation on the register-data in order to provide convolution-output-data. The controller is configured to: load input-data from the register-file-memory into the first-register as the first-register-data; and load: (i) input-data from the register-file-memory, or (ii) the first-register-data from the first-register, into the second-register as the second-register-data.

Claims (50)

1. A human-machine-interface system comprising:

a register-file-memory, configured to store input-data;

a first-processing-element-slice comprising:

a first-register configured to store first-register-data; and

a first-processing-element configured to perform an arithmetic and logic operation on the first-register-data to obtain first-convolution-output-data;

a second-processing-element-slice comprising:

a second-register configured to store second-register-data; and

a second-processing-element configured to perform a convolutional neural network algorithm on the second-register-data to obtain second-convolution-output-data; and

a controller configured to:

load input-data from the register-file-memory into the first-register as the first-register-data; and

load one of the input-data from the register-file-memory and the first-register-data from the first-register into the second-register as the second-register-data, wherein the controller is configured to load one of the input-data and the first-register-data based on a subsampling rate between the input-data and convolution-output-data.

2. The human-machine-interface system of claim 1 , wherein the controller is configured to:

load a first subset of the input-data from the register-file-memory into the first-register as the first-register-data; and

load a second subset of the input-data from the register-file-memory into the second-register as the second-register-data, wherein the first subset of input-data is different from the second subset of input-data.

3. The human-machine-interface system of claim 1 , further comprising:

an intermediate-register; and

wherein the controller is configured to:

load the first-register-data from the first-register into the intermediate-register as intermediate-data; and

load the intermediate-data from the intermediate-register into the second-register as the second-register-data.

4. The human-machine-interface system of claim 3 , further comprising a plurality of intermediate-registers, serially connected between the first-register and the second-register.

5. The human-machine-interface system of claim 1 , further comprising:

one or more further-processing-element-slices, such that the first-processing-element-slice, the second-processing-element-slice and the one or more further-processing-element-slices comprise a sequence of processing-slices,

wherein each of the one or more further-processing-element-slices comprises:

a further-register configured to store further-register-data;

a further-processing-element configured to perform a convolutional neural network algorithm on the further-register-data to obtain further-convolution-output-data;

wherein the controller is configured to:

load one of input-data from the register-file-memory and register-data from an earlier register in the sequence of processing-slices into each further-register as the further-register-data based on the subsampling rate.

6. The human-machine-interface system of claim 1 , further comprising:

a weights-memory, configured to provide weights-data to each processing-element of the processing-element-slices;

wherein each processing-element of the processing-element-slices is configured to perform the arithmetic and logic operation based on the weights-data.

7. The human-machine-interface system of claim 1 , further comprising:

a bias-memory, configured to provide bias-data to each processing-element of the processing-element-slices;

wherein each processing-element of the processing-element-slices is configured to perform the arithmetic and logic operation based on the bias-data.

8. The human-machine-interface system of claim 1 , wherein the:

register-file-memory comprises a register-file-block associated with each of the processing-element-slices; and

the controller is configured to load input-data into the first-register of the first-processing-element-slice or the second-register of the second-processing-element-slice, respectively, from the associated register-file-block.

9. The human-machine-interface system of claim 8 , wherein the controller is configured to load input-data into the register of a processing-element-slice from a register-file-block associated with a different processing-element-slice.

10. The human-machine-interface system of claim 1 , further comprising a look-up-table configured to apply a non-linear function to the convolution-output-data provided by each of the processing-elements to obtain feature-map-output-data.

11. The human-machine-interface system of claim 10 , wherein the controller is configured to write the feature-map-output-data into the register-file-memory.

12. The human-machine-interface system of claim 11 , wherein a third-processing-element is configured to add the feature-map-output-data to obtain output-classification-data.

13. The human-machine-interface system of claim 1 , wherein the input-data is representative of non-contact human-machine-interface-signals.

14. The human-machine-interface system of claim 1 , wherein the input-data comprises analogue sensor data.

15. The human-machine-interface system of claim 1 , wherein the human-machine-interface system comprises a gesture recognition system.

16. The human-machine-interface system of claim 1 , wherein the controller is further configured to:

during a first processing cycle:

load input-data from the register-file-memory into the first-register as the first-register-data, and

load input-data from the register-file-memory into the second-register as the second-register-data; and

during a subsequent processing cycle:

load the first-register-data from the first-register into the second-register as the second-register-data, and

load input-data from the register-file-memory into the first-register as the first-register-data.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 22, 2022
From: PINEDA DE GYVEZ, JOSE DE JESUS; FATEMI, HAMED; MORO PÉREZ, GONZALO; CORPORAAL, HENDRIK
To: NXP B.V.
Reel/Frame 061185/0837 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 3, 2018
From: PINEDA DE GYVEZ, JOSE DE JESUS; FATEMI, HAMED; MORO PÉREZ, GONZALO; CORPORAAL, HENDRIK
To: NXP B.V.
Reel/Frame 045423/0102 →
Priority Claims (1)
EP 17166619 · Apr 13, 2017 · regional
Continuity (1)
Related Publication 20180300138A1 · Oct 18, 2018
Cited By (2)
US 12,366,925 US 12,462,448