Methods for designing hybrid neural networks having physical and digital components
Systems and methods for designing a hybrid neural network comprising at least one physical neural network component and at least one digital neural network component. A loss function is defined within a design space composed of a plurality of voxels, the design space encompassing one or more physical structures of the at least one physical neural network component and one or more architectural features of the digital neural network. Values are determined for at least one functional parameter for the one or more physical structures, and the at least one architectural parameter for the one or more architectural features, using a domain solver to solve Maxwell's equations so that a loss determined according to the loss function is within a threshold loss. Final structures are defined for the at least one physical neural network component and the digital neural network component based on the values.
1. A computer-implemented method for designing a hybrid neural network comprising at least one physical neural network component having one or more physical structures and configured to perform physical computations and at least one digital neural network component having one or more architectural features and configured to perform digital computations, the method comprising:
defining a loss function within a design space composed of a plurality of voxels, the design space encompassing the one or more physical structures of the at least one physical neural network component and the one or more architectural features of the at least one digital neural network component, the loss function corresponding to at least one inference metric of the hybrid neural network resulting from:
an interaction between an input signal at an operative wavelength of the one or more physical structures of the at least one physical neural network component; and
processing, by the at least one digital neural network component, the result of the interaction between the input signal and the one or more physical structures of the at least one physical neural network component;
determining, using a computer system, values for at least one functional parameter for the one or more physical structures, and at least one architectural parameter for the one or more architectural features, using a domain solver to solve Maxwell's equations so that a loss determined according to the loss function is within a threshold loss, wherein the at least one functional parameter corresponds to a structure of the at least one physical neural network component, and wherein the at least one architectural parameter corresponds to an architectural structure of the at least one digital neural network component;
defining a final structure of the at least one physical neural network component based on the values for the at least one structural parameter; and
defining a final architectural structure of the at least one digital neural network component based on the values for the at least one architectural parameter.
2. The method of claim 1 , wherein the final structure of the at least one physical neural network component includes one of more physical features, the one or more physical features including at least one of: a curvature of a scattering plate, a spacing between scattering plates, a substrate, a beam splitter, an amplifier or a modulator.
3. The method of claim 1 , wherein the one or more architectural features include at least one of: a number of neural network layers, a type of neural network layers, a number of neural network neurons, or a number of neural network channels.
4. The method of claim 1 , wherein the at least one inference metric of the hybrid neural network includes at least one of an inference speed or an inference accuracy.
5. The method of claim 1 , wherein the input signal includes at least one of an electromagnetic wave or an acoustic wave.
6. The method of claim 1 , wherein the at least one functional parameter includes at least one of: a refractive index, a reflection angle, a diffraction angle, or a transmissivity value.
7. The method of claim 1 , wherein the at least one architectural parameter includes at least one of a weight value or a bias value.
8. The method of claim 1 , wherein the domain solver includes a finite difference time domain solver.
9. The method of claim 1 , wherein the physical computations comprise physical convolutions and the digital computations comprise digital convolutions.
10. A system for designing a hybrid neural network comprising at least one physical neural network component having one or more physical structures and configured to perform physical computations and at least one digital neural network component having one or more architectural features and configured to perform digital computations, the system comprising:
one or more processors; and
computer storage storing executable computer instructions in which, when executed by the one or more processers, cause the one or more processors to perform operations comprising:
defining a loss function within a design space composed of a plurality of voxels, the design space encompassing the one or more physical structures of the at least one physical neural network component, and the one or more architectural features of the at least one digital neural network component, the loss function corresponding to at least one inference metric of the neural network resulting from:
an interaction between an input signal at an operative wavelength of the one or more physical structures of the at least one physical neural network component; and
processing, by the at least one digital neural network component, the result of the interaction between the input signal and the one or more physical structures of the at least one physical neural network component;
determining values for at least one functional parameter for the one or more physical structures, and at least one architectural parameter for the one or more architectural features, using a domain solver to solve Maxwell's equations so that a loss determined according to the loss function is within a threshold loss, wherein the at least one functional parameter corresponds to a structure of the at least one physical neural network component, and wherein the at least one architectural parameter corresponds to an architectural structure of the at least one digital neural network component;
defining a final structure of the at least one physical neural network component based on the values for the at least one structural parameter; and
defining a final architectural structure of the at least one digital neural network component based on the values for the at least one architectural parameter.
11. The system of claim 10 , wherein the final structure of the at least one physical neural network component includes one of more physical features, the one or more physical features including at least one of: a curvature of a scattering plate, a spacing between scattering plates, a substrate, a beam splitter, an amplifier or a modulator.
12. The system of claim 10 , wherein the one or more architectural features include at least one of: a number of neural network layers, a type of neural network layers, a number of neural network neurons, or a number of neural network channels.
13. The system of claim 10 , wherein the at least one inference metric of the hybrid neural network includes at least one of an inference speed or an inference accuracy.
14. The system of claim 10 , wherein the input signal includes at least one of an electromagnetic wave or an acoustic wave.
15. The system of claim 10 , wherein the at least one functional parameter includes at least one of: a refractive index, a reflection angle, a diffraction angle, or a transmissivity value.
16. The system of claim 10 , wherein the at least one architectural parameter includes at least one of a weight value or a bias value.
17. The system of claim 10 , wherein the domain solver includes a finite difference time domain solver.
18. The system of claim 10 , wherein the physical computations comprise physical convolutions and the digital computations comprise digital convolutions.
19. A neural network comprising:
at least one physical neural network component having one or more physical structures and configured to perform physical computations; and
at least one digital neural network component having one or more architectural features and configured to perform digital computations, the system comprising:
wherein the at least one physical neural network component and the at least one digital neural network component are designed by:
determining values for at least one functional parameter for the one or more physical structures of the at least one physical neural network component, and at least one architectural parameter for the one or more architectural features of the at least one digital neural network component, using a domain solver to solve Maxwell's equations so that a loss determined according to a loss function is within a threshold loss;
defining a final structure of the at least one physical neural network component based on the values for the at least one structural parameter; and
defining a final architectural structure of the at least one digital neural network component based on the values for the at least one architectural parameter.
20. The neural network of claim 19 , wherein:
the loss function is defined within a design space composed of a plurality of voxels, the design space encompassing the one or more physical structures of the at least one physical neural network component, and the one or more architectural features of the at least one digital neural network component, the loss function corresponding to at least one inference metric of the neural network resulting from:
an interaction between an input signal at an operative wavelength of the one or more physical structures of the at least one physical neural network component; and
processing, by the at least one digital neural network component, the result of the interaction between the input signal and the one or more physical structures of the at least one physical neural network component.