Optical soft skin system for multimodal sensing
An optical soft skin system for multimodal sensing comprising: a flexible waveguide substrate ( 10 ) being able to change its light wave propagation when encountering different deformation patterns; at least one light source ( 12 ) located within the waveguide substrate ( 10 ) arranged so as to transmit light into the waveguide substrate ( 10 ); at least one light receptor ( 14 ) located within the waveguide substrate ( 10 ) to receive light that has passed through the waveguide substrate ( 10 ) and to provide a dataset of the properties of the received light; and a processor ( 20 ) for decoding the dataset into at least one of strain and displacement through a computational model to determine a sensing modality of the waveguide substrate ( 10 ).
1 . An optical soft skin system for multimodal sensing comprising:
a flexible waveguide substrate being able to change its light wave propagation when encountering different deformation patterns;
at least one light source located within the waveguide substrate arranged so as to transmit light into the waveguide substrate;
at least one light receptor located within the waveguide substrate to receive light that has passed through the waveguide substrate and to provide a dataset of the properties of the received light, each light receptor being sensitive to a different spectrum of light transmitted by each at least one light source; and
a processor configured to decode the dataset into at least one of strain and displacement through a computational model to determine a sensing modality of the waveguide substrate to modes of bending, pressing, stretching and twisting.
2 . The optical soft skin system for multimodal sensing of claim 1 , wherein the waveguide substrate is made of higher refractive index material compared to that of an environment in which the optical soft skin system is being used, so that light therein is trapped within the waveguide substrate according to the principle of total internal reflection.
3 . The optical soft skin system for multimodal sensing of claim 2 , wherein the material is Polydimethylsiloxane or hydrogel.
4 . The optical soft skin system for multimodal sensing of claim 2 , wherein the waveguide substrate has an interwoven structure.
5 . The optical soft skin system for multimodal sensing of claim 2 , wherein the waveguide substrate is coated with an outer layer to reduce the influence of external light on the waveguide substrate.
6 . The optical soft skin system for multimodal sensing of claim 1 , wherein said computational model used by said processor is a machine learning model that has been trained a on a plurality of the datasets acquired under different sensing modalities.
7 . The optical soft skin system for multimodal sensing of claim 6 , wherein the plurality of the datasets has been enlarged by employing a finite element (FE) based data enrichment method to enlarge the dataset for model training.
8 . The optical soft skin system for multimodal sensing of claim 1 , wherein the processor is configured to execute a machine learning method for decoding different sensing modalities.
9 . The optical soft skin system for multimodal sensing of claim 1 , wherein the waveguide substrate has a heterogeneous composition.
10 . The optical soft skin system for multimodal sensing of claim 9 , wherein the waveguide substrate is comprised of multiple layers of materials so as to influence wave propagation.
11 . The optical soft skin system for multimodal sensing of claim 9 , wherein the heterogeneous composition has periodic variations in material properties to enhance sensitivity to different sensing modalities.
12 . The optical soft skin system for multimodal sensing of claim 1 , wherein the natural, un-deformed shape of the waveguide substrate is a flat rectangular prism.
13 . The optical soft skin system for multimodal sensing of claim 1 , wherein the waveguide substrate is of varying thickness in the direction of light propagation in the waveguide.
14 . The optical soft skin system for multimodal sensing of claim 1 , wherein wave propagation in the waveguide substrate is affected by changes in refraction, reflection, diffraction, and polarization induced by the waveguide substrate deformation patterns.
15 . The optical soft skin system for multimodal sensing of claim 1 , wherein the at least one light source emits light over different spectra, including at least one of invisible light and visible light.
16 . The optical soft skin system for multimodal sensing of claim 1 , wherein the at least one light receptor detects light intensity, wavelength, phase, or a combination thereof.
17 . The optical soft skin system for multimodal sensing of claim 1 , wherein each light receptor is sensitive to the entire spectrum of light transmitted by the at least one light source.
18 . The optical soft skin system for multimodal sensing of claim 1 , wherein the at least one light receptor is sensitive to the entire spectrum of transmitted light, or the at least one light receptor is sensitive to a specific range of the transmitted light.
19 . The optical soft skin system for multimodal sensing of claim 1 , wherein the at least one light source is a light-emitting diode and the at least one light receptor is a photodiode.
20 . The optical soft skin system for multimodal sensing of claim 1 , wherein there are at least three light sources which are light emitting diodes providing red, green and blue light outputs, respectively.
21 . The optical soft skin system for multimodal sensing of claim 1 , wherein the waveguide substrate has particles embedded therein that cause the transmitted light to be reflected, refracted, dispersed, absorbed or polarized when interacting with the particles.
22 . The optical soft skin system for multimodal sensing of claim 1 , wherein the waveguide substrate is fabricated with patterned channels or slots to enhance the effect of deformation on the transmitted light and the magnitude of association in mapping between mechanical changes and optical changes of the waveguide substrate.
23 . The optical soft skin system for multimodal sensing of claim 1 , further including a display and wherein a detected at least one of strain and displacement induced by bending, pressing, stretching, and twisting is visually reconstructed and presented in real time in three degrees of freedom on the display by the processor.
24 . The optical soft skin system for multimodal sensing of claim 1 , wherein design parameters including the waveguide substrate thickness, placement and number of light sources and light receptors, type of detected and transmitted light, and waveguide substrate shape can be customized to each surface prior to fabrication through numerical analysis.
25 . The optical soft skin system for multimodal sensing of claim 24 wherein the numerical analysis is finite-element (FE) analysis.
26 . The optical soft skin system for multimodal sensing of claim 1 , wherein a simulated environment improves the quantity and quality of data for model training through finite element (FE) based data enrichment, where a simulated environment has the waveguide substrate modeled as a hyper-elastic material meshed in C3D8RH linear brick elements according to an elastic modulus, Poisson's ratio and parameters of MooneyRivlin hyper-elastic model.
27 . The optical soft skin system for multimodal sensing of claim 1 , wherein the soft skin is portable and further includes a wireless data transmitter for transmitting sensor signals.
28 . The optical soft skin system for multimodal sensing of claim 1 , wherein the soft skin is designed to be worn by a user as a wearable clothing device.
29 . The optical soft skin system for multimodal sensing of claim 28 wherein the wearable clothing device is a glove on the hand of the user.
30 . The optical soft skin system for multimodal sensing of claim 1 , wherein the optical soft skin system is integrated into a robotic system for proprioception.
31 . The optical soft skin system for multimodal sensing of claim 1 , wherein the soft skin system further includes solid wire or liquid wire for electrical connection.
32 . The optical soft skin system for multimodal sensing of claim 1 , wherein the different sensing modalities further include an overall three dimensional surface morphology of the waveguide substrate.