Human skin-inspired neuromorphic sensors

INTRODUCTION

Human skin is one of the most sophisticated and adaptive sensory organs, capable of perceiving a wide array of stimuli, such as pressure, temperature, texture, and other physical or chemical factors, with remarkable sensitivity and precision. This sensory capability enables dynamic interaction with environments, performing complex tasks and maintaining homeostasis. Replicating these diverse sensing functions in artificial systems is a key research area in materials science, robotics, and wearable technology^[1,2]. Skin-inspired sensors, also known as artificial skin, are designed to mimic the mechanical, electrical, and sensory properties of human skin. In addition to mimicking basic sensory functions, multifunctional sensors integrate multiple sensing modalities into a single platform that can simultaneously detect various environmental factors such as strain, humidity, and biochemical markers^[3,4]. These sensors typically utilize flexible or stretchable materials that can detect a range of environmental stimuli, from mechanical pressure and tactile feedback to changes in temperature and humidity. Advanced materials, such as dielectric elastomers, conductive polymers, and piezoelectric materials, provide the necessary sensitivity and responsiveness for real-time sensory feedback^[5].

However, the perception information signals from these flexible sensors are continuous analog signals, which can be converted into discrete digital signals using traditional analog-to-digital converters (ADC)^[6,7]. This conversion process results in massive data processing requirements, leading to increased need for communication speed and energy costs due to the distance between memory and computing units. In contrast, biological sensory organs can in situ detect and process external stimuli, transmitting the processed information directly to the brain for final analysis and decision-making. The human brain is a complex system consisting of a network of approximately 100 billion neurons interconnected through 100 trillion synapses. The power consumption of the brain performing its incredible feats is nearly 20 W, whereas a standard computer needs about 250 W to accomplish the same tasks^[8,9].

To overcome these challenges, researchers have investigated neuromorphic computing systems, which are inspired by the neuro-synaptic framework of the human brain. When combined with neuromorphic computing technology, skin-inspired neuromorphic sensors have shown great potential to create truly intelligent, adaptive systems capable of responding to sensory input in a manner that mirrors human-like cognition and decision-making capabilities^[10]. By processing sensory data locally and efficiently, these systems can reduce latency, lower energy consumption, and enable more responsive interactions with the environment. Neuromorphic systems leverage massive parallelism to reduce energy consumption in signal processing systems and address the bottlenecks of the von Neumann architecture^[11]. Unlike traditional architectures, neuromorphic computing integrates memory and processing within the same module, facilitating compute-in-memory (CIM) capabilities that enhance communication speed and reduce energy costs. Traditional complementary metal-oxide semiconductor (CMOS) technology relies on intricate auxiliary circuits and large capacitors to emulate biodynamics, posing challenges in large-area manufacturing, device integration, and complex circuit design.

To address these issues, researchers have focused on developing artificial synapses by leveraging emerging nonvolatile memory devices, such as nonvolatile memristors, diffusive memristors, synaptic transistors, etc. Although volatile memory devices are not typically used for long-term data storage, complementary nonvolatile devices can provide fast, short-term memory capabilities for dynamic processing. Synaptic transistors can be designed in various configurations, such as bottom-gated, side-gated, floating gate, or top-gated, depending on the desired electrical characteristics and applications^[12]. These different gate structures offer flexibility in controlling the characteristics of transistors, such as threshold voltage, switching speed, and retention properties, showing potential for mimicking synaptic functions in neuromorphic systems^[13]. Since the first oxide-based resistive switches were demonstrated as memristors in 2008, memristors have been used as computing units in the form of crossbar arrays. A crossbar array could perform parallel matrix calculations, enabling CIM capabilities^[14,15]. Memristors play an important role in motion detection^[16], robot navigation^[17], and so on. In addition to oxide-based resistive switches, other resistive neuromorphic compute units, including ferroelectric artificial synapses^[18,19], ion-intercalation resistors, and memtransistors with a three-terminal structure, exhibited similar properties^[20]. Recently, memristors based on atomically thin sheets of organic and inorganic materials, such as 2D hexagonal boron nitride and organic polymers such as polyimide covalent organic framework (PI-NT COF), 2,7-dioctyl[1]benzothieno[3,2-b][1]benzothiophene (C₈-BTBT), poly(vinylidenefluoride-co-trifluoroethylene) (PVDF-TrFE), metal-organic frameworks (MOFs), etc., have been fabricated. These memristors exhibit fast switching ratios and low energy consumption^[21-25], resulting in high computation efficiency to support human-like data processing in neuromorphic systems. The fabrication of these devices can be achieved through techniques such as photolithography, printing methods, dry etching and other advanced fabrication processes, enabling scalable and cost-effective production for integration into artificial intelligence (AI) and electronic circuits^[26-28].

Intelligent algorithms enhance skin-inspired neuromorphic sensor systems by boosting their adaptability, data processing efficiency, decision-making accuracy, and energy efficiency, enabling them to respond dynamically and autonomously in complex environments. They will lead to more precise, real-time performance and proactive capabilities across a range of smart applications. With advancements in AI, particularly through artificial neural networks (ANNs) and spiking neural networks (SNNs), sensor systems can process data more intelligently, make better decisions, and interact more effectively with their surroundings. Early ANNs, known as perceptrons, were restricted to solving linear classification problems^[29]. With the development of ANNs, the backpropagation algorithm was introduced to train neural networks with hidden layers. However, the limitations of traditional ANNs, such as high computational costs, energy inefficiency, and inability to handle temporal data effectively, have driven the need for more advanced models. After that, the emergence of the third generation of ANNs, known as SNNs^[26], presented an event-driven signal processing approach and a promising alternative for breaking the von Neumann bottleneck. Compared with ANNs, the biggest challenge for SNNs is the training approach. Due to their complex dynamics and the non-differentiable nature of spiking pulses, backpropagation algorithms cannot be directly used in SNNs. The connection strength among neurons depends on the relative time difference between the pulses emitted by presynaptic and postsynaptic neurons. Based on this principle, spike-timing-dependent plasticity (STDP) is used to tune the connection strength in synapses^[27]. However, the accuracy of SNN algorithms is weaker than that of ANNs. To improve the accuracy of SNNs, some new technologies inspired by ANNs are being employed to address the training challenges^[30].

Building upon recent advancements in neuromorphic computing and artificial synapses, researchers have explored the synergy between intelligent processing systems and sophisticated sensory inputs^[31-33]. Human skin covers a vast surface area and integrates diverse sensory modalities, necessitating sensor arrays that can similarly encompass extensive regions to accurately replicate its sensory capabilities. To effectively replicate the comprehensive sensory capabilities, the integration of skin-inspired sensors with neuromorphic architecture holds significant potential for enhancing the adaptability and efficiency of artificial sensory platforms. By merging multimodal sensing artificial skin with parallel processing neuromorphic devices, it becomes feasible to develop neuromorphic sensors that closely emulate the responsiveness and decision-making processes of the human body^[34-36]. Despite these advantages, there are still significant challenges to be addressed in the development of neuromorphic sensors. Traditional CMOS technology, which is widely used in the fabrication of electronic devices, faces limitations in terms of large-area manufacturing, device integration, and complex circuit design when attempting to mimic the biodynamics of biological systems. Additionally, the integration of emerging memory devices, such as memristors and synaptic transistors, into neuromorphic systems remains a challenging yet essential area of research. These devices offer the potential for fast, energy-efficient data processing, but their integration into large-scale systems requires overcoming challenges related to fabrication, scalability, and reliability.

In this study, we present a recent comprehensive review on human skin-inspired sensors combined with neuromorphic computing technologies. Different from previously published reviews^[37-39], we focus on the role of neuromorphic computing within the sensor systems themselves, as opposed to emphasizing its integration at the system level, including the specific sensor modalities and the integration of neuromorphic devices such as memristors, transistors, etc., as shown in Figure 1. First, we introduce the multimodal perception of skin-inspired sensors in their principles and types. Subsequently, we discuss the application of neuromorphic devices including transistors and memristors in skin-inspired sensors and provide detailed descriptions for each. In the section on neural network algorithms, we introduce the latest developments in algorithms in neuromorphic computing. Then we present examples of on-chip neuromorphic computing systems in applications such as human health monitoring, robotic skin, and other related fields, highlighting their potential for real-time processing and adaptive responses in these areas. Finally, we offer insights into the prospects of large-scale manufacturing of human skin-inspired neuromorphic sensors, emphasizing the technological advancements required to enhance scalability, performance, and integration into real-world applications.

Figure 1. Schematic structures of skin-inspired neuromorphic sensors.

MULTIMODAL PERCEPTION

The human perception system facilitates high-level cognition and learning through the integration and interaction of vision, hearing, touch, smell, and other senses. Numerous studies have utilized flexible sensors in skin-inspired systems to achieve perceptive abilities. To fully simulate the human perceptual system, it is essential to model multiple types of sensory signals.

These neuromorphic sensors can only realize their full potential through multimodal perception, as shown in Figure 2. Multimodal perception refers to the integration of heterogeneous data acquired from various sensors (such as vision, touch, hearing, etc.) to provide a comprehensive understanding of the environment or target^[40]. The core of this process lies in data fusion and collaborative analysis, which includes the following key principles: first, data acquisition involves obtaining multimodal data from different sensors (e.g., tactile arrays, temperature sensors), where each modality has distinct physical properties and spatiotemporal resolutions^[41]. The data collected from these sensors is often complementary or redundant (e.g., vision cannot perceive object hardness, requiring assistance from tactile sensors). Through appropriate fusion strategies, the information utilization rate is optimized, and the accuracy of each modality’s measurements is enhanced. Once data collection is completed, feature extraction and representation are required to process the heterogeneous data. For example, features from visual data are typically extracted using convolutional neural networks (CNNs), while tactile data is analyzed by extracting pressure signal features based on time sequences^[42]. Subsequently, methods such as sparse coding and graph neural networks are employed to establish associative models between different modalities^[43]. For instance, tactile data can be integrated with visual texture features to improve object recognition accuracy. Finally, data fusion is performed. In strongly correlated scenarios (e.g., synchronized vision and touch), multimodal raw data is directly concatenated at the data level. In weaker correlation or asynchronous scenarios (e.g., combining visual recognition with voice commands), each modality is independently processed before the decision results are fused. By combining these two approaches and optimizing the joint model, multimodal perception is achieved. In this section, we summarize recent developments in skin-inspired sensors, which have shown potential to achieve multimodal perception.

Figure 2. Schematic diagram of the on-chip neuromorphic computing system. The workflow of such systems is as follows: skin-inspired sensors collect various environmental signals from the environment (such as temperature, pressure, humidity, contact, proximity, etc.), which are then preliminarily processed by corresponding neural devices. A subsequent modeling step transforms the sensor data into a format suitable for neural network input, employing techniques such as feature extraction, normalization, or dimensionality reduction. Finally, different neural network algorithms are utilized for deep processing, thereby making corresponding responses or obtaining the required data.

Types of skin-inspired sensors

Skin-inspired sensors are designed to mimic the sensory abilities of human skin and are commonly used in applications such as electronic skin (e-skin), smart prosthetics, bionic hands, robotic skin, wearable devices, and multimodal sensing systems. In recent years, sensors have increasingly developed towards multifunctionality and large-scale arrays. Integrating more functions into a single sensor meets the diverse measurement needs for various physical factors. The manufacturing of larger-scale sensor arrays is essential for skin-like functional sensing.

To integrate multiple functions into a single sensor, each type of sensor must exhibit superior performance. For example, temperature sensors emulate the skin’s ability to sense temperature changes in the environment or objects. Resistive temperature sensors detect temperature through changes in resistance, particularly exhibiting high sensitivity in lower temperature ranges. One example involved a resistive temperature sensor with a thin Pt film deposited on a polyimide (PI) substrate, forming a Pt-based 9-channel array resistive temperature sensor^[58] [Figure 3A]. Resistance temperature detectors (RTDs) measure temperature by detecting changes in material resistance, offering high accuracy and stability. These sensors could detect temperature changes through electrical fields or signals, suitable for flexible and wearable devices. Lead titanate (PbTiO₃) is a commonly used material in RTDs, and the ionic organic hydrogel (PC₁₀₀T₅₀N_66.7) with high ionic conductivity (2.7 S·m^-1) is another sensing material for RTD fabrication with excellent temperature-sensitive properties for dynamic temperature monitoring^[59]. Thin-film temperature sensors, typically made from conductive thin-film materials (e.g., metal oxides and carbon-based materials), detect temperature through changes in resistance, offering high sensitivity and ease of skin contact. A novel micro-three-dimensional (3D) structure with better malleability was designed, which also took advantage of the fast response of a two-dimensional thin film. The sensor enabled real-time temperature measurement on-site, offering advantages such as small thermal mass and fast response time^[60] [Figure 3B].

Figure 3. Schematic diagrams of temperature sensors, pressure sensors, and multifunctional sensors. (A) Layer-by-layer description of a Pt-based flexible 3 × 3 array temperature sensor. Reproduced with permission^[58]. Copyright 2024, Materials Today Nano; (B) Flexible micro-3D thin-film sensor for temperature measurement. Reproduced with permission^[60]. Copyright 2021, Microsystems & Nanoengineering; (C) Robust flexible pressure sensors made from conductive micropyramids. Reproduced with permission^[61]. Copyright 2020, ACS Nano; (D) Structure of the TCTS array. Reproduced with permission^[64]. Copyright 2024, ACS Nano; (E) Schematic diagram of a flexible capacitive pressure sensor based on electrospun PI nanofiber membrane. Reproduced with permission^[68]; (F) Schematic illustration of the fully printed trimodal (Proximity-pressure-temperature) sensor sheet. Reproduced with permission^[71]. Copyright 2021, Advanced Materials Technologies; (G) The left expanded view of the multilayered construction of an entire 3DAE-Skin device. The right expanded view of a representative functional unit. Reproduced with permission^[73]. Copyright 2024, Science. 3D: Three-dimensional; TCTS: triboelectric capacitive-coupled tactile sensor; PI: polyimide.

Pressure sensors have a wide range of applications in skin-like functional sensors. Tactile sensors emulate the skin’s ability to sense pressure, touch, and texture, primarily for detecting contact, pressure, and surface features. In skin-inspired sensing, various types of pressure sensors, including capacitive, piezoresistive, optical, and self-powered pressure sensors, have been extensively explored to achieve high sensing performance. The piezoresistive pressure sensor is a commonly used pressure sensor based on the strain effect. When the object undergoes deformation, the resistance inside the sensor changes, allowing it to detect the values of pressure, stretching, or bending. One example involved a flexible, high-sensitivity, and robust pressure sensor made from a PDMS/carbon nanotube composite material, exhibiting fast response and low operating voltage^[61] [Figure 3C]. A graphene-based piezoresistive pressure sensor with tunable sensing performance was fabricated using a template^[62]. Additionally, some self-powered tactile sensors without external power sources harvest energy from the environment to operate and continuously monitor pressure changes^[63]. By integrating capacitive sensor arrays for static spatiotemporal mapping and friction generator sensors for dynamic tactile recognition, a flexible bimodal friction-capacitive coupled tactile sensor array was created to enable active dynamic tactile sensing^[64] [Figure 3D]. A smart neuromorphic tactile sensor based on a triboelectric nanogenerator (TENG) achieved self-powered pressure sensing capabilities^[65]. Another commonly used pressure sensor is the capacitive pressure sensor, which detects touch or contact by measuring changes of capacitance. A recent design featured a pressure sensor with a multi-size planar structure, inducing charge exchange between adjacent electrodes upon external touch, providing ultra-high sensitivity. Each pixel could be fabricated within a several-micrometer range and a tiny pressure of 0.02 Pa would result in a 750% increase in the relative capacitance, equivalent sensitivity of 3.75 × 10⁵ kPa^-1 for 0-0.05 Pa, exceeding all the previous reports to date^[66]. A new flexible PDMS-based capacitive tactile sensor array was fabricated to measure normal and shear force distributions. Measurement of a single sensor shows that the full-scale range of detectable force is about 10 mN, which corresponds to 131 kPa in three directions^[67]. Another example involved a flexible capacitive pressure sensor based on electrospun PI nanofiber films as the dielectric layer. The sensor with such a dielectric layer exhibited high sensitivity (2.204 kPa^-1 at 3.5-4.1 Pa), wide scale range (0-1.388 MPa), low detection limit (3.5 Pa) and good cyclic stability (> 10,000 cycles)^[68] [Figure 3E].

Apart from improving the performance of various sensors, another important research challenge is how to integrate these sensors while maintaining their performance. Achieving the integration of multiple sensor functions into large-area flexible sensor arrays, while ensuring the preservation of excellent performance, holds great promise for future applications^[69]. By combining various sensing functions, multifunctional sensors have been developed to simultaneously detect temperature, pressure, and tactile stimuli^[70]. These sensors not only enhance the diversity and intelligence of the sensing system, but also provide more sensory feedback, closely simulating the sensory capabilities of human skin. Such sensors find wide applications in e-skin, smart prosthetics, bionic hands, robotic skin, wearable devices, and multimodal sensing systems. To achieve multi-mode sensing, a fully printed flexible trimodal sensor sheet, containing 4 × 4 pressure sensor units, 2 × 2 temperature sensor units, and 1 proximity sensor unit, could simultaneously achieve temperature, pressure, and proximity sensing was designed^[71] [Figure 3F]. Overall, multi-mode sensing devices have made significant advancements.

However, despite the tremendous potential of these multi-mode sensing devices technically, they still face several key drawbacks and challenges. First, multi-mode sensing devices are designed to simultaneously detect multiple physical fields, but interactions and signal interference between different fields can occur. In simple terms, changes in temperature and humidity may influence the measurements of pressure sensors, or electromagnetic fields and mechanical vibrations may affect the accuracy of sensor readings. This cross-field interference can result in inaccurate measurements, thereby affecting the overall performance of the sensor. For example, by employing a pressure-temperature decoupling strategy, a highly sensitive iontronic bimodal sensor with pressure-temperature discriminability exhibited a maximum force error of 5.9% in the 3-10 N range^[72]. Additionally, this single-unit sensor may cause larger measurement errors when arranged in an array. In the design process, multi-mode sensing devices must deliver high precision across multiple domains, often requiring a trade-off between sensitivity, stability, and response time. Due to the different characteristics of each physical field, optimizing the sensor to maintain high precision and sensitivity across various scenarios is a complex challenge. Over-optimization of one feature could adversely affect the measurement accuracy of other fields. Furthermore, as sensors are used over time, their performance may degrade, and there is a lack of long-term performance testing in current research. Inspired by the excellent decoupling capabilities of human skin, a 3D architecture for an e-skin (3DEA) device was developed to simultaneously measure the modulus and curvature of an object through normal force, shear force, and strain. However, it integrated only a 5 × 5 array of multifunctional sensor units (each sensor is approximately 12 mm × 12 mm)^[73] [Figure 3G]. These sensor arrays often suffer from crosstalk, reduced performance, and challenges in functional integration. To address the issue of area, researchers have designed an ion-electron pressure sensor integrated with a 32 × 32 array of pressure sensor units. These skin-inspired iontronic sensors exhibit unexpectedly high sensitivity (365 kPa^-1) and an ultra-broad range (1.7 Pa to 1,000 kPa), with a total size of approximately 32 cm × 32 cm. Although this sensor array featured a large scale, it only served a single function^[74].

Current research faces limitations in sensor array size, sensing functions, and performance, which fail to meet the integration requirements of advanced systems. Additionally, most sensor arrays lack intelligence, with little integration of neuromorphic devices such as memristors or neural network algorithms to enhance performance. Combining these elements can significantly improve sensor capabilities. Future research should focus on developing large-scale, multifunctional, and intelligent sensor arrays to address these challenges.

NEURAL DEVICES

In biology, signals are conveyed through neurons interconnected in complex networks. When a signal reaches a certain threshold, it is transmitted to the next neuron as a positive signal. Currently, memristors and transistors serve as synaptic devices in neuromorphic computing. A transistor consists of a channel layer made of semiconductor materials, a dielectric layer, and source and drain terminals. By tuning the gate voltage, the current from source to drain can update synaptic weights. Neuromorphic computing using memristors focuses on mimicking the brain’s synaptic functions. A memristor is a novel device that represents the relationship between magnetic flux and charge, alongside resistors, capacitors, and inductors. Memristors typically comprise electrode layers and functional material layers. Functional materials in memristors are classified into inorganic and organic categories. Inorganic materials include HfOx, NiOx, TiOx, TaOx, etc., while organic materials encompass conductive polymers, such as poly(3,4-ethylenedioxythiophene):polystyrene sulfonate (PEDOT:PSS), Nafion and other small organic molecules^[75-77]. Hewlett-Packard (HP) was the first to experimentally verify the existence of memristors in 2008 by introducing a device composed of a double layer of TiO₂ thin film (Pt/TiO₂/Pt)^[78]. Since this groundbreaking discovery, memristors have been extensively utilized to model synaptic connections in simulated neural networks. By integrating memristors with components such as capacitors, researchers can effectively replicate the signal transmission mechanisms of biological synapses, thereby enhancing the complexity and efficiency of neuromorphic systems. Advanced artificial neurons utilizing volatile NbOx memristors have been developed to execute threshold-driven spiking and spatiotemporal integration. Beyond these traditional functions, these neurons supported dynamic logic operations, including the exclusive-OR (XOR) function and multiplicative gain modulation, which surpassed the limitations of simple point neuron models. Fully memristive neural networks, composed of volatile NbOx-based neurons and nonvolatile TaOx-based synapses, have demonstrated robust pattern recognition capabilities through online learning and coincidence detection. A Pt/Ti/NbOx/Pt/Ti volatile memristor was placed in series with a load resistor, while a capacitor was connected in parallel to generate spiking signals^[79]. These developments highlight the significant potential of memristive technologies in sophisticated and efficient neuromorphic systems. Furthermore, field-effect transistors with tunable gate voltages could mimic the variable strength of synaptic connections by adjusting the channel conductivity, thereby enabling the dynamic modulation of signal transmission between simulated neurons similarly to biological synapses.

In a recent study, a single-electrode mode TENG and a MoS₂ transistor were vertically coupled, allowing MoS₂ tribotronic to function as an interactive smart tactile switch with distinct on/off behavior^[80]. The system mimicked the natural arrangement of touch receptors and neurons by integrating TENG technology with transistors for signal generation and processing. This self-powered system operated with event-driven mechanisms, similar to biological nerve responses to tactile stimuli. A ring oscillator, combined with a transistor, generated spike information, while an inverting amplifier circuit functioned as a synaptic architecture, encompassing multiple synaptic connections of an artificial cuneate neuron^[81] [Figure 4A]. Each artificial tactile afferent is composed of a receptor (blue) and a leaky integrate-and-fire (LIF) neuron (brown), with V_DD denoting the drain voltage [Figure 4B]. The LIF neurons were implemented in hardware, while SNN algorithm was executed in software to extract tactile features from the encoded tactile information^[82]; finally, spike timing-based coding in neuromimetic tactile system achieved object classification based on SNNs. Flexible pressure sensors are essential in e-skin applications, enabling seamless interaction between biomedical prosthetics, robots, and their environments. These sensors are also promising for long-term medical diagnostics and mobile biomonitoring. For example, flexible pressure-sensitive organic thin-film transistors with a peak sensitivity of 8.4 kPa^-1 and rapid response times under 10 milliseconds have been developed^[83] [Figure 4C]. As shown in Figure 4D, an active-matrix array for reducing interconnect wiring is fabricated; however, it lacks circuits capable of generating spiking signals. To construct a monolithic soft e-skin system, a monolithically integrated, low-voltage, and flexible e-skin was designed to closely mimic the sensory feedback and mechanical properties of natural skin. This advanced e-skin system was capable of multimodal perception, neuromorphic signal processing, and closed-loop actuation. It was achieved with the high-permittivity elastomeric dielectric and stretchable organic transistor^[84] [Figure 4E]. Besides this, other researchers have tried to integrate the sensors and transistors vertically. The pressure-sensitive transistor was made by laminating two separate layers: the first layer comprised the lower source and drain electrodes together with the semiconducting polymer, whereas the second layer included the gate electrode and the microstructured dielectric^[85]. The integration of a microstructured PDMS dielectric with the high-mobility semiconducting polyisoindigo-bithiophene-siloxane in a monolithic transistor enabled its application in health monitoring. However, all existing pressure sensors suffer from the interference between stretching and pressure sensing accuracy. Transistors based on organic material undergo reversible redox reactions under an electric field, leading to changes in conductivity^[86]. By using this characteristic, a highly stretchable and sensitive pressure sensor was achieved through an ionic capacitive mechanism and a hierarchical microstructure has shown the ability to realize the accurate sensation of physical interactions on soft robotic skin^[87]. A neuro-inspired monolithic artificial tactile neuron (NeuroMAT) was fabricated by an ion trap and release dynamics (iTRD)-iongelgated-synaptic transistor to emulate the tactile recognition and learning of human skin with low power consumption^[88] [Figure 4F]. This tactile neuron could be attached to the robotic hand to grasp the object by programming the required force [Figure 4G]. Transistors could also function as synapses to receive signals from multiple sensors. The artificial system consists of multiple sensors and multi-gate synaptic transistors, which are used to replicate the complex sensory and responsive functions of human skin. For example, a pyramid pattern single-walled carbon nanotube (SWCNT) flexible tactile sensor array was integrated into the artificial sensory system to sense stimuli^[89]. The artificial sensory system, which integrated multiple tactile sensors, an oscillator, an I-V transimpedance amplifier (TIA), and multi-gate synaptic transistors, was developed; it could sense, process the external tactile stimulus, and trigger the actuator [Figure 4H]. This comprehensive system effectively mimicked somatosensory feedback functions, demonstrating its ability to imitate the biological somatosensory system.

Figure 4. Neuromorphic computing in skin-inspired sensor systems using transistors. (A) Schematic of the bioinspired artificial tactile afferent nervous system. Reproduced with permission. Copyright 2021^[81], Nano Energy; (B) Structure of an artificial tactile afferent neuron. Reproduced with permission. Copyright 2024^[82], Science; (C) Stretchable transistor manufactured based on photolithography technology; (D) Illustrations of an active-matrix sensor array. (C and D) Reproduced with permission. Copyright 2024^[83], Nature; (E) Low-voltage-driven artificial soft e-skin system. Reproduced with permission. Copyright 2023^[84], Science; (F) Schematic of NeuroMAT; (G) The EPSC of NeuroMAT-integrated robotic hand to grip an object. (F and G) Reproduced with permission. Copyright 2023^[88], Science Advances; (H) An artificial neuromorphic somatosensory system. Reproduced with permission. Copyright 2022^[89], NPJ Flexible Electronics. EPSC: Excitatory postsynaptic current.

Compared with transistors, memristors have shown more advantages in biological signal processing. Memristors enable high-density and parallel integration through crossbar array architectures for efficient and scalable neural network implementations. Their nonvolatile nature ensures data persistence without continuous power, reducing overall energy consumption and enhancing system reliability. They also support CIM operations, effectively mitigating the von Neumann bottleneck by performing calculations directly within the memory matrix, which accelerates processing speeds and lowers energy costs. Additionally, memristors exhibit low power consumption and minimal heat generation, making them an ideal option for large-scale, energy-efficient neuromorphic systems. They provide flexible synaptic behavior through tunable resistance states that can closely mimic the dynamic synaptic connections of biological brains, which enhances learning and memory capabilities. The simplified device structure of memristors facilitates easier manufacturing and cost-effective production, while their high scalability and integration compatibility with existing semiconductor technologies enable seamless incorporation into advanced neuromorphic architectures as well. For example, the coexistence of negative differential resistance (NDR) and resistance switching (RS) behaviors in a memristive device with Ag/ZnOx/TiOy/indium tin oxide (ITO) structure was fabricated to sense temperature and information storage^[90]. This multifunctional device leveraged the dual properties of NDR and RS to enhance both sensing accuracy and data retention capabilities. Besides inorganic oxides, polymer-based biomaterials were also used as active layers in the production of memristors. A polyurethane sponge sensor pressure sensor and chitosan-based memristor were combined to realize the perception of external signals and the functions of perception, memory, and data processing by adjusting the synaptic weight^[91] [Figure 5A]. The circuit connection diagram of single pressure and a memristor is shown in Figure 5B, and the I-V curves, current on/off ratio, and repeated test performance of chitosan-based memristor are shown in Figure 5C. A pressure sensory array and memristive spiking neuron array were integrated to acquire stimulation, process and recognize the pressure signals. The instantaneous spike frequency maps obtained by the memristive spiking neural array were used to train the SNN, and the neural network successfully extracted and learned the spatial features of spike frequency maps for different letters, achieving a classification accuracy of approximately 98%^[92] [Figure 5D-F]. The integration of a pressure sensor and NbOx-based memristor is illustrated in Figure 5G, using the circuit diagram of spiking nociceptor by switching high resistance state (Roff) and low resistance state (Ron) to generate spiking signals [Figure 5H], an intelligent sensor is able to detect harmful stimuli^[93] [Figure 5I]. Besides this, the memristor with a Ti/Pt/NbOx/Ti/Pt structure integrated the pressure sensors to act as multisensory perception. A 3 × 3 array of multimode-fused spiking neuron was fabricated [Figure 5J], and the circuit diagram of spiking neuron was used to generate the spiking signals [Figure 5K]. Through this method, temperature and pressure analog information were fused into one spike train; a SNN algorithm was simulated to recognize objects^[94] [Figure 5L].

Figure 5. The artificial tactile sensory neural system. (A) The architecture and SEM of memristor; (B) The integration of pressure and memristor; (C) The performance of a chitosan-based memristor. (A-C) Reproduced with permission^[91]. Copyright 2024, Advanced Science; (D) Architecture of the artificial tactile sensory neural system; (E) I-V curves for the memristor; (F) pressure sensors array to acquire different letters via SNN algorithm. (D-F) Reproduced with permission^[92]. Copyright 2022, Advanced Electronic Materials; (G) Scheme of artificial spiking nociceptor; (H) “Threshold” feather of neural synapse at different intensities of force; (I) Output spikes at different pressure thresholds. (G-I) Reproduced with permission^[93]. Copyright 2022, IEEE Electron Device Letters; (J) Multimode-fused perception system to tactile perception; (K) Schematic circuit diagram of the multimode-fused perception system; (L) A SNN classifier for training the spiking signals. (J-L) Reproduced with permission^[94]. Copyright 2022, Advanced Materials. SEM: Scanning electron microscope; SNN: spiking neural network.

NEURAL NETWORK ALGORITHMS

AI learning techniques have been integrated into sensory systems to enhance their ability to adapt, learn, and make accurate predictions based on the input data. AI learning can be broadly categorized into two main approaches: global error-driven and local neuroscience-inspired learning. The global error-driven learning approach focuses on minimizing a global error metric across the entire model. Techniques such as gradient descent are commonly used, where the system calculates the difference between predicted and actual outputs and adjusts weights across the entire network to reduce error. Deep learning models, such as neural networks trained with backpropagation, are prime examples of global learning. Although these models often require extensive datasets and substantial computational resources, they can achieve high accuracy in complex tasks.

Inspired by the human brain, local learning focuses on updating the model through mechanisms that utilize information available at individual neurons or specific layers. Local learning rules, such as Hebbian learning (cells that fire together, wire together) and STDP, adjust connections based on local activity and temporal relationships. This approach is biologically plausible, resulting in models that are more interpretable and energy efficient. However, achieving performance comparable to error-driven methods may require the development of new architectures and algorithms. While error-driven methodologies prioritize performance optimization and high accuracy, local learning paradigms draw inspiration from the mechanisms of biological learning systems, aiming for enhanced efficiency and robustness. An intriguing strategy is the integration of these two approaches within a single network. For example, a spike-based hybrid plasticity model using a brain-inspired meta-learning paradigm and a differential spiking dynamics model with parameterized local correlation-driven learning was established with significantly higher performance in several image classification tasks (99.50% vs. 95.00%), such as on the modified national institute of standards and technology (MNIST) dataset^[95].

The common algorithms used in skin-like sensors are ANN and SNN. ANN algorithms can be learned by updating the weighted value, with the signals of neural networks typically being continuous values. ANN algorithms include deep learning and CNNs. The primary distinction between ANN and SNN lies in signal processing: ANN computations involve continuous values, whereas SNN inputs consist of spikes with timing information^[96] [Figure 6A]. In biological neurons, the flow of Na⁺ (sodium) and K⁺ (potassium) ions across the neuronal membrane is fundamental for action potential generation - brief electrical impulses that constitute neuronal pulses^[97]. When a neuron is stimulated, Na⁺ channels open, allowing Na⁺ ions to influx, which depolarizes the membrane potential. If the depolarization reaches a certain threshold, an action potential is triggered, propagating the pulse along the axon. Subsequently, K⁺ channels open to repolarize the membrane, restoring the resting potential. This ionic exchange and the resulting action potentials are critical for neuronal communication and synaptic plasticity, the ability of synapses to strengthen or weaken over time based on activity levels. The concept of membrane potential is directly modeled to emulate the electrical dynamics of biological neurons. Each artificial neuron within a SNN maintains an internal state that represents its membrane potential, which is analogous to the membrane potential in biological neurons. The illustration of biological synaptic plasticity mechanisms and neuronal dynamics is shown in Figure 6B. The leaky LIF model is one of the commonly used neuron models in SNN; the model imparts SNNs with brain-like neural dynamics, making them more energy-efficient and capable of higher computational performance, closely resembling the functioning of the biological brain. Combined with hardware implementations such as memristors, LIF neurons play a crucial role in SNNs for advanced neuromorphic computing. Unlike traditional CMOS applications, LIF neuron was implemented by a memristor. To improve the accuracy of SNNs, hybrid learning approaches that combine local STDP learning with global error-driven learning have been employed^[98] [Figure 6C]. STDP is a local learning rule where the strength of synaptic connections is adjusted based on the relative timing of pre- and postsynaptic spikes. This biologically inspired mechanism enables SNNs to learn temporal patterns without the need for global error gradients. In neuromorphic systems, volatile and symmetrically threshold-switching VO₂ memristors have been employed in the neuromorphic system, leveraging their dynamic behavior to create compact LIF and adaptive LIF neurons. These neurons were integrated into a long short-term memory SNN, enabling effective decision-making and accurate analysis of physiological data^[99] [Figure 6D]. Additionally, LIF neurons could be implemented using diffusive memristors. For example, an 8 × 8 1T1R memristive synapse crossbar was integrated with eight diffusive memristor-based artificial neurons [Figure 6E]. By integrating nonvolatile memristive synapses with diffusive memristors, a fully memristive ANNs was implemented for pattern classification using unsupervised learning method^[100] [Figure 6F]. Furthermore, the unsupervised learning method in a probabilistic neural network that utilized metal-oxide memristive devices as multi-state synapses was implemented to effectively cluster and interpret complex and unlabelled data in real-world scenarios^[101]. As the spiking deep learning paradigm gained momentum, however, traditional programming frameworks struggled to meet the growing demands for automatic differentiation, parallel computation acceleration, and efficient processing and deployment of neuromorphic datasets. To address these limitations, the SpikingJelly framework was proposed to optimize the performance and scalability of SNNs^[102] [Figure 6G]. In contrast to previous works, CNNs have been fully implemented by using eight 2,084-cell memristor arrays, demonstrating scalability to other ANNs and establishing a viable memristor-based non-von Neumann hardware solution for deep neural networks and edge computing^[103] [Figure 6H].

Figure 6. AI learning in neuromorphic computing. (A) A comparative illustration of ANN and SNN neuron models. Reproduced with permission^[96]. Copyright 2022, Advanced Electronic Materials; (B) A depiction of the processes underlying biological synaptic plasticity and the patterns of neuronal activity. Reproduced with permission^[95].Copyright 2022, Nature Communications; (C) Supervised global learning and local STDP unsupervised learning for digit classification. Reproduced with permission^[98]. Copyright 2019, Nature; (D) Schematic of the VO₂ memristor-based adaptive LIF neuron and operation flow. Reproduced with permission^[99]. Copyright 2023, Nature Communications; (E) The integrated memristive neural network; (F) Input patterns via a 4 × 4 input array with triangular waveform stimulation. (E and F) Reproduced with permission^[100]. Copyright 2018, Nature Electronics; (G) Architecture of the neuromorphic framework. Reproduced with permission^[102]. Copyright 2023, Science Advances; (H) Memristor-based hardware system with reliable multi-level conductance states and on-chip training. Reproduced with permission^[103]. Copyright 2020, Nature. AI: Artificial intelligence; ANN: artificial neural network; SNN: spiking neural network; STDP: spike-timing-dependent plasticity; LIF: leaky integrate-and-fire.

The integration of memristors, transistors, and flexible sensors in neuromorphic computing systems has the potential to revolutionize both AI and biological signal processing. By combining the unique properties of memristors - such as their ability to “remember” electrical states - with advanced algorithms such as ANN and SNN, researchers are creating more efficient, adaptive, and intelligent systems. The development of system-on-chip (SOC) solutions will further enhance the practicality and energy efficiency of neuromorphic computing, enabling a wide range of applications in fields such as robotics, healthcare, Internet of Things (IoT), and environmental monitoring.

ON-CHIP NEUROMORPHIC COMPUTING SYSTEM

Traditional analog computing systems suffer from limitations due to environmental noise and high energy consumption. By integrating neuromorphic computing SOC, these challenges can be mitigated. On-chip learning and tightly coupled analog computing frameworks enable the design of more compact, energy-efficient systems. Neuromorphic SOC can perform real-time processing tasks such as pattern recognition, sensory input analysis, and adaptive decision-making. The on-chip integration of learning algorithms and neuromorphic hardware is a promising direction for future developments in skin-inspired neuromorphic sensors. This emerging technology has made significant strides in enhancing adaptive sensing and processing capabilities in intelligent systems. By mimicking the sensory mechanisms of human skin and the information-processing capabilities of the nervous system, it aims to enable more sophisticated real-time interactions and decision-making in complex environments. By combining skin-inspired sensors with neuromorphic computing, instant multimodal sensing (e.g., touch, temperature, pressure) can be achieved. Leveraging the principles of the biological nervous system, these technologies enable the processing of complex sensory data and its wireless transmission. This capability allows for the provision of highly sensitive, real-time feedback, thereby enhancing the system’s ability to rapidly adapt and respond to dynamic environmental changes^[104,105] [Figure 7A and B]. Integrated systems that combine skin-inspired sensors and neuromorphic computing can generally be categorized into the following areas.

Figure 7. A schematic diagram of a system combining skin-inspired sensors with neuromorphic computing, including electronic skin, smart prosthetics, wearable devices, and robotic skin. (A) A schematic diagram of Radial arterial pulse monitoring system and the configuration of the pulse monitoring system. Reproduced with permission^[104]. Copyright 2019, Advanced Healthcare Materials; (B) Photographs showed a flexible integrated sensing-feedback system. Reproduced with permission^[105]. Copyright 2024, Nature Communications; (C) A Kind of schematic structure of the multifunctional e-skin. Reproduced with permission^[106]. Copyright 2022, Advanced Science; (D) Schematic of multifunctional self-healable electronic skin system consisting of a strain sensor with a capacitive structure. Reproduced with permission^[107]. Copyright 2018, Nature Nanotechnology; (E) Schematic diagram of e-skin with hair permeability and pressure-temperature perception. Reproduced with permission^[108]. Copyright 2022, ACS Applied Materials & Interfaces; (F) Schematic diagram of a wearable flexible sweat-sensing platform, utilizing e-skin materials for both physical and chemical sensing, designed for real-time multiplexed perspiration sweat analysis. Reproduced with permission^[110]. Copyright 2019, Accounts of Chemical Research; (G) Nerve-inspired artificial sensor for robotic integration.Reproduced with permission^[112]. Copyright 2024, Nature Communications.

E-skin, designed to mimic the sensory functions of human skin, enables the detection of various stimuli such as pressure, temperature, humidity, and touch. By integrating piezoelectric and thermoelectric materials, it can sense multiple tactile signals. Furthermore, by combining neuromorphic computing, e-skin can simulate human skin’s perception and response mechanisms, allowing for adaptive sensing and intelligent responses. Researchers have reported a multifunctional e-skin that combines multiple sensory functions with intelligent robot control. Through this skin, robots could interact with humans safely and accurately^[106] [Figure 7C]. Other researchers have observed self-repair in conductive nanostructures and dynamic cross-linked polymer networks, enabling the integration of interconnected sensors and lighting devices into a single multifunctional system. It was the first self-repairing, stretchable multi-component e-skin, offering new directions for e-skin development^[107] [Figure 7D]. Additionally, researchers have developed multifunctional e-skin via in-situ 3D printing, capable of hair growth with high precision and consistency. It included temperature, pressure, and tactile sensor arrays that accurately recognize various stimuli at different positions^[108] [Figure 7E].

The combination of skin-inspired sensors with neuromorphic computing enables prosthetics to respond more naturally and precisely. Researchers have developed a hand posture recognition system using surface electromyographic signals from the flexor and extensor muscles, allowing precise control of bionic hands. A dual-channel surface electromyography (EMG) signal recognition system could identify hand postures and control the corresponding gestures of a custom-built bionic hand^[109]. Combining skin-inspired sensors with neuromorphic computing technology also shows significant potential in wearable devices, particularly in health monitoring. These smart health devices, including smartwatches and health-monitoring patches, use neuromorphic algorithms to monitor real-time physiological signals and predict health conditions. By modeling and predicting real-time signals through SNNs, these devices can detect early signs of health issues. For instance, a SNN can predict a user’s exercise or fatigue state based on heart rate patterns, even detecting potential health risks in advance. Researchers have developed a wearable, flexible sweat-sensing platform for real-time, multi-channel sweat analysis. Integrated ion electrophoresis modules enabled automatic and programmatic sweat extraction. These sensors held great potential for monitoring dehydration, diagnosing cystic fibrosis, drug monitoring, and non-invasive blood glucose detection^[110] [Figure 7F].

Robotic skin integrates skin-inspired sensors with neuromorphic computing, allowing robots to “feel” and perceive their environment. Using SNNs, the system processes sensory signals and quickly adjusts the robot’s actions based on tactile inputs such as touch pressure or object contact. A recent study designed a flexible tactile sensing array based on capacitive mechanisms, with a polyethylene terephthalate (PET) substrate, PDMS dielectric layer, and convex-concave contact layers. The PDMS layer featured four types of microstructures, enabling tactile feedback control for robotic arms, showing potential for obstacle avoidance^[111].

The multimodal perception system is also a type of integrated system that combines skin-inspired sensors and neuromorphic computing. These systems can simultaneously process tactile information, auditory information, and simulate the comprehensive processing capabilities of human senses. Data from sensor arrays, including tactile and sound sensors, are processed through neuromorphic computing units (e.g., SNN). The system can perceive and learn based on the combined patterns of touch and sound, enabling more complex tasks such as intelligent voice control and touch-sound synchronization. Researchers have combined pressure-activated organic electrochemical synaptic transistors with artificial mechanoreceptors to detect the directional movement of object vibrations. By processing spike-encoded signals through a deep learning model, the spatiotemporal characteristics of tactile patterns were effectively distinguished, achieving high recognition accuracy^[112] [Figure 7G]. Another study proposed a multimodal bionic tactile sensor module capable of sensing surface geometries, forces, vibrations, and temperatures, exploring and mimicking human tactile perception capabilities^[113].

In recent years, significant progress has been made in the development of multimodal sensing systems. At the same time, such integrated systems often exhibit superior performance. We often focus on the system’s skin-inspired neuromorphic sensors, materials, sensitivity or accuracy, power consumption, response time, and their task to assess the system’s specific performance and the latest developments. An overview is provided in Table 1. However, as integrated systems, they still face several key limitations and challenges. Multimodal sensing systems typically require many sensors and significant computational resources, resulting in high energy consumption. Furthermore, the manufacturing costs of current sensing systems remain elevated, and their production processes are often complex, which presents a barrier to large-scale deployment and widespread application. For e-skin and robotic skin, long-term wearability and comfort still need further improvement. Specifically, in terms of simulating human skin, there is still room for enhancement in softness, breathability, and overall comfort. Integrated systems require larger sensor arrays, but these large arrays are prone to crosstalk, degraded performance, and difficulties in functional integration. Current research systems are typically small-scale, and substantial advancements will require further breakthroughs in materials, performance, and low energy consumption.

CONCLUSION AND OUTLOOK

Skin-inspired neuromorphic sensors showed great potential to revolutionize future robotics, healthcare, wearables, and smart textiles. The advantages of these systems, such as real-time responsiveness, adaptability, and energy efficiency, make this research area highly promising. Although significant progress has been made, some challenges remain in this research field, such as the scaling of sensor arrays, signal interference, neuromorphic SOC technologies, etc. These obstacles need to be addressed for the broader implementation of e-skin systems.

To tackle scaling and signal interference issues, future research should prioritize specific strategies aimed at enhancing sensor accuracy and the scalability of multimodal sensor arrays. One promising direction involves the development of advanced materials, such as flexible conductive polymers, dielectric elastomers, and novel nanomaterials. These materials can help create large-scale, flexible sensor arrays with high sensitivity and reliability. Furthermore, innovations in nanotechnology and 3D printing hold the potential to enable the scalable production of these sensor arrays, ensuring that they can conform seamlessly to a variety of surfaces while maintaining performance. Additionally, novel hybrid sensor modalities, where different sensors complement each other to detect various signals, can be designed and implemented to overcome signal interference. Such an approach will improve the overall reliability and accuracy of the sensor networks.

Parallel to sensor advancements, refining neuromorphic computational frameworks and addressing scalability and miniaturization challenges are pivotal for the progression of these technologies. Neuromorphic systems aim to emulate the efficiency and adaptability of biological neural networks, yet scaling these systems to handle the vast amounts of data generated by large-scale sensor arrays remains a significant hurdle. Future research should focus on optimizing neuromorphic architectures to support high-density sensor integrations without compromising performance. This includes the development of energy-efficient processing units, enhanced interconnects, and scalable algorithms capable of managing and interpreting complex multimodal sensory inputs. Additionally, overcoming miniaturization challenges through advanced fabrication techniques and the integration of novel materials can lead to more compact and efficient neuromorphic systems. Incorporating row-column scanning techniques, multifunctional reconfigurability, large-area sensing arrays, and decoupling algorithms will also be essential in addressing current limitations. These strategies will enable the creation of robust and versatile neuromorphic frameworks capable of supporting expansive and intricate sensor networks, ultimately advancing the capabilities of intelligent, bio-inspired systems.

Furthermore, the development of neuromorphic SOC technologies presents a promising direction by significantly reducing the area required for computation, lowering power consumption, and enhancing overall performance. Effective system integration involves the seamless combination of sensors, computational units, and communication interfaces into a unified platform. This integration is critical for the functionality and reliability of advanced neuromorphic technologies, as it ensures efficient data processing and real-time responsiveness. Neuromorphic SOCs can facilitate the compact and efficient deployment of large-scale sensor arrays by embedding processing capabilities directly within the sensor framework. This not only minimizes latency, but also reduces power, making the systems more sustainable and practical for widespread use. Future research should explore novel integration techniques, such as heterogeneous integration and 3D stacking, to optimize the performance and scalability of neuromorphic SOCs. One potential solution to enhance SOC performance is the consideration of multimodal sensing and neuromorphic device integration. By combining these technologies into a unified platform, we can achieve compact integration of sensor arrays and computational units, improving system functionality and responsiveness. This integration can be further optimized using laminated electronics, in which the devices (which cannot be fabricated using traditional CMOS technologies) can be realized by integrating different layers (wafers) of structures; each layer can be completed with standard CMOS fabrication techniques on a single wafer. Take energy storage devices as an example, one can first use semiconductor processes to fabricate electrodes and solid-state electrolyte structures on different wafers, then a novel wafer-scale energy storage chip can be realized through multi-wafer hybrid integration of the wafers with different structures. This approach enables the integration of high-performance sensors and processing units to facilitate the development of advanced multifunctional neuromorphic systems. Additionally, developing standardized interfaces and modular designs can promote interoperability and ease the integration of diverse components. By advancing neuromorphic SOC technology, researchers can achieve more compact, energy-efficient, self-powered, and high-performance intelligent systems, paving the way for their application in various fields, including healthcare, robotics, and wearable devices.

By addressing these challenges, the integration of skin-inspired sensors with neuromorphic computing can pave the way for the next generation of intelligent, adaptive systems. These systems will be capable of sensing, learning, and responding to their environments in ways that closely mimic human perception and action, thereby enabling more sophisticated and human-like interactions in various applications.