SKPNet: snake KAN perceive bridge cracks through semantic segmentation

2.1. Preliminary

The main architecture of our method is illustrated in Figure 1. The input first passes through a U-Net backbone to extract multi-scale features, which are then fused using a CWSA module 2.2. Subsequently, information is further extracted via DSC 2.3 with tubular offsets. Finally, the segmentation output is generated through a classification head composed of KAN 2.4 layers.

Figure 1. Flowchart of SKPNet.

Firstly, The U-Net with residual connections is adopted as the backbone of our model, where three stages of 2$$ \times $$ resampling facilitate the extraction of multi-scale information from the input. During the upsampling process, we adapt resampling parameters corresponding to those used in the downsampling stages, ensuring that the scale at each upsampling stage matches its corresponding downsampling stage. In each upsampling operation, residual connections are utilized to fuse information from the corresponding downsampling layers, which helps mitigate the loss of detailed information caused by downsampling. The specific process is given in

where $$ x_{1} \in \mathbb{R}^{3 \times H \times W} $$ represents the input image to be detected, and $$ Down(\cdot) $$ and $$Up(\cdot) $$ represent the downsampling and upsampling operations, respectively. $$Conv(\cdot) $$ and $$ Res(\cdot) $$ denote the convolution operation and residual connection, respectively. Subsequently, we design a CWSA module that can process multi-scale features and capture internal relationships using an attention mechanism. It accepts the results produced by each upsampling operation and utilizes self-attention to extract the correlations between each channel. This operation can integrate large-scale visual detail information with small-scale semantic information. This process can be given in

where $$ CWSA(\cdot) $$ represents the operation of CWSA, and $$ Concat(\cdot) $$ represents concatenation operations. Additionally, DSC is integrated to adaptively extract tubular cracks. It is equipped with an offset specially designed for tubular structures, enabling the convolutional kernel to dynamically deform according to the input, which helps preserve the integrity of the information. This will be discussed further in Section "DSC". DSC can be expressed as

where $$ DSC(\cdot) $$ represents the operation of the DSC module. Finally, a classification head composed of KAN layers is used to classify each pixel, outputting the final detection results. The final outputs can be given in

where $$ KANs $$ represents operations in the classification head.

2.2. CWSA

Given an input feature map $$ X \in \mathbb{R}^{B \times C \times H \times W} $$, where $$ B $$ indicates the batch size, $$ C $$ stands for the number of channels, $$ H $$ represents the height, and $$ W $$ denotes the width; the channel-wise attention is computed using self-attention as follows. First, the input tensor is reshaped into a sequence of channel descriptors by applying global average pooling as

Next, the query, key, and value matrices for self-attention are computed as

where $$ W_q, W_k, W_v \in \mathbb{R}^{C \times C} $$ are learned weight matrices. Then, the attention scores are computed by calculating the similarity between the query and key matrices using dot-product attention as

where $$ A \in \mathbb{R}^{B \times C \times C} $$ represents the attention map between channels. Finally, the output is computed by multiplying the attention map with the value matrix as

where $$ X' \in \mathbb{R}^{B \times C} $$ is the reweighted channel descriptor. The attention weights are then applied back to the original feature map by

where $$ \tilde{X} \in \mathbb{R}^{B \times C \times H \times W} $$ is the final channel-attended feature map. CWSA is incorporated to effectively leverage the multi-scale information output from the U-Net model. This integration allows the network to dynamically reweight channel features, enhancing ability of the model to focus on critical information while suppressing noise. By capturing long-range dependencies and emphasizing relevant features across different scales, the CWSA mechanism significantly improves the accuracy and robustness of crack detection in complex images.

2.3. DSC

In this study, DSC^[19] is introduced to improve boundary delineation and feature extraction. Given an input tensor $$ {\mathbf X} \in \mathbb{R}^{B \times C \times H \times W} $$, where $$ B $$ is the batch size, $$ C $$ is the number of channels, and $$ H \times W $$ represents the spatial dimensions. DSC dynamically adjusts the offsets of convolutional kernels based on the geometric properties of the cracks as shown in Figure 2. The left panel illustrates standard convolution, where fixed kernel positions are applied, represented by green dots. The right panel demonstrates DSC, where the white dots represent the original kernel positions, and the green dots indicate the adjusted positions after applying offsets. This tubular offset mechanism allows DSC to better capture the detailed information of tubular structures.

Figure 2. Comparison between standard convolution and DSC. DSC: Dynamic snake convolution.

This dynamic adjustment allows the convolution operation to align more effectively with irregular shapes, such as the edges and contours of cracks in concrete surfaces. Unlike standard convolutions, which apply fixed filters across spatial dimensions, DSC modifies the receptive field of kernels by learning deformation offsets during training. This leads to more precise crack boundary detection by capturing subtle variations in crack shapes and orientations. Mathematically, for each convolutional operation at spatial location $$ (i, j) $$, the kernel $$ {\mathbf K} $$ is adjusted with learned offsets $$ \Delta(i, j) $$, such that the convolution result at $$ (i, j) $$ becomes

where $$ \Delta m, \Delta n $$ represent the dynamically learned offsets that adjust the position of the kernel. By incorporating these dynamic offsets, DSC enhances the ability of the model to detect complex crack structures, ensuring more accurate segmentation and classification results.

2.4. KAN: Kolmogorov-Arnold network

In the final stage of our model, the traditional MLP is replaced with a KAN^[20] to leverage its superior function approximation capabilities. The MLP is commonly structured as a series of matrix multiplications and nonlinear activation functions, defined as

where $$ {\mathbf Z} $$ is the input, $$ W_k $$ represents the weight matrices, and $$ \sigma $$ denotes a nonlinear activation function applied between each layer. The symbol $$ \circ $$ represents the composition of matrix multiplication and activation functions. While effective, this architecture may have limitations in capturing highly complex dependencies due to its fixed linear transformations between layers. In contrast, the KAN leverages the Kolmogorov-Arnold representation theorem, which asserts that any multivariate continuous function can be represented as a finite composition of continuous univariate functions. The KAN is formulated as

where each $$ \Phi_k $$ is a set of univariate continuous functions defined as

where $$\phi_{q, p}$$ represents the learned univariate mappings between input and output dimensions. These univariate mappings are implemented through splined activation functions.

where

and

where, $$ w_b $$, $$ w_s $$, and $$ c_i $$ are all trainable parameters. During the initialization phase, $$ w_s $$ is set to 1 and $$ \text{spline}(x) \approx 0 $$, while $$ w_b $$ is initialized according to the Xavier initialization method. This structure enables KAN to capture more complex and nonlinear dependencies between input features compared to the fixed transformations of an MLP. By replacing the MLP with KAN, we introduce a more flexible framework capable of approximating complex functions, which is particularly beneficial for tasks such as crack segmentation. The enhanced ability to capture subtle relationships in the data results in improved segmentation accuracy and robustness, especially in handling irregular and intricate patterns present in crack structures.

2.5. Experimental configuration

2.5.1. Dataset

The dataset employed for this experiment is derived from the SDNET2018^[21] collection. SDNET2018 offers over 56,000 images depicting concrete structures, including bridge decks, walls, and pavements, intended to facilitate the development, validation, and testing of algorithms for detecting concrete cracks. To evaluate the performance of our optimized model, 500 images were selected from the dataset, specifically targeting bridge crack surfaces. These images have a resolution of 512 × 512 pixels and contain three color channels. The pixel-level annotations were carefully generated using the LabelMe tool, with white pixels indicating crack areas and black pixels marking the background. For data augmentation, horizontal flipping and rotations were applied. After flipping, the images were rotated by 90 and 180 degrees, ensuring the corresponding labels underwent the same transformations to preserve alignment. As a result of data augmentation, the dataset size was expanded to 1,600 images. The dataset was then divided into two portions, with 70% used for training and 30% reserved for testing. As shown in Figure 3, this displays some samples from the dataset. The sample images in Figure 3 show the range of crack widths, surface conditions, and other environmental factors, such as color shifts, coarse cracks, and fine cracks. Combined with data augmentation, these images effectively reflect the variety of crack images encountered in real-world environments.

Figure 3. Display of samples from the dataset.

In addition to SDNET2018, we utilized the newly developed CrackVision dataset^[22], which combines 12,000 images derived from 13 publicly available crack datasets. This unified dataset addresses the limitations of small-scale datasets and inconsistent annotation standards by employing consistent image processing techniques to produce standardized masks. Furthermore, to mitigate the issue of class imbalance commonly encountered in crack datasets, images with fewer than 5,000 crack pixels were excluded. Data augmentation techniques, including Gaussian noise addition and random rotations, were applied to further diversify the dataset.

2.5.2. Training settings

The network model described in this paper was implemented using PyTorch. Table 1 provides a detailed summary of the experimental setup. During training, the Adam optimizer is employed to fine-tune the weights of network. The batch size was set to 8, and the learning rate was initialized at 1e-4. We recorded the loss changes during training, as shown in Figure 4. After 150 epochs, the model stabilized, and performance on the test set no longer improved. Therefore, we selected 150 epochs as the performance recording point.

Table 1

Experimental setup configuration

Type	Statement
Operating system	Ubuntu 22.04
RAM	64 G
CPU	Intel i9-13900k
GPU	NVIDIA GeForce RTX 4090
CUDA version	12.4
Pytorch version	2.4.1
Python version	3.9.19

Figure 4. Visualization of training process. (A) Losses during training; (B) mIoU during training. mIoU: Mean intersection over union.

3.1. Ablation study

The effectiveness of different modules in our proposed model was investigated through ablative experiments. The steps were as follows: (1) Experimenting with the model using only the CWSA mechanism, denoted as A1; (2) Incorporating the DSC with tubular offsets, denoted as A2; (3) Adding the KAN structure in the classification head, denoted as A3; (4) The final configuration includes all three components: CWSA, DSC, and KANs, denoted as A4. The results of the ablative experiment evaluation metrics are presented in Table 2. It provides a detailed comparison of performance of the model with different combinations of the proposed modules. The inclusion of the CWSA mechanism alone (A1) resulted in an mIoU of 0.857. The CWSA module, integrated with the U-Net architecture, helps to fuse multi-scale information, enabling the model to capture both fine and coarse details of the cracks more effectively. Adding the DSC module (A2) improved the mIoU to 0.869. DSC is particularly effective for capturing tubular crack structures, as it adapts well to the deformable, continuous nature of cracks, enhancing the model's ability to detect crack continuity and handle variations in crack shapes. Incorporating the KAN structure (A3) yielded an mIoU of 0.846. KAN boosts the model's expressive power, allowing it to better handle complex crack patterns and improve generalization, particularly in challenging conditions. The final configuration, which includes all three components (A4), achieved the highest mIoU of 0.877. Compared to the baseline value of 0.833, the addition of each component has progressively improved the model's performance, demonstrating the complementary strengths of the CWSA, DSC, and KAN modules in enhancing crack detection accuracy.

Additionally, Figure 5 illustrates the visual results of our ablation study, specifically comparing the performance with and without the DSC module. The top row shows the crack detection results without the DSC module, where the cracks appear less continuous and more fragmented. In contrast, the bottom row demonstrates the results with the DSC module, where the cracks are detected with greater continuity and accuracy. These visual results corroborate the quantitative improvements shown in Table 2, highlighting the significant impact of the DSC module on enhancing crack detection performance.

Figure 5. Comparison of ablation experiments for the DSC module. DSC: Dynamic snake convolution.

3.2. Comparative performance analysis of different models

In the field of crack detection, we conducted a comparative analysis of our proposed method against several classical approaches. The models compared include: (1) U-Net^[23], a symmetrical network that employs stacking operations; (2) FCN^[24], a semantic segmentation network that can handle input images of any size; (3) PSPNet^[25], an image segmentation network based on spatial pyramid pooling and spatial pyramid attention mechanisms; (4) AFSM-Net^[26], which synergistically integrates Transformer and attention mechanism modules, building upon the DeepLabV3+ network model; and (5) Adaptive^[27], a traditional algorithm for crack detection in digital images. During the training of these networks, the fluctuations of mIoU for each network are documented. As depicted in Figure 6, our proposed SKPNet model demonstrates a distinct advantage over U-Net, FCN, PSPNet, and the traditional adaptive threshold method. Our model exhibited superior crack segmentation accuracy and excellent continuity in segmenting consecutive lines compared to the other methods. U-Net, while effective, lacks precision in segmenting fine details. FCN can approximate the contours of cracks but suffers from poor segmentation continuity. PSPNet, despite extracting crack information from multiple scales, is affected by background noise, leading to compromised segmentation accuracy. The traditional adaptive threshold method is the least effective, as it struggles to capture the semantic information of images and faces challenges in integrating higher-level semantic understanding.

Figure 6. Results of different methods in the test dataset.

Table 3 provides a detailed comparison of experimental metrics for different methods, including mIoU, F1-score, Precision, Recall, and the number of parameters. Each result in the table represents the average performance obtained from multiple experiments. Among the deep learning network methods, FCN exhibits the poorest performance with an mIoU of 0.507. There is a significant performance gap between traditional methods and deep learning approaches. From Table 3, it can be observed that our modified SKPNet algorithm outperforms other compared algorithms in terms of accuracy-related metrics, achieving an mIoU of 0.877, F1-score of 0.865, Precision of 0.843, and Recall of 0.889, despite having a moderate parameter count of 31.15M. Furthermore, SKPNet demonstrates efficient inference performance, with an average processing time of 4.2 ms for a 512 × 512 image on a GPU. Moreover, SKPNet achieves 52.424 GFLOPS, representing a modest 1.28% increase compared to U-Net (51.762 GFLOPS), while demonstrating a substantial improvement in segmentation performance, with an mIoU of 0.877 compared to 0.832. This indicates that the improved SKPNet algorithm enhances crack detection accuracy within an acceptable parameter range, enabling the network to better fulfill the task of bridge crack detection. The results of experiments on the CrackVision test dataset are presented in Table 4. Among the compared methods, SKPNet demonstrates superior performance, achieving the highest IoU (0.660) and F1-score (0.774). This indicates that SKPNet consistently outperforms baseline methods, including FCN, U-Net, DeepCrack^[28], SegFormer^[29], HrSegNet^[30], Hybrid-Segmentor^[22].

Figure 7A illustrates the performance comparison of various segmentation models, including SKPNet, FCN, holistically-nested edge detection (HED)^[31], CCS-Net^[32], and U-Net, across multiple epochs. SKPNet, the proposed method, consistently demonstrates superior performance throughout training. At early epochs, such as epoch 20, SKPNet achieves an mIoU of 0.780, significantly outperforming the baseline models. As the training progresses, SKPNet further improves, reaching an mIoU of 0.856 at epoch 120, the highest among all compared methods. While U-Net and CCS-Net show competitive results, with mIoUs of 0.805 and 0.810 respectively at epoch 120, they still lag behind SKPNet. On the other hand, HED and FCN, despite showing gradual improvements, exhibit consistently lower performance. These results clearly demonstrate the effectiveness and robustness of SKPNet in segmentation tasks, particularly in capturing detailed structural features. Figure 7B illustrates the performance of SKPNet on the test set in terms of F1-score, Precision, and Recall over 200 epochs. The results demonstrate a consistent improvement during the initial training phase, with all three metrics gradually stabilizing as training progresses. Notably, the F1 Score exhibits steady growth, reaching a plateau after approximately 150 epochs, which indicates a balance between Precision and Recall. This suggests that SKPNet maintains robust generalization capabilities on unseen data while achieving high segmentation accuracy.

Figure 7. Model performance on the test dataset at different epochs. (A) mIoU performance of different methods at different epochs; (B) Performance of SKPNet on the test dataset at different epochs. mIoU: Mean intersection over union.

Additionally, we evaluated the performance of our model (SKPNet) and the baseline model (U-Net) on more challenging test cases^[33], as shown in Figure 8. Panels (a) and (b) depict scenarios with complex textured backgrounds and strong background interference, while (c), (d), and (e) illustrate examples involving fine cracks and severe illumination or shadow interference. From the output results of U-Net, it is evident that under strong background interference, its detection results are prone to noise, leading to false positives. For instance, in (a), U-Net misclassifies the shadowed region at the top as part of the crack, and in (d), it fails to detect cracks located in shadowed areas. In contrast, our model, SKPNet, demonstrates a higher accuracy in extracting crack regions and performs more robustly under challenging conditions with background interference and shadow effects. These experimental results further validate the robustness and superiority of SKPNet in handling complex environments.

Figure 8. Challenges in handling adverse cases for crack segmentation.

Authors' contributions

Methodology and writing - original draft preparation: Ruan, Y.

Methodology and reviewed the manuscript: Wang, D.

Data processing and figure plotting: Yuan, Y.

Commentary and critical review: Yang, X.; Jiang, S.

Availability of data and materials

The datasets supporting the findings of this study are submitted as Supplementary Materials along with the manuscript.

Financial support and sponsorship

This research was funded by the National Natural Science Foundation of China (Grant No. 62103068, Grant No. 52478141, Grant No. 62003063, Grant No. 62205039), Science and Technology Research Program of Chongqing Municipal Education Commission of China (Grant No. KJQN202100745, Grant No. KJZD-K202400709), and the Natural Science Foundation of Chongqing, China (Grant No. CSTB2022NSCQ-MSX1599).

Conflicts of interest

Wang, D. is Junior Editorial Board Member and Yang, X. is Editor in Chief of the journal Intelligence & Robotics. They were not involved in any steps of editorial processing, notably including reviewer selection, manuscript handling, or decision-making. The other authors declare that there are no conflicts of interest.

Ethical approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Copyright

© The Author(s) 2025.