A small-sample time-series signal augmentation and analysis method for quantitative assessment of bradykinesia in Parkinson's disease

1. INTRODUCTION

Parkinson's disease (PD) is the second most prevalent neurodegenerative disease, which affects 1%–2% of individuals above 65^[1]. The Global Burden of Disease Study 2016 reported that China has about 23% of the global PD patients^[2]. By 2030, this proportion will reach 50%^[3]. This substantial number of affected individuals will pose significant medical challenges and place a heavy economic burden on society.

PD typically presents with varying motor symptoms, including bradykinesia, tremors, rigidity, gait disturbance, and postural instability^[4,5]. As the most characteristic clinical symptom, bradykinesia manifests as a general slowness of movement, hesitations, and a reduction in movement amplitude or speed during continuous motion^[6]. According to the most recent clinical diagnostic criteria, the diagnosis of PD is based on the presence of bradykinesia plus at least one among tremor, rigidity, and postural instability, which makes bradykinesia the cornerstone of the disease^[7]. Therefore, accurate bradykinesia assessment can facilitate PD's clinical diagnosis and long-term monitoring. Currently, the clinical assessment of bradykinesia is mainly based on the Movement Disorders Society United PD Rating Scale part III (MDS-UPDRS-part III), which primarily evaluates the motor performance of the patient's upper and lower limbs, such as finger and toe tapping. The severity of bradykinesia is scored as 0-4 points, where 0 indicates normal, and 4 indicates severe condition^[8]. However, the scale rating relies on the doctor's judgment and clinical expertise, which may introduce a degree of subjectivity and variation among individuals^[9]. Besides, the manual evaluation assessments are based on an overall impression of movement, making it challenging to discern the minor differences^[10]. Therefore, developing an accurate and objective method for evaluating bradykinesia represents a significant challenge and a current area of focus in PD diagnosis and therapy.

In recent years, a variety of intelligent instruments, such as inertial sensing units, gyroscopes, and accelerometers, have been employed for the quantitative assessment of bradykinesia in PD^[11-15]. These instruments can collect movement data from patients with PD. However, their ability to directly capture movement features is not always reliable, and they may generate errors due to fusion algorithms. Motion capture systems employ markers on the patient's limbs to monitor movement. These markers are tracked by a specialized capture system that records their positions, enabling motion information collection. This technology outperforms other instruments in capturing accurate and direct movement data, which is vital for improving the precision of quantitative evaluations of bradykinesia in patients with PD.

Currently, researchers have utilized motion capture systems for quantitative assessments of bradykinesia in PD. Given that clinical data often comprises small samples, analysis of motion capture data has predominantly been manual feature extraction followed by traditional machine learning techniques. Das et al. have extracted features such as movement amplitude and average speed from gait and toe tapping tasks. Using support vector machine (SVM) methods, they differentiated between patients with mild and severe symptoms with an accuracy of about 90%^[10]. Wahid et al. used gait-based motion capture data to extract features such as stride length and gait time, applying methods such as random forest and SVM to distinguish between patients with PD and healthy controls, achieving an accuracy of 92.6%^[16]. However, these methods have limitations in uncovering latent features, and the classification accuracy depends on the choice of features. In recent years, deep learning networks, such as convolutional neural networks (CNN) and long short-term memory (LSTM) networks, have started to be applied in the field of quantifying bradykinesia in PD^[17,18]. Classification methods based on deep learning networks can overcome the limitations of manual feature extraction and learn low-dimensional feature representations but are more susceptible to the limitations posed by the amount of data compared to traditional machine learning methods. On the other hand, clinical data usually have the characteristic of small samples, and motion capture data are typical multidimensional time series signals. Since the classic data augmentation methods, such as slicing, jittering, and rotation, result in temporal information distortion, methods based on dynamic time warping (DTW) can address this issue by maintaining the temporal relationships of time series signals^[19,20].

In this paper, to address the characteristics of small sample sizes of clinical data, we design a network classification approach based on DTW data merge and attentional prototype networks (DTW-TapNet). Firstly, a DTW-based data merge method is employed for data augmentation. Secondly, random grouping is used for dimensionality reorganization of time series, followed by convolution operations to learn features from multivariate time series data. The approach then incorporates attention mechanism and prototype learning to optimize the distance of the class prototypes of time series, achieving a low-dimensional feature representation of the training set and thus reducing the dependency on data volume. Based on motion capture data collected from patients with PD and healthy controls during upper and lower limb movements, the proposed DTW-TapNet classification method aims to achieve an objective and accurate assessment of bradykinesia in PD.

2. METHODS

This paper designs a DTW-TapNet network based on DTW data merge and attentional prototype network, and the network structure is shown in Figure 1. The network structure includes DTW data merge, feature extraction, and attentional prototype network, which are discussed in the following sub-sections.

Figure 1. DTW-TapNet network structure diagram. DTW: Dynamic time warping.

2.1 Dataset

The study was approved by the local ethics committee of Tianjin Huanhu Hospital (No. 2019-56). All subjects provided written informed consent in accordance with the Declaration of Helsinki to participate in this study. The subjects in the dataset consisted of 36 patients with PD (27 male and nine female) and eight age-matched healthy controls (three male and five female).

The experimental paradigm consisted of finger and toe tapping tasks, which referred to the MDS-UPDRS and were commonly used to assess the bradykinesia in PD^[8,21,22]. For the finger tapping task, the subjects were instructed to rapidly and consistently tap the index finger against the thumb. For the toe tapping task, they were instructed to tap their toes on the ground. Moreover, the subjects were required to complete ten consecutive cycles for both tasks, and the score ratings were performed by a professional doctor.

The motion capture data were collected during the experiment with a sampling rate of 60 Hz, which consisted of the three-dimensional coordinates of the reflective markers. For the finger tapping task, two markers were placed at the tip of the thumb and the index finger, respectively. For the toe tapping task, the marker was placed at the toe. The marker location diagram is shown in Figure 2. Considering the use of hands and medication, multiple sets of motion capture data can be collected for each subject, and the data with missing values was removed due to the obstruction. In total, the sample sizes are 165 and 169 for the finger and toe tapping tasks, respectively. In addition, since patients with a rating of 4 could not complete the task, the final dataset contains the rating of 0, 1, 2, and 3. Specifically, the sample sizes of each rating are 33, 74, 44, and 14 for the finger tapping task and 31, 68, 48, and 22 for toe tapping. Since the sample sizes do not conform to the long-tail distributions, it is sufficient to validate the effectiveness of the proposed method.

Figure 2. Marker location of motion capture system in finger tapping (top) and toe tapping tasks (bottom).

For the raw data, a band-pass filter of 0.3-20 Hz was used to retain the main frequency band of the limb movement. After that, zero-padding was employed to increase the length of shorter data, while a proportional scaling method was applied to reduce the length of longer data. Thus, the data length was standardized to 500 data points. Finally, the data was standard deviation normalized and used as the input of the proposed network. In this paper, the hold-out method was employed to validate the effectiveness of the classification method, and the dataset was divided into training and test sets with an approximately 7:3 ratio. The datasets were augmented two times using the DTW data merge method for the network training, and the final sample size of the training set is 360 for both the finger and toe tapping tasks.

2.2 DTW-based data augmentation

This paper adopts the DTW data merge method to achieve data augmentation, which can address the problem of temporal information distortion by preserving temporal relationships during augmentation. It employs a DTW algorithm to obtain the optimal match between the two signals. This algorithm stretches or compresses the two signals to identify corresponding similar points. The set of these corresponding points is referred to as the optimal warping path [Figure 3].

Figure 3. Example of the optimal warping path.

For two signals, $$ X=(x_{1}, x_{2}, ..., x_{N}) $$ with $$ N $$ elements and $$ Y=(y_{1}, y_{2}, ..., y_{M}) $$ with $$ M $$ elements, the optimal warping path is obtained using the DTW algorithm as $$ W=(w_{1}, w_{2}, ..., w_{k}, ..., w_{L}) $$, where $$ w_{k}=(w_{k, i}, w_{k, j}) $$, $$ k=1, 2, ..., L $$, $$ i=1, 2, ..., N $$, and $$ j=1, 2, ..., M $$. $$ w_{k, i} $$ and $$ w_{k, j} $$ denote the corresponding points of the two signals. After obtaining the optimal warping path between the two signals, a random element $$ w_{r}=(w_{p, r}, w_{q, r}) $$ was selected where $$ r $$ is chosen from a Gaussian distribution $$ \mathcal N(\dfrac{L}{2}, \dfrac{L}{10}) $$. According to $$ w_{r} $$, the $$ X $$ and $$ Y $$ are sliced and concatenated to generate a new time series $$ C=(x_{1}, x_{2}, ..., x_{w_{p, r}}, y_{w_{q, r}}, ..., y_{N}) $$.

2.3 Feature extraction

Patients with PD typically exhibited decreased movement amplitude and velocity during continuous limb movements, which can be captured in the spatio-temporal domain of motion capture data. This paper uses the one-dimensional CNN to extract the spatial-temporal features of motion capture signals and characterize the movement differences in PD patients with varying degrees of bradykinesia and healthy controls. The motion capture data for the finger and toe tapping tasks have dimensions of six and three, respectively. Given the similarity between these two datasets, we have established a unified feature extraction network structure [Figure 4]. To capture the interactive features between the multivariate dimensions, the data were reorganized into three groups, each formed by randomly selecting two dimensions. Each of these groups is then fed into one-dimensional convolutional blocks. The global average pooling layer is used to extract different dimensional spatio-temporal features. Finally, the outputs of the pooling layer are concatenated, and the latent features are derived through a fully connected layer. The architecture of each one-dimensional convolutional block comprises a one-dimensional convolution layer, a batch normalization layer, and a rectified linear unit (ReLU). The last two convolutional blocks employ a parameter sharing mechanism to reduce the number of network parameters.

Figure 4. Feature extraction neural network structure diagram.

For the feature extraction neural network, the learning rate is set to $$ 5\times10^{-6} $$, and the training is terminated when the change in loss is less than $$ 1\times10^{-9} $$. In addition, the dropout and regularization methods are added to reduce the effect of overfitting. In this paper, the dropout parameter is set to 0.8, and the parameter of L2 regularization is set to 0.5.

2.4 Attentional prototype network

This paper uses an attentional prototype learning approach to train a distance-based loss function^[23]. The prototype learning method is suitable for datasets with small sample sizes and can alleviate overfitting. It establishes a prototype representation for each category, and the output category is determined by comparing the distance between the output features of the feature extraction neural network and the prototype representation [Figure 5]. The prototype representation for each category is computed as the weighted sum of the output features from the feature extraction neural network, utilizing all training samples belonging to that category, as given in Equation 1.

(1) $$ \begin{equation} \boldsymbol{c}_k=\sum\limits_i \boldsymbol{A}_{k, i}\boldsymbol{H}_{k, i} \end{equation} $$

Figure 5. Schematic diagram of prototype learning.

where $$ \boldsymbol{H}_{k, i} $$ is the element of the feature matrix for the $$ i $$-th sample in category $$ k $$, $$ \boldsymbol{A}_{k, i} $$ is the corresponding weight, and $$ \boldsymbol{c}_k $$ is the prototype representation for category $$ k $$. The weights for each category are learned using an attention mechanism, as given in Equation 2.

(2) $$ \begin{equation} \boldsymbol{A}_k=\operatorname{softmax}\left(\boldsymbol{w}_k^T \tanh \left(\boldsymbol{V}_k \boldsymbol{H}_k^T\right)\right) \end{equation} $$

where $$ \boldsymbol{w}_k $$ and $$ \boldsymbol{V}_k $$ are the training parameters of the attention model. After obtaining the prototype representation for each category, the distance between the features of the test set samples and the prototype representations for each category can be calculated. The prototype representation with the closest distance is then determined as the category to which the sample belongs, as given in Equation 3.

(3) $$ \begin{equation} d\left(f_{\Theta}(\boldsymbol{x}), \boldsymbol{c}_k\right)=\left\|f_{\Theta}(\boldsymbol{x})-\boldsymbol{c}_k\right\|^2 \end{equation} $$

Next, the softmax function is used to calculate the probability of the sample belonging to each category, as given in Equation 4.

(4) $$ \begin{equation} p_{\Theta}(y=k \mid \boldsymbol{x})=\frac{\exp \left(-d\left(f_{\Theta}(\boldsymbol{x}), \boldsymbol{c}_k\right)\right)}{\sum_i \exp \left(-d\left(f_{\Theta}(\boldsymbol{x}), \boldsymbol{c}_k\right)\right)} \end{equation} $$

Finally, the loss function is calculated, as given in Equation 5, and the Adam optimization algorithm is used to update network parameters to minimize the loss function.

(5) $$ \begin{equation} J(\Theta)=-\log p_{\Theta}(y=k \mid \boldsymbol{x}) \end{equation} $$

The hyperparameter settings for the DTW-TapNet network structure designed in this paper are presented in Table 1.

Table 1

Hyperparameter settings of the proposed network

Network name	Network layer	Parameters setting
		Upper limb	Lower limb
Feature extraction network	Conv1	Layers count 256, kernel size $$ (8, -) $$, stride $$ (1, -) $$	Layers count 256, kernel size $$ (8, -) $$, stride $$ (1, -) $$
	Conv2	Layers count 256, kernel size $$ (5, -) $$, stride $$ (1, -) $$	Layers count 256, kernel size $$ (5, -) $$, stride $$ (1, -) $$
	Conv3	Layers count 128, kernel size $$ (3, -) $$, stride $$ (1, -) $$	Layers count 128, kernel size $$ (3, -) $$, stride $$ (1, -) $$
	Fc1	Unit count 300	Unit count 400
	Fc2	Unit count 100	Unit count 200
Attention network	AttFc1	Unit count 128	Unit count 128
	Output	Unit count 1	Unit count 1

3. RESULTS

This paper employs the confusion matrix (CM) to represent the classification results of the classification method. Each row in the matrix corresponds to the true class, and each column represents the predicted class. It can be given as Equation 6.

where $$ k_{ij} $$ denotes the number of samples that belong to class $$ i $$ and are classified as class $$ j $$. $$ N $$ represents the dimension of the matrix, corresponding to the total number of classes. This paper employs accuracy ($$ Acc $$), weighted precision ($$ WP $$), and kappa coefficient ($$ KC $$) as evaluation metrics to assess the overall performance of the classification method. The respective formulas are given as Equations 7-9.

where $$ S $$ denotes the total number of samples in the dataset.

The present study compares the proposed method with the three comparison methods, and confusion matrices for the four classification methods of finger and toe tapping tasks are shown in Figures 6 and 7, respectively. Classification performance metrics calculated from the CM are presented in Table 3. For the finger tapping task, the proposed method achieves an accuracy of 75.56%, a weighted precision of 76.77%, and a kappa coefficient of 0.64. In comparison, the CPN method achieves an accuracy of 73.47%, a weighted precision of 74.64%, and a kappa coefficient of 0.61. The LSTM method achieves an accuracy of 62.22%, a weighted precision of 63.64%, and a kappa coefficient of 0.45. The SVM method achieves an accuracy of 60.00%, a weighted precision of 60.40%, and a kappa coefficient of 0.41. Relative to the comparison methods, the proposed method shows improvements of 8.89%-15.56% in accuracy, 9.22%-16.37% in weighted precision, and 0.13-0.23 in kappa coefficient.

Figure 6. Confusion matrix of the four classification methods of upper extremity movements. CPN: Convolutional prototype network; LSTM: long short-term memory; SVM: support vector machine.

Figure 7. Confusion matrix of the four classification methods of lower extremity movements. CPN: Convolutional prototype network; LSTM: long short-term memory; SVM: support vector machine.

For the toe tapping task, the proposed method achieves an accuracy of 81.63%, a weighted precision of 84.65%, and a kappa coefficient of 0.73. In comparison, the CPN method achieves an accuracy of 66.67%, a weighted precision of 67.55%, and a kappa coefficient of 0.51. The LSTM method achieves an accuracy of 63.27%, a weighted precision of 62.48%, and a kappa coefficient of 0.46. The SVM method achieves an accuracy of 61.22%, a weighted precision of 60.92%, and a kappa coefficient of 0.45. The proposed method shows improvements of 8.16%-20.41% in accuracy, 10.01%-23.73% in weighted precision, and 0.12-0.28 in the kappa coefficient.

4. DISCUSSION

Motion analysis techniques offer a more objective and detailed view of the patient's motor abilities, which can detect subtle changes and nuances in movement that might not be noticeable to clinical observations. Previous studies have combined motion analysis techniques, such as optical motion capture systems and wearable sensors, with machine learning methods to assess the bradykinesia in PD^[28-30]. These studies derived the classic motion features and primarily achieved the distinction between healthy controls and PD patients. Compared with these studies, this study employed the deep learning-based method to learn the latent features and subdivide the bradykinesia in PD.

Clinical data often exhibit the characteristic of a small sample size. This paper employs a data augmentation method using DTW data merge. It involves finding the optimal match between the two time series and then concatenating them, ensuring that the concatenation points of the two time series have similar temporal characteristics. Besides, DTW data merge can enhance robustness against noise and temporal misalignment in time series^[31]. The prototype learning method learns low-dimensional feature representations for time series to reduce the data requirements^[23]. The attention mechanism, calculating the distance between prototypes and feature embeddings for classification, can also address the issue of small sample sizes^[32]. Comparative results with traditional classification methods demonstrate the effectiveness of the proposed method. Furthermore, compared to the CPN classification method, the result indicates that the attentional prototype networks significantly improve classification performance. The attentional prototype network, effective for addressing small-sample classification problems, incorporates an attention mechanism that enhances the extraction of delayed features. This improvement contributes to improved performance in the classification of time-series signals^[33].

Currently, the assessment of bradykinesia in PD primarily relies on clinical scales. Despite the application of various intelligent instruments to achieve more objective and accurate quantitative assessments, there remains a lack of detailed differentiation of bradykinesia, and the classification accuracy can be further improved^[10,34]. This paper introduces an effective solution for the accurate assessment of the degree of bradykinesia in PD by combining high-precision motion capture data and a small-sample classification method designed for time series signals.

This study has some limitations, which are planned as the focus of our future research. One main limitation is that PD is a heterogeneous condition rather than a disease. Hence, it is ambitious to draw generalizable conclusions. Besides, the test-retest reliability is absent in our study. Larger sample sizes with segmented PD types should be considered in future work to further validate our method's effectiveness and the test-retest reliability. On the other hand, the previous study has explored the potential of integrating multimodal signals, which improved the motion function assessment^[35]. The proposed method can be extended to the multimodal signals, which promises to achieve a more accurate assessment of bradykinesia in PD.

5. CONCLUSIONS

This paper addresses the classification of small-sample time series signals and proposes a classification method based on DTW data merge and attentional prototype networks. Firstly, the method employs DTW data merge for data augmentation. Subsequently, a random grouping method is used to reorganize the dimension of time series, followed by convolution operations to extract features in multivariate time series. The attention mechanism and prototype learning are introduced to train low-dimensional feature representations of time series, thus reducing the dependency on data volume. The proposed method is applied to motion capture data of upper and lower limb movements. Experimental results indicate that the DTW data merge method, attention mechanism, and prototype learning modules effectively reduce the data volume requirements. Additionally, the use of attention prototype networks significantly improves classification performance. The proposed method can be effectively applied to the classification of small-sample time series signals and achieve an accurate assessment of the degree of bradykinesia in PD.

DECLARATIONS