An optimized strategy for density prediction of intermetallics across varied crystal structures via graph neural network

Data sources and preprocessing

Density data for binary intermetallic compounds are collected using the API provided by the materials project website^[28]. Only compounds composed of metallic elements are included, while those containing non-metallic elements or noble gases are excluded to ensure data quality and consistency. To guarantee thermodynamic stability, only compounds with an “energy above hull” value of 0 are retained. This selection ensures high stability and reliability for practical use, minimizing prediction errors caused by unstable compounds. Each downloaded data entry includes the following: Material ID, chemical formula, crystal system, density, and crystallographic information file (CIF). CIF is a standardized format used to record the space group, unit cell parameters, atomic coordinates, and atom types of crystalline materials. After data collection, the raw data is cleaned and preprocessed to ensure data quality and enhance the model’s predictive performance. Incomplete or outlier data points are excluded, such as those without density or crystal structure values. A total of 7,253 binary intermetallic compound density data are gathered. These compounds cover seven major crystal structure types: cubic, hexagonal, monoclinic, orthorhombic, trigonal, tetragonal, and triclinic. The diversity of these structural types better represents the complexity of intermetallic compounds in practical applications, providing comprehensive training data for subsequent machine learning modeling.

The pie chart in Figure 1A shows the proportion of data for each crystal structure. Cubic and hexagonal structures are the most common, comprising 29% and 28.3% of the data, respectively. The triclinic structure has the fewest data points, representing only 0.5%. In addition, the periodic table in Figure 1A shows the frequency of elements in the intermetallic compound density dataset. The higher the frequency, the darker the corresponding color. The violin plot illustrates the density distributions of compounds across six crystal structures, as shown in Figure 1B. Each structure is represented by a distinct violin plot. The shape and width of each plot reflect the density distribution, and the white dots within the plots indicate the median density for each crystal structure.

Figure 1. Crystal structures and density distributions of binary intermetallic compounds. (A) Distribution of the number of elements and compounds across different crystal structures; (B) Density distributions of compounds within seven crystal structures.

Dataset splitting and feature construction

The dataset in this study is split using the 80/20 method, with 80% of the data used for the training set and the remaining 20% for the test set. Moreover, to further assess the model’s generalization capability and prediction accuracy on complex compounds, density data from 237 ternary and 33 quaternary intermetallic compounds are collected as an independent test set. These multicomponent compound samples cover various element combinations and complex crystal structures, enabling a more comprehensive evaluation of the model’s performance. This study employs several common evaluation metrics to comprehensively evaluate the model’s performance. These include regression metrics such as root mean square error (RMSE)^[29], mean absolute error (MAE)^[30], and the coefficient of determination (R²)^[31], as well as classification metrics such as precision, recall, F1-score, and receiver operating characteristic (ROC) curves^[32].

The Python library XenonPy is leveraged to introduce 58 properties of 94 elements (H to Pu), covering multiple aspects such as electronic structures, physicochemical properties, and microscopic geometric structures^[33]. An additional nine features related to bond energy are included, resulting in a total of 67 elemental features for constructing machine learning features and serving as input parameters for IGNN model nodes. Detailed information on the 67 element features is given in Table 1. The min-max scaling method is used to normalize the feature set, mapping all values between 0 and 1 to eliminate the influence of scale differences on model performance^[34]. This process reduces the model’s sensitivity to specific numerical ranges, preventing adverse effects on model training caused by large data ranges. Furthermore, it preserves the relative distribution of the data while ensuring consistent learning efficiency across different features during model training.

Machine learning algorithms and GNN model

In this study, various commonly used traditional machine learning models, along with the constructed GNN model, are used to predict the density of binary intermetallic compounds. The traditional machine learning models included linear regression (LR), support vector machine (SVM), K-nearest neighbors (KNN), random forest (RF), and EXtreme gradient boosting (XGBoost). LR is a fundamental regression model that assumes a linear relationship between independent and dependent variables. Although its structure is simple, it can still perform effectively when the data exhibits linear characteristics. SVM conducts classification and regression by identifying a hyperplane in the feature space that maximizes class separation. It is particularly effective for small datasets and capable of addressing nonlinear issues. KNN is an instance-based learning method that makes predictions by calculating the distance between test samples and training samples. RF consists of multiple decision trees, each trained on different subsets of data and features. The final prediction is obtained by averaging the outputs or majority voting of all the trees, offering high robustness and noise resistance. XGBoost is an optimized implementation of gradient boosting decision tree, known for its precision and efficiency. It minimizes prediction errors by combining multiple weak learners with weighted contributions, and is capable of handling high-dimensional data and missing values.

The main idea in this study is to represent the crystal structure as graph structure data with nodes corresponding to atoms and edges corresponding to chemical bonds. The entire graph and the corresponding adjacency matrix are constructed based on the locations and connectivity of atoms in the crystal. The properties of the nodes in the graph are represented by physical and chemical properties inherent to the atoms, such as atomic concentration, atomic radius, and boiling point. Mathematically, let $$ \mathcal{G=(V,\varepsilon)} $$ be an undirected graph, where $$ \mathcal{V} $$ = {v₁, v₂, …, v_n} is the set of n nodes or vertices, and ε is the set of m edges. The adjacency matrix A ∈ $$ \mathbb{R} $$^|v|×|v| defines the topological structure of the crystal graph, and node feature matrix X ∈ $$ \mathbb{R} $$^|v|×D includes the properties of atoms. A_uv∈ A equals 1 if (u, v) ∈ ε and A_uv equals 0 otherwise. x_v is the feature vector of node v, and D is the dimension size of the feature vector. Specifically, each node represents an atom, and the set $$ \mathcal{V} $$ contains all the atoms in the crystal. Each edge connects two nodes and represents the existence of chemical bonding between these two atoms. The construction of the edges considers the distances between the atoms and the nature of the chemical bonds, which allows the graph-structured data to accurately capture the interactions between atoms inside the crystal.

To completely take advantage of the valuable information in crystal graph-structured data, a novel IGNN model is proposed for crystal property prediction based on the topological information and atomic properties in the crystal graph. The model can efficiently capture complex associated relationships between atoms based on the topological characteristics of the given crystal graph-structured data, combining with the atomic features from the prior knowledge and employing a message-passing mechanism for information aggregation, which enables entire graph representation learning to obtain the specific properties of the crystal graph. The method enables the automatic learning of effective representations from the basic compositions and topological information of crystal graph-structured data without the requirement of extensive human-designed feature engineering, allowing the reliable and accurate prediction of material properties. The architecture includes the following critical components: (1) crystal structure transformation; (2) node features construction; and (3) GNN-based graph representation learning.

(1) Crystal structure transformation. Let Atoms = {a₁, a₂, …, a_n} denote the set of atoms in the unit cell, where a_i indicates the i-th atom. The distance r is assigned to identify the neighbors of each atom in the unit cell, determining the connectivity between the atoms. This enables the transformation of the 3D spatial information of the crystal into graph-structured data. The connectivity between atoms is recorded by constructing an adjacency matrix, ensuring that their interactions are accurately reflected in the graph. The relationship between atoms and their neighbors can be expressed as

where N (a_i) denotes the neighbors of atom a_i, Atoms is the set of all atoms in the crystal. d (a_i, a_j) is the distance between atoms a_i and a_j. Specifically, for each atom, neighbors are identified within 6 Å radius, where these atoms share a Voronoi face and the interatomic distance is shorter than the sum of the Cordero covalent radii to within a 0.25 Å tolerance^[20,35].

(2) Node features construction. To obtain prior knowledge about the atoms, feature representations are constructed for each node in the crystal graph. Specifically, atomic properties are extracted as attributes of the nodes in the corresponding crystal graph, derived from the physical and chemical properties of the atoms, informed by prior knowledge. An eigenvector x_i for atom a_i is defined, containing multiple related physical and chemical properties such as atomic concentration, atomic radius, boiling point, etc. The feature vector can be given as follows:

where the f_j (a_i) denotes the j-th feature associated with atom a_i and m is the total number of features. The feature vectors of all atoms are combined into a feature matrix X, which is defined as follows:

The introduction of prior knowledge allows the model to effectively utilize these physical and chemical properties in the proposed method, thus improving the ability of the model to understand and predict crystal properties.

(3) GNN-based graph representation learning. GNNs are employed as a learning strategy for crystal graph-structured data, using a multi-layer architecture designed to capture complex interactions and global properties between atoms in the given crystal. Specifically, Batch Normalization is utilized as a regularizer to alleviate the internal covariate shift problem in deep neural networks. Normalization and linear transformation enable the mean and variance of the input data in each layer of the network to be within a certain range^[36]. The method of data normalization in Batch Normalization is given in

where μ and σ are statistics for the current row and are unlearnable. γ and β are the scale and shift parameters to be learned to control the variance and mean of y_i.

After executing Batch Normalization, a linear layer is introduced to further transform the feature representations of the nodes, which enables the mapping of the batch-normalized feature matrix to a new feature space. This linear layer adjustment enhances the model’s capacity to learn higher-order feature representations by altering the dimensions of the features, thereby boosting the model’s expressive power. After obtaining the preprocessed node feature matrix, a l-layer GNN architecture is designed to discern complex patterns and relationships in the crystal graph-structured data. The GNN model is more expressive and generalizable when dealing with crystal structure data by stacking l-layer GNNs. The GNN primarily focuses on identifying a node aggregation function to effectively aggregate node features X, resulting in updated node embedding. For instance, the GCN model with the following layer-wise propagation rule:

where H^(l) is the matrix of activations in the l-layer, σ is the activation function, and W is a layer-specific trainable weight matrix. $$ \tilde{{\mathrm{\mathbf{A}}}} $$ = A + I_Nis normalized adjacency matrix with added self-connections. I_Nis the identity matrix and $$ \tilde{{\mathrm{\mathbf{D}}}}_{ii}=\sum_j\tilde{{\mathrm{\mathbf{A}}}}_{ij} $$.

In the spectrum domain theory, existing methods commonly utilize the convolution theorem and filtering operations in the spectral domain to define convolution operations, such as computing the eigen-decomposition of the normalized Laplace matrix of a graph^[19]. The graph convolution in GCN is defined as a first-order Chebyshev polynomial of the graph Laplace matrix, while the graph convolution adopted by the topology adaptive graph convolutional network (TAGCN) method is defined as a multiplication of the polynomials of the graph adjacency matrix^[37]. In contrast to the simplified formal spectral method that is equivalent to propagating vertex features in a graph employed in the GCN method, the K-localized filter is employed in TAGCN to explore crystal graph-structured data. Specifically, a set of receptive fields of size ranging from 1 to K is defined to learn localized features on graph-structured data. This approach is given in

Finally, after the k-layer feature aggregation and transformation, the features learned by the GNN are effectively transformed into the final density prediction results. This transformation is accomplished through two linear layers, interspersed with a nonlinear activation function between them, enhancing the model’s ability to predict complex outcomes from the graph-structured data. The learning process for density of crystal structure can be expressed as follows:

where y denotes the predicted density of crystal structure, and H^(k) is the output feature matrix after obtaining k-layers of aggregation. W⁽¹⁾ and W⁽²⁾ are the learnable weight matrices for two linear layers, and b⁽¹⁾ and b⁽²⁾ are the corresponding bias vectors. According to Equation (6), the initial node features H⁽⁰⁾ = X, where X represents the raw input features of the nodes in the graph. After k-layers, the final node-level features learned by the GNN are represented as H^(k), which serves as the transformed representation X’= H^(k).

For our method, hyperparameter optimization is conducted over 500 trials using Hyperopt optimizer, with 80% for training and 20% for validation. Final classification accuracies are reported on the test set. All experiments are conducted on a Linux server with Nvidia RTX 4090 GPU (24GB memory). All methods and GNN basic architectures are implemented according to PyTorch 2.1.2 and basic modules of PyTorch Geometric 2.5.3. Key training parameters are fixed through hyperparameter optimization, including a learning rate of lr ≈ 0.002, weight decay w ≈ 0.003, a batch size of 512, and 512 hidden channels. AdamW is used as the optimizer, PowerMeanAggregation is selected for global pooling, and the model includes three graph convolutional layers as learners. The detailed flow chart is shown in Figure 2.

Figure 2. A flowchart for predicting the density of intermetallic compounds using machine learning and GNN models, involving collecting raw data, creating feature pools, building models, and evaluating model performance. GNN: Graph neural network.

Machine learning modeling for specific crystal structures

To conduct an in-depth analysis and improve the accuracy of density prediction, this study first establishes machine learning models based on the seven different crystal structures. This process of modeling utilizes the five common machine learning models mentioned above. Each crystal structure’s intermetallic compounds are extracted into individual data subsets. The input features selected are variables closely related to density, such as atomic mass, atomic radius, electronegativity difference, and lattice parameters. Six statistical treatments are applied to the elemental property data, such as calculating weighted averages, weighted variances, geometric means, harmonic means, maximum values, minimum values, and weighted sums. An initial feature set comprising 290 representative density features is generated, providing robust data support for comprehensively exploring the multifactorial influences on intermetallic compound density.

Figure 3 presents the performance of machine learning models in predicting density across different crystal structures. A bar chart in the bottom right compares the accuracy of various machine learning models within a given crystal structure. Adjacent to this, a scatter plot shows the relationship between the experimental density values and those predicted by the most effective machine learning model for intermetallic compounds. Figure 3A shows the machine learning model predictions for the Hexagonal system. The bar chart indicates that among the five machine learning models, XGBoost achieves the highest accuracy with R² = 0.9798, while the SVM model has the lowest accuracy with R² = 0.8538. The scatter plot in Figure 3A compares XGBoost predictions with calculated values by DFT, where the points cluster closely around the diagonal line (y = x), demonstrating a strong agreement between the predicted and experimental densities. XGBoost still shows the best prediction accuracy for the cubic system, with R² close to 0.98, as shown in Figure 3B. However, for the monoclinic system, R² = 0.9825 is achieved by the RF model, which outperforms XGBoost, as depicted in Figure 3C. Figure 3D-F shows that XGBoost achieves the highest accuracy for the remaining systems including orthorhombic, trigonal, and tetragonal systems with R² values of 0.9692, 0.9825, and 0.9826, respectively. From Figure 3, it can be seen that the data points for each model (especially XGBoost) are mostly aligned along the diagonal line, indicating that the predictions are generally accurate and demonstrating the potential of machine learning models for predicting the density of intermetallic compounds.

Figure 3. Performance comparison of machine learning models for density prediction across crystal structures. (A) Hexagonal; (B) Cubic; (C) Monoclinic; (D) Orthorhombic; (E) Trigonal; (F) Tetragonal. Bar charts compare the predictive accuracy of different machine learning models for each crystal structure, while the scatter plots show the relationship between calculated values by DFT and predictions from the best-performing model. DFT: Density functional theory.

Machine learning modeling on aggregated crystal structures

In the above section, machine learning models are built based on data subsets for specific crystal structures, achieving high prediction accuracy. However, this single crystal structure-based modeling approach has limitations in practical applications, especially when studies require density prediction on mixed data of multiple crystal structures. In this section, an approach without distinguishing crystal structures is adopted, integrating the data from all seven crystal structures for unified modeling to explore the performance of traditional machine learning models on this complex dataset. The model results are shown in Figure 4A and B. Among the traditional machine learning models, the accuracy of simple models such as LR or KNN remained almost the same or slightly increased, while the prediction accuracy of complex models such as RF or XGBoost decreased to varying degrees. For example, the accuracy of the XGBoost model that shows optimal performance in the above section decreased from 0.98 to 0.9468. Figure 4C-E shows scatter plots of the predictions from the LR, SVM, and XGBoost models. From these images, it can be seen that the prediction results of the LR and SVM models are poor, with some points deviating far from the diagonal line. The XGBoost model has higher accuracy than the other two, but a few data points deviate significantly from the diagonal, reducing prediction accuracy. A review of these compounds revealed that they all have polymorphic forms in the dataset. For instance, the test set includes the compound In₃Hg with an “Orthorhombic” structure and an experimental density of 1.4236 g/cm³. However, in the training set, In₃Hg is classified under a “Hexagonal” structure with a significantly higher experimental density of 8.6732 g/cm³. This discrepancy likely contributes to the XGBoost model inaccurately predicting the density of In₃Hg as 7.5879 g/cm³ in the test set. The compound PuPt₃, which exhibits a “Tetragonal” structure and a density of 18.6427 g/cm³ in the training set, appears with a “Cubic” structure and a density of 19.9649 g/cm³ in the test set. The XGBoost prediction is 17.1343 g/cm³, aligning more closely with the training set density.

Figure 4. Performance comparison of different machine learning models and the IGNN model on the test dataset. (A and B) Comparison of model performance, illustrating the R² and RMSE values of different models; (C-F) Scatter plots of predicted values vs. calculated values by DFT. Dots of different colors represent various crystal structure types, with the gray diagonal line indicating perfect agreement between predicted values and calculated values by DFT. IGNN: Intermetallics graph neural network; R²: the coefficient of determination; RMSE: root mean square error; DFT: density functional theory.

The above analysis shows that intermetallic density data for different crystal structures vary significantly. Directly combining all structures’ data for modeling causes traditional models to struggle in recognizing differences between crystal structures, which in turn affects prediction performance. In this context, The study introduces an IGNN model that is better suited for multi-crystal structure data, addressing the limitations of traditional models when applied to mixed crystal structure datasets. As illustrated in Figure 4A and B, the constructed IGNN model clearly outperforms traditional machine learning models, achieving the highest R² value of 0.9884 and the lowest RMSE of 0.4291 g/cm³. Additionally, the scatter plot in Figure 4F shows that the IGNN model can utilize node information in the graph structure to automatically recognize differences between crystal structures and incorporate this information as input for modeling, enhancing its adaptability to multi-structure datasets. The prediction error for the compound In₃Hg decreased from 6.16 to 3.25 g/cm³, and the error for PuPt₃ reduced from 2.83 to 0.82 g/cm³.

To further assess the generalization capability of the IGNN model in handling the density of complex intermetallic compounds, an independent density test set for ternary and quaternary compounds is applied. The extended test set is more complex because the structures of ternary and quaternary compounds involve more types of atoms and more complex crystal structures, providing a more comprehensive examination of the model’s robustness across different compound types and structures. Figure 5A shows the performance of IGNN on the ternary compound test set, with an R² value of 0.9852 and an RMSE of 0.3313 g/cm³, indicating a high prediction accuracy for the ternary system. The IGNN prediction results on the quaternary compound test set are shown in Figure 5B, with an R² of 0.9694 and an RMSE of 0.3340 g/cm³. Compared to ternary compounds, the prediction accuracy for quaternary compounds is slightly lower, likely due to the higher structural complexity of quaternary compounds. From the scatter plot distribution, the IGNN model’s prediction results on the ternary and quaternary compound test sets are highly consistent with calculated values by DFT, with most data points concentrated near the ideal diagonal, indicating minimal deviation between predicted values and calculated values by DFT. The data points for each crystal structure are distributed relatively evenly, without significant deviation for any particular crystal structure. This suggests that the IGNN model demonstrates strong stability and generalization ability when handling diverse crystal structures.

Figure 5. IGNN model performance on density prediction for ternary and quaternary intermetallic compounds. (A) Prediction results on ternary compounds; (B) Prediction results on quaternary compounds (Dots of different colors represent various crystal structure types). IGNN: Intermetallics graph neural network.

Classification prediction and visualization

In studying the crystal structure and density prediction of intermetallic compounds, it is necessary not only to perform accurate regression predictions but also to conduct density classification analysis to explore the influence of different crystal structures on model classification. To gain a better understanding of the IGNN model’s performance in classification tasks, t-SNE is employed to visualize classification results across different layers of the IGNN model^[38]. Additionally, a comparative analysis of the classification performance of IGNN and various traditional machine learning models is conducted.

t-SNE is a widely used dimensionality reduction technique that projects high-dimensional data into a two-dimensional space, facilitating visual assessments of a model’s ability to differentiate between categories. In this study, t-SNE is employed to visualize the feature representations at each layer of the IGNN model [Figure 6]. Figure 6A presents the t-SNE visualization of the input layer. In the input layer, data points from various crystal systems are not yet clearly separated, suggesting that the model has not deeply separated data features at this initial stage. The t-SNE visualization of the normalization layer is shown in Figure 6B. Following normalization, the data distribution becomes more concentrated, with emerging boundaries between different crystal systems, suggesting initial feature refinement. The t-SNE visualization of the global pooling layer is shown in Figure 6C. The data points from different crystal systems demonstrate a clear clustering tendency, reflecting the model’s capacity to aggregate node features and capture significant structural differences among the crystal systems. Figure 6D displays the t-SNE visualization of the output layer, where data points from different crystal systems are distinctly separated, indicating a strong classification outcome. The features extracted at the output layer of the IGNN model possess high discriminative power, creating clear boundaries between data from different crystal structures in two-dimensional space. Through the t-SNE visualization analysis of each layer in the IGNN model, a progressive enhancement in the model’s ability to distinguish among different crystal structures is observed as the layers deepen. This systematic feature extraction and aggregation approach provides IGNN with a distinct advantage in complex density classification tasks.

Figure 6. t-SNE visualization of density classification predictions at different IGNN model layers. (A) Input layer; (B) Normalization layer; (C) Global pooling layer; (D) Output layer. Each subplot shows the output from a different layer, with data points in two-dimensional space colored by a crystal system. t-SNE: t-Distributed stochastic neighbor embedding; IGNN: intermetallics graph neural network.

To further verify the performance of the IGNN model in classification tasks, this study compares it with several traditional machine learning models, including LR, RF, KNN, XGBoost, and SVM. Figure 7A presents the performance comparison of these models, covering evaluation metrics such as Precision, Recall, F1 score, and micro-average area under the curve (AUC_micro). Compared to these traditional models, IGNN shows superior performance in all evaluation metrics, reaching high levels in area under the curve (AUC), Precision, and F1 scores. To more intuitively display the performance of each model in classification tasks, Figure 7B-E shows the multi-class ROC curves for the SVM, KNN, XGBoost, and IGNN models. The ROC curve reflects the model’s classification ability, with a larger AUC indicating stronger discrimination power. In Figure 7B, SVM performs decently in some categories. Due to its rigid classification boundaries, it lacks flexibility in multi-crystal structure classification, leading to a lower average AUC value and difficulty in competing with IGNN. The KNN model shows a relatively low average AUC score in multi-class classification, with a flatter curve, as seen in Figure 7C. Particularly in different crystal structure categories with close densities, KNN struggles to make accurate distinctions, demonstrating certain limitations. As depicted in Figure 7D, the XGBoost model shows higher AUC values in some categories, though its overall average AUC score remains below that of IGNN. As shown in Figure 7E, the ROC curves for each category in IGNN exhibit high AUC values, demonstrating its excellent classification ability. IGNN captures subtle differences among crystal structures using its graph structure, achieving superior performance in multi-class classification tasks.

Figure 7. Performance comparison of machine learning models and IGNN model in classification tasks. (A) Comparison of performance metrics for different models in classification tasks, including Precision, Recall, F1 Score, and micro-average AUC; (B-E) Multi-class ROC curves for different models, with larger AUC values indicating stronger classification ability. (B) SVM; (C) KNN; (D) XGBoost; (E) IGNN. IGNN: Intermetallics graph neural network; AUC: Area under the curve; ROC: receiver operating characteristic; SVM: support vector machine; KNN: K-nearest neighbors; XGBoost: EXtreme gradient boosting.

From the above results, it can be concluded that the IGNN model captures complex relationships between atoms in crystal structures via graph-level representations, inherently understanding spatial and bonding configurations. Especially, for polymorphic intermetallic compounds, IGNN incorporates relevant features from the graph structure, and thus reduces the need for extensive feature engineering, minimizes potential biases, thereby enhancing the model generalization capabilities. Compared to traditional models (e.g., SVM, KNN, XGBoost, etc.) that rely on fixed feature vectors, IGNN demonstrates significant advantages in structural awareness, feature interactions, generalization ability, and flexibility. It captures complex dependencies and nonlinear relationships, and exhibits superior performance across different crystal structures and compositions.