A review of causality-based fairness machine learning

4.1. Group causality-based fairness notions

Group fairness notions aim to discover the difference in outcomes of AI decision models across different groups. The value of an individual's sensitive attribute reflects the group he (or she) belongs to. Considered an example of salary prediction where $$ s^+ $$ and $$ s^- $$ represent male and female groups, respectively. Some representative group causality-based fairness notions are introduced as follows.

4.1.2. Path-specific fairness

The causal effect of sensitive attribute on the outcome can be divided into direct effect and indirect effect, and it can be deemed fair or discriminatory by an expert. Direct discrimination can be captured by the causal effects of $$ S $$ on $$ Y $$ transmitted along the direct path from $$ S $$ to $$ Y $$, while indirect discrimination is measured using the causal effect of $$ S $$ on $$ Y $$ along causal paths from $$ S $$ to $$ Y $$ that pass through redlining/proxy attributes.

Figure 5(a) represents the causal graph of a simple example of a toy model of loan decision AI model, where Race is treated as the sensitive attribute and Loan is treated as the decision. Since ZipCode can reflect the information of Race, ZipCode is a proxy for the sensitive attribute, that is to say, ZipCode is a redline attribute. Thus, the causal effects spreading along the path Race$$ \rightarrow $$Loan are then considered to be direct discrimination, and the causal effects spreading along the path Race$$ \rightarrow $$ZipCode$$ \rightarrow $$Loan are considered to be indirect discrimination. Note that the causal effects spreading along the path Race$$ \rightarrow $$Income$$ \rightarrow $$Loan are explainable bias since it is reasonable to deny a loan to an applicant if he (or she) has a low income. That is to say, the partial difference in loan issuance across different race groups can be explained by the fact that some racial groups in the collected data tend to be underpaid.

Figure 5. Two alternative graphs for the loan application system. (a) A causal graph of the loan application system, where Race is the sensitive attribute and Loan is the decision. (b) A causal graph of the system after removing unresolved discrimination. (c) A causal graph of the system that is free of proxy discrimination.

Path-specific effect ^[10] is a fine-grained assessment of causal effects, that is, it can evaluate the causal effect transmitted along certain paths. Thus, it is used to distinguish among direct discrimination, indirect discrimination, and explainable bias. For any set of paths $$ \pi $$, the $$ \pi $$-specific effect can be computed as below:

(5) $$ \begin{equation} PSF_{\pi}(y) = P(y|do(s^+|\pi, s^-|\bar{\pi}))-P(y|do(s^-)) \end{equation} $$

where $$ P(y_{s^+|\pi, s^-|\bar{\pi}}) $$ denotes the distribution of $$ Y=y $$ where the intervention $$ do(s^+) $$ (i.e., force $$ S $$ had $$ s^+ $$) is only transmitted along path $$ \pi $$ while the intervention $$ do(s^-) $$ (i.e., actual world $$ S=s^- $$) is transferred along the other paths (denoted by $$ \bar{\pi} $$). If $$ \pi $$ contains all direct edge from $$ S $$ to $$ Y $$, $$ PSF_{\pi}(y) $$ measures the direct discrimination. If $$ \pi $$ contains all indirect paths from $$ S $$ to $$ Y $$ that pass through redlining/proxy attributes, $$ PSF_{\pi}(y) $$ evaluates the indirect discrimination. If $$ \pi $$ contains all indirect paths from $$ S $$ to $$ Y $$ that pass through explaining attributes, $$ PSF_{\pi}(y) $$ assesses the explainable bias.

4.2. Individual causality-based fairness notions

Different from group fairness notions that measure the differences in the outcome of decision models between advantaged groups and disadvantaged ones, individual fairness notions aim to examine whether the outcome of decision models is fair to each individual in the population. Some representative group causality-based fairness notions are discussed here.

5.1. Pre-processing Causality-based methods

Pre-processing methods update the training data before feeding them into a machine learning algorithm. Specifically, one idea is to change the labels of some instances or reweigh them before training to limit the causal effects of the sensitive attributes on the decision. As a result, the classifier can make a fairer prediction ^[36]. On the other hand, some studies propose to reconstruct the feature representations of the data to eliminate discrimination embedded in the data ^{[38, 39]}.

For example, Zhang et al.^[27] formalized the presence of discrimination as the presence of a certain path-specific effect, and then framed the problem as one maximizing the likelihood subject to constraints that restrict the magnitude of the $$ PSE $$. To deal with unidentifiable cases, they mathematically bound the $$ PSE $$. CFGAN ^[40], which is based on Causal GAN ^[41] to learn the causal relationship between the attributes, adopts two generators to separately simulate the underlying causal model that generates the real data and the causal model after the intervention and two discriminators to produce a close to real distribution and to achieve total effect fairness, counterfactual fairness, and path-specific fairness. Salimi et al.^{[42, 43]} leveraged the dependencies between sensitive and other attributes, which is provided by the causal knowledge, to add or remove samples from the collected datasets in order to eliminate the discrimination. Nabi et al.^[44] only considered mapping generative models for $$ P(Y, {\bf V} \setminus {\bf W} |{\bf W}) $$ consisting of some attributes $$ {\bf W} $$ to "'fair" versions of this distribution $$ P*(Y, {\bf V} \setminus {\bf W} |{\bf W}) $$ and ensured that $$ P*({\bf W})=P({\bf W}) $$. PSCF-VAE ^[45] achieves path-specific counterfactual fairness by modifying the observed values of the descendant attribute of the sensitive attribute on the unfair causal path during testing, leaving the underlying data-generation mechanism unaltered during training.

5.2. In-Processing Causality-based methods

In-processing methods eliminate discrimination by adding constraints or regularization terms to machine learning models ^[46-50]. If it is allowed to change the learning procedure for a machine learning model, then in-processing can be used during the training of a model either by incorporating changes into the objective function or imposing a constraint.

For example, multi-world fairness algorithms ^[12] add constraints to the classification model that require satisfying the counterfactual fairness. To address the unidentifiable situation and alleviate the difficulty of determining causal models, it combines multiple possible causal models to make approximately fair predictions. A tuning parameter $$ \alpha $$ is used to modulate the trade-off between fairness and accuracy. Hu et al.^[51] proposed to learn multiple fair classifiers simultaneously from a static training dataset. Each classifier is considered to perform soft interventions on the decision, whose influence is inferred as the post-intervention distributions to formulate loss functions and fairness constraints. Garg et al.^[52] proposed to penalize the discrepancy between real samples and their counterfactual samples by adding counterfactual logit pairing (CLP) to the loss function of the algorithm. Similarly, the authors of ^[53] proposed to add constraints terms during the training to eliminate the difference in outcome between two identical individuals, one from the real world and one from the counterfactual world that belongs to another sensitive group. Kim et al.^[54] addressed the limitation that some causality-based methods cannot distinguish between information caused by the intervention (i.e., sensitive variables) and information related to the intervention by decomposing external uncertainty into intervention-independent variables and intervention-related ones. They proposed a method called DCEVAE, which can estimate the total effect and counterfactual effects in the absence of full causal maps.

5.3. Post-processing Causality-based methods

Post-processing methods modify the outcome of the decision model to make fairer decisions ^{[55, 56]}. For example, Wu et al.^[57] adopted the c-component factorization to decompose the counterfactual quantity, identified the sources unidentifiable terms, and developed the lower and upper bounds of counterfactual fairness in unidentifiable situations. In the post-processing stage, they reconstructed the trained decision model so as to achieve counterfactual fairness. The counterfactual privilege algorithm ^[58] maximizes the overall benefit while preventing an individual from obtaining beneficial effects exceeding the threshold due to the sensitive attributes, so as to make the classifier achieve counterfactual fairness. Mishler et al.^[59] suggested using doubly robust estimators to post-process a trained binary predictor in order to achieve approximate counterfactual equalized odds.

5.4. Which mechanism to use

We discuss the various mechanisms for enhancing fairness above. Here, we further compare these mechanisms and discuss the advantages and disadvantages of them, respectively. This section provides insights into how to select suitable mechanisms for use in different scenarios based on the characteristics of these mechanisms. Every type of mechanism has its advantages and disadvantages.

The pre-processing mechanism can be flexibly adapted to the downstream tasks since it can be used with any classification algorithm. However, since the pre-processing mechanism is a general mechanism where the extracted features can be widely applicable for various algorithms, there is high indeterminacy regarding the accuracy of the trained decision models.

Similar to the pre-processing mechanism, the post-processing mechanism also can be flexibly used in any decision model. Post-processing mechanisms are easier to fully eliminate discrimination for the decision models, but the accuracy of the decision models depends on the performance they obtained in the training stage ^[60]. Furthermore, post-processing mechanisms require access to all information of individuals during testing, which may be unavailable because of reasons of privacy protection.

The in-processing mechanism is beneficial to enable a balance between accuracy and fairness of the decision model, which is achieved by explicitly modulating the trade-off parameter in the objective function. However, such mechanisms are tightly coupled with the machine learning algorithm itself and are difficult to optimize in the application.

Based on the above discussion and the studies that attempt to comprehend which mechanism is best to use in certain situations ^{[61, 62]}, we can say that there is no single mechanism that outperforms the others in all cases, and the choice of suitable mechanisms depends on the availability of sensitive variables during testing, the characteristics of the dataset, and the desired fairness measure in the application. For example, when there exists evident selection bias in a dataset, it is better to select the pre-process mechanism for use than the in-process one. Therefore, more research is needed to develop robust fairness mechanisms or to design suitable mechanisms for practical scenarios.

6.1. Data missing

One major challenge for fairness-enhancing algorithms is to deal with the biases inherent in the dataset that is caused by missing data. Selection biases are due to the distribution of collected data, which cannot reflect the real characteristics of disadvantaged groups. Martínez-Plumed et al.^[63] learned that selection bias is mainly caused by individuals in disadvantaged groups being reluctant to disclose information, e.g., people with high incomes are more willing to share their earnings than people with low incomes, which results in bias inference that training in the training institution helps to raise earnings. To address this problem, Bareinboim et al.^[64] and Spirtes et al.^[65] studied how to deal with missing data and repair datasets that contain selection biases by causal reasoning, in order to improve fairness.

On the other hand, the collected data represent only one side of the reality, that is, these data do not contain any information about the population who were not selected. Biases may arise that decide which data are contained or not contained in the datasets. For example, there is a dataset that records the information of individuals whose loans were approved and the information about their ability to repay their loans. Although the automatic decision system that satisfies certain fairness requirements is constrained based on this dataset to predict whether they repay their loan on time, such a predictor may be discriminatory when it is used to assess the credit score of further applicants, since populations not approved for loans are not sufficiently representative in the training data. Goel et al.^[66] used the causal graph-based framework to model the causal process of possible missing data for different settings by which different types of decisions are made in the past, and proved some data distributions can be inferred from incomplete available data based on the causal graph. Although the practical scenarios they discussed are not exhaustive, their work shows that the causal structure can be used for determining the recoverability of quantities of interest in any new scenario.

A promising solution for dealing with missing data can be found in causality-based methods. We see that causality can provide tools to improve fairness when the dataset suffers from discrimination caused by missing data.

6.2. Fair recommender Systems

Recommenders are recognized as the most effective way to alleviate information overloading. Nowadays, recommender systems have been widely used in variable applications, such as ecommerce platforms, advertisements, news articles, jobs, etc. They are not only used to analyze user behavior to infer users' preferences so as to provide them with personalized recommendations, but they also benefit content providers with more potential of making profits. Unfortunately, there exist fairness issues in recommender systems ^[67], which are challenging to handle and may deteriorate the effectiveness of the recommendation. The discrimination embedded in the recommender systems is mainly caused by the following aspects. User behaviors in terms of the exposed items make the observational data confounded by the exposure mechanism of recommenders and the preference of the users. Another major cause of discrimination in recommender systems is that disadvantage items reflected in the observational data are not representative. That is to say, some items may be more popular than others and thus receive more user behavior. As a result, recommender systems tend to expose users to these popular items, which results in discrimination towards unpopular items and leads to the systems not providing sufficient opportunities for minority items. Finally, one characteristic of recommender systems is the feedback loop. That is, the systems exposes to the user for determining the user behavior, which is circled back as the training data for the recommender systems. Such a feedback loop not only creates biases but also intensifies biases over time, resulting in "the rich get richer" Matthew effect.

Due to the usefulness of causal modeling ^[10], removing discrimination for recommender systems from a causal perspective has attracted increasing attention, where the cause graph is used for exposing potentially causal relationships from data. On the one hand, most discrimination can be understood with additional confounding factors in the causal graph and the effect of discrimination can also be inferred through the causal graph. On the other hand, recommendation can be considered as an intervention, which is similar to treating a patient with a specific drug, requiring counterfactual reasoning. What happens when recommending certain items to the users? The causal model has the potential to answer this question. For example, Wu et al.^[68] focused on fairness-aware ranking and proposed to use path-specific effects to detect and remove the direct and indirect rank discrimination. Zhao et al.^[69] and Zheng et al.^[70] considered the effect of item popularity on user behavior and intervened in the item popularity to make fair recommendations. Zhang et al.^[71] attributed popularity bias in the recommender systems to the undesirable causal effect of item popularity on items exposure and suggested intervening in the distribution of the exposed items to eliminate this causal effect. Wang et al.^[72] leveraged counterfactual reasoning to eliminate the causal effect of exposure features on the prediction. Li et al.^[73] proposed generating embedding vectors independent of sensitive attributes by adversarial learning to achieve counterfactual fairness. Huang et al.^[74] regarded causal inference as bandits and performed $$ do $$-operator to simulate the arm selection strategy to achieve fairness towards individuals.

Nowadays, the explainability of recommender systems is increasingly important, which improves the persuasiveness and trustworthiness of recommendations. When addressing the fairness issues of recommender systems from the causal perspective, the explanation of recommendations can also be provided from the effects transmitted along the causal paths. Thus, we are confident that causal modeling will bring the recommendation research into a new frontier.

6.3. Fair natural language processing

Natural language processing (NLP) is an important technology for machines to understand and interpret human natural language text and realize human–computer interaction. With the development and evolution of human natural language, the natural language is characterized by a certain degree of gender, ethnicity, region, and culture. These characteristics are sensitive in certain situations, and inappropriate use can lead to prejudice and discrimination. For example, Zhao et al.^[75] found that the datasets associated with multi-label object classification and visual semantic role labeling exhibit discrimination towards gender attribute, and, unfortunately, the model trained with these data would further amplify the disparity. Stanovsky et al.^[76] provided multilingual quantitative evidence of gender bias in large-scale translation. They found that, among the eight target languages, all four business systems and two academic translation systems tend to translate according to stereotype rather than context. Huang et al.^[77] used counterfactual evaluations to investigate whether and how language models are affected by sensitive attributes (e.g., country, occupation, and gender) to generate sentiment bias. Specifically, they used individual fairness metrics and group fairness metrics to measure counterfactual sentiment bias, conducted model training on news articles and Wikipedia corpus, and showcased the existence of sentiment bias.

Fair NLP is a kind of NLP without bias or discrimination with sensitive attributes. Shin et al.^[78] proposed a counterfactual reasoning method for eliminating the gender bias of word embedding, which aims to disentangle a latent space of a given word embedding into two disjoint encoded latent spaces, namely the gender latent space and the semantic latent space, to achieve disentanglement of semantic and gender implicit descriptions. To this end, they used a gradient reversal layer to prohibit the inference about the gender latent information from semantic information. Then, they generated a counterfactual word embedding by converting the encoded gender into the opposite gender and used it to produce a gender-neutralized word embedding after geometric alignment regularization. As such, the word embedding generated by this method can strike a balance between gender debiasing and semantic information preserving. Yang and Feng ^[79] presented a causality-based post-processing approach for eliminating the gender bias in word embeddings. Specifically, their method was based on statistical correlation and half-sibling regression, which leverages the statistical dependency between gender-biased word vectors and gender-definition word vectors to learn the counterfactual gender information of an individual through causal inference. The learned spurious gender information is then subtracted from the gender-biased word vectors to remove the gender bias. Lu et al.^[80] proposed a method called CDA to eliminate gender bias through counterfactual data augmentation. The main idea of CDA is to augment the corpus by exchanging gender word pairs in the corpus and constructing matching gender word pairs with causal interventions. As such, CDA breaks associations between gendered and gender-neutral words and alleviates the problem that gender bias increases as loss decreases when training with gradient descent.

There exists a certain degree of bias and fairness issues in word embedding, machine translation, sentiment analysis, language models, and dialog generation in NLP. At present, most studies only focus on a single bias (such as gender bias), and there is a lack of research results on other biases or eliminating multiple biases at the same time. Therefore, how should we analyze and evaluate the mechanism and impact of multi-bias in word embedding and machine learning algorithms? Establishing effective techniques for eliminating various biases in word embedding and machine learning algorithms requires further research which needs to be carried out for fair NLP.

6.4. Fair medical

Electronic health records (EHRe) contain large amounts of clinical information about heterogeneous patients and their responses to treatments. It is possible for machine learning techniques to efficiently leverage the full extent of EHRs to help physicians make predictions for patients, thus greatly improving the quality of care and reducing costs. However, because of discrimination implicitly embedded in EHRs, the automated systems may introduce or even aggravate the nursing gap between underrepresented groups and disadvantaged ones. The prior works on eliminating discrimination for clinical predictive models mostly focus on statistics-based fairness-enhancing approaches ^{[81, 82]}. In addition, they do not provide an effective evaluation of fairness to individuals, and the fairness metrics they used are difficult to verify. Some recent studies focus on assessing fairness for clinical predictive models from a causal perspective ^{[83, 84]}. For example, Pfohl et al.^[83] proposed a counterfactual fairness notion to extend fairness to the individual level and leveraged variational autoencoder technology to eliminate discrimination against certain patients.

6.5. Causal analysis packages

This section introduces some representative packages or software for causal analysis, which are helpful for us to develop causality-based fairness-enhancing approaches. These packages can be roughly divided into two categories: one for discovering potential causal structure in data and the other for making causal inferences. Table 2 summarizes typical packages or software for causal analysis.

TETRAD ^[85] is a full-featured software for causal analysis after considerable development where it can be used to discover the causal structure behind the dataset, estimate the causal effects, simulate the causal models, etc. TETRAD can accept different types of data as input, e.g., discrete data, continuous data, time series data, etc. The users can choose the appropriate well-tested causal discovery algorithms it integrates to search causal structure, as well as input prior causal knowledge to limit the search. In addition, TETRAD can parameterize the causal model and simulate the data according to the existing causal diagram. Causal-learn package ^[86] is the Python version of TETRAD. It provides the implementation of the latest causal discovery methods ranging from constraint-based methods, score-based methods, and constrained functional causal models-based methods to permutation-based methods. In addition, there are many packages for causal discovery ^[87-89]. Tigramite ^[88] focuses on searching causal structure from observational time series data. In addition to providing classic causal discovery algorithms, gCastle ^[89] provides many gradient-based causal discovery approaches.

CausalML ^[90] is a Python package which encapsulates many causal learning and causal inference approaches. One highlight of this package is uplift modeling, which is used to evaluate the conditional average treatment effect (CATE), that is, to estimate the impact of a treatment on a specific individual's behavior.

Mediation ^[91] is an R package which is used in causal mediation analysis. In other words, it provides model-based methods and design-based methods to evaluate the potential causal mechanisms. It also provides approaches to deal with common problems in practice and random trials, that is, to handle multiple mediators and evaluate causal mechanisms in case of intervention non-compliance.

Causaleffect ^[92] is an R package which is the implementation of ID algorithm. ID algorithm is a complete identification of causal effects algorithm, which outputs the expression of causal effect when the causal effect is identifiable or fails to run when the causal effect is unidentifiable. DoWhy ^[93], a Python package, also focuses on causal inference, that is, it takes causal graphs as prior knowledge and uses Pearl's $$ do $$-calculus method to assess causal effects.

These packages used for causal analysis assist in developing causality-based fairness-enhancing methods, which are mainly reflected in exposing the causal relationship between variables and evaluating the causal effects of sensitive attributes on decision-making. However, they cannot be used directly to detect or eliminate discrimination. Although there are many software packages for detecting and eliminating discrimination, e.g., AI Fairness 360 Open Source Toolkit ^[94], Microsoft Research Fairlearn ^[95], etc, we are still lacking a package that integrates causality-based approaches.

7.1. Causal discovery

Causality-based fairness approaches require a causal graph as additional prior knowledge of input, where the causal graph describes the mechanism by which the data are generated, that is, it reveals the causal relationship between variables. However, in practice, it is difficult for us to obtain the correct causal graph. A basic way to discover the causal relationship between variables is to conduct randomized controlled trials. Randomized controlled trials consist of randomly assigning subjects (e.g. individuals) to treatments (e.g. gender), and then comparing the outcome of all treatment groups. Unfortunately, in many cases, it may not be possible to undertake such experiments due to prohibitive costs, ethical concerns, or they are physically impossible to carry out. For example, to understand the impact of smoking, it would be necessary to force different individuals to smoke or not smoke. As another example, to understand whether hiring decision models are gender-biased, it would be necessary to change the gender of a job applicant, which is an impracticality. Researchers are therefore often left with non-experimental, observational data, and they have developed numerous methods for uncovering causal relations, i.e., causal discovery. Causal discovery algorithms can be roughly classified into the following three categories: constraint-based, score-based, and those exploiting structural asymmetries.

Constraint-based approaches conduct numerous conditional independence tests to learn about the structure of the underlying causal graph that reflects these conditional independence. Constraint-based approaches have the advantage that they are generally applicable, but the disadvantages are that faithfulness is a strong assumption and that it may require very large sample sizes to get good conditional independence tests. Furthermore, the solution of this approach to causal discovery is usually not unique, and, in particular, it does not help determine the causal direction in the two-variable case, where no conditional independence relationship is available.

Score-based algorithms use the fact that each directed acyclic graph (DAG) can be scored in relation to the data, typically using a penalized likelihood score function. The algorithms then search for the DAG that yields the optimal score. Typical scoring functions include the Bayesian information criterion ^[96], Bayesian–Gaussian equivalent score ^[96], and minimum description length (as an approximation of Kolmogorov complexity) ^{[97, 98]}.

Structural asymmetry-based algorithms take into account the setting that it is impossible to infer the causal direction from observations alone when the data distribution admits structural causal models indicating either of the structural directions $$ X_i \to X_j $$ or $$ X_i \gets X_j $$. To address this problem, structural asymmetry-based algorithms make some additional assumptions about the function of the underlying true data-generating structure, so that they can exploit asymmetries to identify the direction of a structural relationship. These asymmetries manifest in various ways, such as non-independent errors, measures of complexity, etc. Existing methods that exploit such asymmetries are typically local solutions, as they are only able to test one edge at a time (pairwise/bivariate causal directionality), or a triple (with the third variable being an unobserved confounder) ^[99].

In the absence of intervention and manipulation, observational data leave researchers facing a number of challenges: First, there may exist hidden confounders, which are sometimes termed the third variable problem. Second, observational data may exhibit selection bias. For example, younger patients may generally prefer surgery, while older patients may prefer medication. Third, most causal discovery algorithms are based on strong but often untestable assumptions, and applying these strong assumptions to structural or graphical models incites some harsh criticisms.

7.2. Identifiable issue

The identifiable issue is another main obstacle to the application of causal models in fair machine learning. The identifiability of causal effects, including total effect, path-specific effect, and counterfactual effect, has been extensively studied ^{[10, 100-103]}. Many causality-based fairness methods have been proposed to solve this problem so as to achieve fairness more effectively. This section summarizes the main identifiability conditions, the causality-based fairness methods that try to overcome the unidentifiable situations, and their limitations.

7.2.1. Identifiability

This review starts with the simplest identifiability condition of total effects, where the causal effect of a sensitive attribute $$ S $$ on decision $$ Y $$ is computed by $$ P(Y|do(S=s)) $$, which denotes the distribution of decision $$ Y $$ after the intervention $$ S=s $$. Similar to causal discovery, randomized controlled trials can also be used to assess the total effect by performing an intervention on the sensitive attributes to completely avoid unidentifiable situations of the total effect. A randomized controlled trial may allow us not only to identify causal relationships but also to estimate the magnitude of these relationships. However, as mentioned in Section 7.1, it is impossible for us to undertake such experiments in many cases. The causal effect of the sensitive attribute $$ S $$ on decision $$ Y $$ may not be uniquely assessed from observational data and causal graph alone, i.e., the unidentifiable situation. In Markovian models, the total effect is always identifiable, since $$ P(Y|do(S=s)) $$ is always identifiable ^[10]. The simplest case is when there is no confounder between $$ S $$ and $$ Y $$ (see Figure 7(a)). In this case, the causal effect $$ P(Y|do(S=s)) $$ is consistent with the conditional probability $$ P(y|S=s) $$. As such, the total effect can be computed as follows:

(15) $$ \begin{equation} TE(y)=P(y|do(S=s^+))-P(y|do(S=s^-))=P(y|s^+)-P(y|s^-) \end{equation} $$

Figure 7. Simple causal graphs under Markovian assumption.

If there exists an observational confounder between $$ S $$ and $$ Y $$ (see Figure 7(b)), the total effect is also identifiable by summing the probabilities of all values $$ c $$ in the domain of the observable confounder $$ C $$:

(16) $$ \begin{equation} TE(y) = P(y|do(S=s^+))-P(y|do(S=s^-)) = \sum\limits_C P(y|s^+, c)P(c) - \sum\limits_C P(y|s^-, c)P(c) \end{equation} $$

where the formula $$ \sum_C P(y|s, c)P(c) $$ in Equation (16) is also called the back-door formula. If there are no hidden confounders within the observable attributes, then the causal effect $$ P(y|do(S=s)) $$ can also be computed by the following formula:

(17) $$ \begin{equation} P(y|do(s)) = \sum\limits_{{\bf V} \setminus \{S, Y\}, Y=y} \prod\limits_{V \in {\bf V} \setminus \{S\}} P(v|Pa(V)) \end{equation} $$

where $$ Pa(V) $$ represents the parent variables of $$ V $$. It is easy to see that the total effects computed by Equations (16) and (17) can produce the same result.

For semi-Markovian models, the causal effect $$ P(y|do(s)) $$ is not always identifiable, hence the total effect is not always identifiable. The causal effect $$ P(y|do(s)) $$ is identifiable if and only if it can be reduced to a $$ do $$-free expression (i.e., turning the intervention operator $$ do(s) $$ to observational probabilities) by $$ do $$-calculus ^[10]. $$ do $$-calculus is composed of three inference rules: (ⅰ) insertion/deletion of observations, i.e., $$ P(y_s|{\bf z}, w)=P(y_s|{\bf z}) $$ provided that $$ Y $$ and $$ W $$ are dependence-separated at fixed $$ S $$ and $$ {\bf Z} $$ after all arrows leading to $$ S $$ have been deleted in causal graph; (ⅱ) action/observation exchange, i.e., $$ P(y|do(s), {\bf z})=P(y|s, {\bf z}) $$ if $$ Y $$ and $$ S $$ are probabilistically conditionally independent at fixed $$ {\bf Z} $$ after deleting all arrows starting from $$ {\bf Z} $$ in causal graph; and (ⅲ) deletion of actions, i.e., $$ P(y|do(s))=P(y) $$ if there are no causal paths between $$ S $$ and $$ Y $$.

$$ do $$-calculus has been proven to be complete, that is, it is sufficient to derive all identifiable causal effects by $$ do $$-calculus ^[100]. However, it is difficult to determine the correct order of application of these rules, and the wrong order may misjudge the identifiability of causal effects or produce a very complex expression. To address this issue, several studies attempt to give the explicit graphical criteria and map them to simple and concise $$ do $$-free expressions ^{[101, 104]}. A simple case of the identifiability of the causal effect $$ P(y|do(s)) $$ is when the sensitive attribute $$ S $$ is not influenced by any confounder ^[105]. In other words, the causal effect of $$ S $$ is identifiable, if all parents of $$ S $$ are observable. Graphically, there is no bi-directional edge connected to $$ S $$. Formally, the causal effect $$ P(y|do(s)) $$ can be computed as follows:

(18) $$ \begin{equation} P(y|do(s)) = \sum\limits_{{\bf pa}(S)} P(y|s, pa(S))P({\bf pa}(S)) \end{equation} $$

where $$ pa(S) $$ represents the values of parent variables of $$ S $$.

A complex case where the causal effect of $$ S $$ on $$ {\bf V}'={\bf V} \setminus \{S\} $$ is identifiable is that there may exist a bi-directed edge connected to the sensitive attribute $$ S $$, but there are no hidden confounders connected to any direct child of $$ S $$^[105]. Graphically, there is no bi-directional edge connected to any child of $$ S $$. If such criterion is satisfied, the causal effect $$ P({\bf v}'|do(s)) $$ is identifiable and is given by:

(19) $$ \begin{equation} P({\bf v}'|do(s)) = (\prod\limits_{v_i \in Ch(s)} P(v_i|{\bf pa}(v_i)))\sum\limits_s \frac{P({\bf v}')}{\prod\limits_{v_i \in Ch(s)} P(v_i|{\bf pa}(v_i))} \end{equation} $$

where $$ Ch(s) $$ denotes the set of $$ S $$'s children and $$ {\bf pa}(v_i) $$ denotes the set of values of $$ V_i $$'s parents. Equation (19) can be easily adapted to assess the effect of the sensitive attribute $$ S $$ on outcome $$ Y $$.

Tian et al.^[105] also found that, although $$ P({\bf v}'|do(s)) $$ is not identifiable, $$ P({\bf w}|do(s)) $$ is still identifiable for some subsets $$ {\bf W} $$ of $$ {\bf V} $$. Specifically, causal effect $$ P({\bf w}|do(s)) $$ is identifiable if there is no bi-directed path connecting $$ S $$ to any of its direct children in $$ \mathcal{G}_{An}({\bf W}) $$, where $$ An({\bf W}) $$ denotes the union of a set $$ {\bf W} $$ and the set of ancestors of the variables in $$ {\bf W} $$. $$ \mathcal{G}_{An}({\bf W}) $$ denotes the sub-graph of $$ \mathcal{G} $$ composed only of variables in $$ An({\bf W}) $$.

Moreover, the causal effect of sensitive attribute $$ S $$ on outcome $$ Y $$$$ P(y|do(s)) $$ can also be computed by using the front-door criterion, if $$ P(y|do(s)) $$ is identifiable. Before introducing the front-door criterion, the concepts of the back-door path and front-door path are first introduced, which are helpful for the understanding of the front-door criterion. A causal path connecting $$ S $$ and $$ Y $$ which begins with an arrow leading to $$ S $$ is called a back-door path (e.g., $$ S \gets ... Y $$), while a path starting with an arrow pointing away from $$ S $$ is called a front-door path (e.g., $$ S \to ... Y $$). If the front-door criterion is satisfied, there exits a mediator variable $$ Z $$ such that: (ⅰ) there are no backdoor paths from $$ S $$ to $$ Z $$; and (ⅱ) all backdoor paths from $$ S $$ to $$ Z $$ are blocked by $$ S $$. Formally, $$ P(y|do(s)) $$ can be computed as follows:

(20) $$ \begin{equation} P(y|do(s)) = \sum\limits_{Z} P(y|s, z) P(z|s) P(s) \end{equation} $$

These criteria can be generalized to the case where there is no bi-directed edges connection between the sensitive attribute and its direct children. Given the fact that the observational distribution $$ P({\bf v}) $$ can be decomposed to a product of several factors, c-component factorization was proposed to decompose the identification problem into smaller problems (i.e., c-components, each of which is a set of observational variables that are connected by a common confounder in the causal graph) to evaluate the causal effect ^{[15, 105]}.

Shpitser et al.^[15] designed a sound and complete algorithm called ID to identify all identifiable causal effects where ID outputs the expression of the causal effect for the identifiable cases. In addition, they proved that all cases of unidentifiable causal effects $$ P(y|do(s)) $$ can be completely attributed to a graphical structure called hedge.

As to the identifiability of counterfactual effect, if complete knowledge of the causal model is known (including structural functions, $$ P({\bf u}) $$, etc.), any counterfactual quantity can be exactly performed using three steps: (ⅰ) abduction, update $$ P({\bf u}) $$ by observation $$ {\bf O}={\bf o} $$ to compute $$ P({\bf u}|{\bf o}) $$; (ⅱ) action, modify causal model $$ \mathcal{M} $$ by intervention $$ do(s) $$ to obtain the post-intervention model $$ \mathcal{M}_s $$; and (ⅲ) prediction, use post-intervention model $$ \mathcal{M}_s $$ and $$ P({\bf u}|{\bf o}) $$ to compute the counterfactual effect $$ P(y_s|s') $$. However, the above method is usually infeasible in practice due to the lack of complete knowledge of the causal model. In most cases, we only have the causal graph and observational data, which makes the counterfactual effect not always identifiable. The simplest unidentifiable case of counterfactual effect is due to the unidentifiability of $$ P(Y=y, Y_s=y') $$. Graphically, there exists "w-graph" in the causal graph (see Figure 8(c)).

Figure 8. (a) The tony causal graph; (b) the counterfactual graph of (a); (c) the W-graph; and (d) the "kite" graph.

The analysis of the identifiability of counterfactual effect $$ P(y_s|s', {\bf o}) $$ concerns the connection between two causal models, $$ \mathcal{M} $$ and $$ \mathcal{M}_s $$; thus, Shpitser et al.^[15] proposed a make-cg algorithm to construct a counterfactual graph $$ \mathcal{G}' $$ which depicts the independence relationship among all variables in $$ \mathcal{M} $$ and $$ \mathcal{M}_s $$. Specifically, make-cg first combines original causal graph and post-interventional causal graph by removing the same exogenous variables, and the duplicated endogenous variables that are not influenced by $$ do(s) $$. The resultant graph is the so-called counterfactual graph, which can be considered as a typical causal graph for a larger causal model. For example, Figure 8(a) shows an example causal graph, while its counterfactual graph of the counterfactual effect $$ P(y_s|s') $$ is shown in Figure 8(b). All the graphical criteria mentioned above for the identifiability of causal effects are applicable to the counterfactual graph and the c-component factorization of the counterfactual graph for performing the counterfactual inference ^[106]. Shpitser et al.^[15] further developed ID* and IDC* algorithms to distinguish the identifiability of counterfactual effects and compute the counterfactual quantity. In addition, Pearl ^[10] further proved the results about the identifiability of counterfactual effects: if all the structural functions $$ {\bf F} $$ are linear, any counterfactual quantity is identifiable whenever we have full knowledge about the causal model. Unfortunately, there is no single necessary and sufficient criterion for the counterfactual effects' identifiable issues in semi-Markovian models, even the linear causal model ^[107].

For the identifiability of path-specific effect $$ PSE_{\pi}(y) $$, it depends on whether $$ P(y_{s^+|\pi, s^-|\bar{\pi}}) $$ is identifiable or not. Unfortunately, $$ P(y_{s^+|\pi, s^-|\bar{\pi}}) $$ is not always identifiable, even in Markovian models. Avin et al.^[107] gave the necessary and sufficient criterion for the identifiability of $$ P(y_{s^+|\pi, s^-|\bar{\pi}}) $$ in Markovian models called recanting witness criterion. The recanting witness criterion is satisfied when there is a variable $$ W $$ along the causal path $$ \pi $$ connected to $$ Y $$ through another causal path not in $$ \pi $$. Consider an example causal model whose causal graph is shown in Figure 8(d), when the causal path that we follow with interest $$ \pi=\{ S \to W \to A \to Y \} $$ with $$ W $$ as witness, then the recanting witness criterion is satisfied. The corresponding graph structure is called "kite" graph. When this criterion is satisfied, $$ P(y_{s^+|\pi, s^-|\bar{\pi}}) $$ is not identifiable and $$ PSE_{\pi}(y) $$ is not identifiable also. Shpitser et al.^[102] generalized this criterion to semi-Markovian models known as recanting district criterion. Specifically, if there exists district $$ D $$ that represents the set of variables not belonging to the set of sensitive attributes $$ S $$, but ancestral of decision $$ Y $$ via a directed path which does not intersect $$ S $$, and nodes $$ z_i $$, $$ z_j $$$$ \in D $$ (possibly $$ z_i=z_j $$), such that there is a causal path $$ S \to Z_i \to ... \to Y $$ in $$ \pi $$ and a causal path $$ S \to Z_j \to ... \to Y $$ not in $$ \pi $$, then the path-specific effect of $$ S $$ on $$ Y $$ is not identifiable.

7.3. Comprehensive definition of fairness

The sources of unfairness in machine learning algorithms are diverse and complex, and different biases have different degrees of impact on unfairness. Since most fairness notions, including causality-based fairness notions, quantify fairness in a single dimension, when comparing the capabilities of different fairness machine learning algorithms, using different fairness measures will often lead to different results. This means that, whether the algorithm is fair or not is relative, which depends not only on the model and data but also on the task requirements. There is a lack of complete and multi-dimensional causality-based fairness definition and evaluation system for fairness, and it is not possible to effectively quantify the fairness risk faced by machine learning algorithms. Therefore, we need to further explore comprehensive causal-based fairness notions and establish a comprehensive multi-dimensional evaluation system for the fairness of machine learning algorithms. In addition, the definition of fairness needs to be combined with the laws and the concept of social fairness of various countries to avoid narrow technical solutions. The proposition of PC-fairness and causality-based fairness notion defined from both macro-level and individual-level ^[114] are useful explorations to solve this problem.

7.4. Achieving fairness in a dynamic environment

The existing works mainly focus on studying the fairness in machine learning in static, no feedback, short-term impact scenarios, without examining how these decisions affect fairness in future applications over time and failing to effectively adapt to evolutionary cycles. At present, the research on the fairness of machine learning shows a trend of dynamic evolution, which requires the definition of fairness and algorithms to consider the dynamic, feedback, and long-term consequences of decision-making systems. This is particularly evident in recommendation systems, loans, hiring, etc. Fortunately, some researchers are modeling the long-term dynamics of fairness in these areas ^[115-119]. D'Amour et al.^[120] regarded dynamic long-term fair learning as a Markov decision process (MDP) and proposed simulation studies to model fairness-enhancing learning in a dynamic environment. They emphasized the importance of interaction between the decision system and the environment. A complementary work ^[121] shows the importance of causal modeling in dynamic systems. However, due to the complexity of the real-world environment, it is impossible to model the real environment at a high level. Besides, current studies are carried out on low-dimensional data. Therefore, how to highlight important dynamics in simulations and effectively use collected data to ensure an appropriate balance between results and real-world applicability and how to adapt to high-dimensional data are current challenges. In addition, future causality-based fairness-enhancing studies can be combined with dynamic game theory for improving fairness in the confrontation environment and research the detection mechanism of dynamic fairness.

7.5. Other challenges

AI has become more and more mature after rapid development. Although most of the research on AI thus far has focused on weak AI, the design of strong AI (or human-level AI) will be increasingly vital and receive more and more attention in the near future. Weak AI only focuses on solving the given tasks input into the program, while strong AI or human-level AI (HLAI) means that its ability of thinking and action is comparable to that of a human. Therefore, developing HLAI will face more challenges. Saghiri et al.^[122] comprehensively summarized the challenges of designing HLAI. As they said, unfairness issues are closely related to other challenges in AI. There is still a gap between solving the unfairness problem in AI alone and building a trustworthy AI. Next, this review discusses the relationship between fairness and the other challenges in AI.

Fairness and robustness. The robustness of the AI model is manifested in its outer generalization ability, that is, when the input data change abnormally, the performance of the AI model remains stable. AI models with poor robustness are prone to crash and, thus, fail to achieve fairness. In addition, the attacker may obtain private information about the training data and even the training data themselves, although the attacker has no illegal access to the data. However, the research on robustness is still in its infancy, and the theory and notions of robustness are still lacking currently.

Fairness and interpretability. The explainability of discrimination is very important to improving users' understanding and trust in AI models, which is even required by law in many fields. On the other hand, interpretability can explain and judge whether the fairness of AI models is satisfied or not, which assists in improving the fairness of AI models. In some important areas, e.g., healthcare, this challenge becomes more serious because it requires that any type of decision-making must be fair and interpretable.

Causality-based methods are promising solutions to these challenges, as they can not only reveal the mechanisms by which data are generated but also enable a better understanding of the causes of discrimination. Of course, there are far more challenges faced by HLAI than those above, and more about the challenges of designing HLAI can be found in Saghiri et al.'s work ^[122].

7.6. Future trends

More realistic application scenarios. Most of the early studies are carried out under some strong assumptions, e.g., the assumption that there are no hidden confounders between observational variables. However, these assumptions are difficult to satisfy in practical applications, which leads to erroneous evaluation. Therefore, the trained model cannot guarantee that it satisfies the fairness requirement. The current studies tend to relax these assumptions and address the unfairness issue of algorithms in more general scenarios.

Privacy protection. Due to legal requirements, sensitive attributes are often inaccessible in real applications. Fairness constraints require predictors to be in some way independent of the attributes of group members. Privacy preservation raises the same question: Is it possible to guarantee that even the strongest adversary cannot steal an individual's private information through inference attacks? Causal modeling of the problem not only is helpful to solve the fairness issue but also enables stronger privacy-preserving than statistics-based methods ^[123]. Combining the existing fairness mechanism with differential privacy is a promising research direction in the near future.

Build a more complete ecosystem. There is an interaction of fairness between applications in the real world. For example, in bank loans, there exists discrimination in loan quotas for groups of different genders, and this unfairness may be caused by the salary level of groups of different genders in the workplace. Therefore, we need to further explore achieving cross-domain, cross-institution collaborative fairness algorithms.

Authors' contributions

Project administration: Yu G, Yan Z

Writing-original draft: Su C, Yu G, Wang J

Writing-review and editing: Yu G, Yan Z, Cui L

Availability of data and materials

Not applicable.

Financial support and sponsorship

None.

Conflicts of interest

All authors declared that they have no conflicts of interest to this work.

Ethical approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Copyright

© The Author(s) 2022.