Given a target dataset and reference dataset with a known sensitive attribute, the tool provides different metrics.

• Metrics to compare the quality and the diversity of the target and the reference dataset for each value of the sensitive attribute.
• Metrics to assess the balance level of the target dataset with respect to the sensitive attribute.

Bias Mitigation

Given a model, a sensitive attribute and a balanced dataset, this service provides the user with some approaches that can mitigate the bias.

First, we recommend to measure the bias of training dataset and if it applies to a generated dataset.

Then we provide guidelines on how to perform pre-processing and in-processing bias mitigation approaches.

Pre-processing mitigation: We propose a reweighing or preferential sampling strategy. It computes a weight for each data instance depending on the value of its sensitive attribute and its label (truth). The weight is then used for the data sampling when building the data batch in the learning process. The weights are obtained with the following equation, they are meant to remove the disparate impact from the training dataset:

with s a value of the sensitive attribute and c a value of the label.

In-processing mitigation: We propose conditional learning, which consists in using the sensitive attribute as a condition for the generation during the learning. This approach is suitable only if each sensitive attribute value is well represented in the training dataset.

Each proposed approach will be illustrated by commented code such that the user can reproduce it on their use case seamlessly.

The tool is a python package with easy-to-use command lines. It comes with commented example configuration files that explain the different parameters of each implemented metric. The tool generates the metrics results in CSV format and as figures.