|
|
|
|
|
|
|||||
|
|
|
||||||||
|
|
|
| Significant advancements have been achieved in the domain of face generation with the adoption of diffusion models. However, diffusion models tend to amplify biases during the generative process, resulting in an uneven distribution of sensitive facial attributes such as age, gender, and race. In this paper, we introduce a novel approach to address this issue by debiasing the attributes in the images generated by diffusion models. Our approach involves disentangling facial attributes by localizing the means within the latent space of the diffusion model using Gaussian mixture models (GMM). This method, leveraging the adaptable latent structure of diffusion models, allows us to localize the subspace responsible for generating specific attributes on-the-fly without the need for retraining. We demonstrate the effectiveness of our technique across various face datasets, resulting in fairer data generation while preserving sample quality. Furthermore, we empirically illustrate its effectiveness in reducing bias in downstream classification tasks without compromising performance by augmenting the original dataset with fairly generated data. |
|
|
| Our approach, GAMMA-FACE shown in the Figure, addresses bias in downstream classification tasks related to sensitive attributes such as gender, race, and age. We utilize GMM in the noisy latent space of diffusion models to segregate features in a high-dimensional space and generate images by sampling uniformly from each component. These images are assigned pseudo-labels for attributes by pre-trained classifiers. Additionally, for downstream tasks like smile, glasses, or hair color classification, we augment original training data with generated data and their pseudo-labels. Our results demonstrate improved overall accuracy and reduced bias (with respect to the protected attributes) in target classification tasks. |
|
| Quantitative results of GAMMA-Face on FairFace and FFHQ using bias evaluation metrics: Bias (B), Bias Amplification (BA), Overall accuracy (Acc.), Bias Performance Coefficient (BPC) and KL divergence (KL). |
|
| An exemplar set of face images generated from GAMMA-Face after localizing the image attributes in the latent space of the DDPM for FFHQ and FairFace datasets. |
Acknowledgements |