Bias Amplification in Visual Recognition: How Structured Models Perpetuate and Exacerbate Social Stereotypes

Paper:
Title: Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Authors: Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang

How Structured Models Learn and Amplify Societal Biases in Web Data

Visual recognition tasks that incorporate language, such as captioning and visual semantic role labeling (vSRL), have emerged as critical avenues for extracting rich semantics from images. These tasks rely on large quantities of labeled data, predominantly sourced from the web. To make coherent predictions even with weak visual support, methods often combine structured prediction and deep learning to model correlations between labels and images.

However, the risk, as quantified by Zhao et al. (2017), is that these methods may "discover and exploit societal biases present in the underlying web corpora". Without proper calibration, the broad adoption of these models can have the inadvertent effect of magnifying stereotypes.

Key Findings: How Existing Bias is Amplified by Models

Analyzing both visual semantic role labeling (vSRL, using the imSitu dataset) and multilabel object classification (MLC, using MS-COCO), researchers confirmed a two-part phenomenon:

Significant Dataset Bias Exists: Datasets for both tasks contain significant gender bias. For example, in vSRL, over 45% of verbs and 37% of objects exhibit bias toward one gender greater than 2:1. The activity "cooking" is 33% more likely to involve females than males in the imSitu training set.

Bias Amplification After Training: After training state-of-the-art structured predictors, the models were shown to further amplify the existing bias.

• Case Example: For the activity "cooking," a trained Conditional Random Field (CRF) model amplified the training set disparity (33% male agent presence) to 68% disparity at test time. The proportion of men in the agent role for "cooking" images dropped from approximately 33% in the training set to 16% in the predicted set.
• Quantification: Trained models amplified existing bias by 5.0% for vSRL, and 3.6% in MLC on average.

The authors emphasize that deploying such uncalibrated systems could not only reinforce existing social bias but also actually make them worse.

Policy and Research Implications: Calibration via Constrained Inference

To mitigate the amplification of bias learned from biased corpora, the researchers proposed a novel constrained inference framework called RBA (Reducing Bias Amplification).

Intervention Mechanism: RBA injects corpus-level constraints to ensure that model predictions follow the distribution observed in the training data. For instance, constraints ensure that the noun "man" occurs in the agent role of "cooking" as often as it did in the original training corpus.

Results and Efficiency: Evaluation demonstrated RBA's effectiveness:

• RBA substantially reduced the average magnitude of bias amplification by 47.5% for MLC and 40.5% for vSRL.
• Crucially, this calibration method resulted in almost no performance loss for the underlying recognition task.

Research Outlook: This work is the first to demonstrate that structured prediction models amplify bias and to propose methods for reducing this effect. Future work includes applying these methods in other structured domains, such as pronoun reference resolution.