This repository contains code needed to replicate experiments discussed in "A Systematic Study of Bias Amplification".

module load anaconda3/2021.05
conda create --name conf_bias_amp python=3.7
conda activate conf_bias_amp

Ensure that torch.cuda.is_available() is true.

Cuda 11.1 isn't strictly necessary, but installing it allows us to get PyTorch 1.9+

module load cuda/11.1 
conda install pytorch torchvision cudatoolkit=11.1 -c pytorch -c nvidia
pip install -r requirements.txt

Ready to run!

Scripts needed to kick-off and analyze each experiment discussed in the paper can be found in respective folders in configs/. Each experiment directory contains a scripts/ directory which contains a sript generate_experiment_configs.py that can be executed to create the model configs and training_measurements*.py scripts for running offline measurements of key metrics like bias amplification and overconfidence. The description.txt file contains a short explanation of the experiment and useful notes for its exectution. The experiment directories should contain an empty models/ in which configs are stored following execution of generate_experiment_configs.py.

Other directories in the repository (ex: datasets\, losses\, models\) contain infrastructure for actually executing the model training process.

As an example, the following steps can be used to generate the FashionMNIST experiment configs:

• /my-project-release/my-project/configs/fashionmnist/scripts $ python generate_experiment_configs.py

After the training the models with the configs, you can generate results with:

• /my-project-release/my-project/configs/fashionmnist/scripts $ python training_measurements.py

Model results are now viewable in /my-project-release/my-project/configs/fashionmnist/scripts/results_overconf.py.

cv_bias_amplification is MIT-licensed, as found in the LICENSE file.