GitXplorerGitXplorer
z

shine

public
7 stars
1 forks
1 issues

Commits

List of commits on branch master.
Verified
7991719cccc609675637c1d1d672f29a42ba5d68

added fallback ratio script (#20)

zzaccharieramzi committed 3 years ago
Verified
2bba35919e8577b3e3502e7c8ce3cb02783cbe6f

Improved Figure 4. (#19)

zzaccharieramzi committed 3 years ago
Verified
f25168c7e61ce39c70dc0e43d874ef03cdaad6f0

Allowed to generate figures from aggregated numbers (#18)

zzaccharieramzi committed 3 years ago
Verified
e829710ac98c725d23690ed7ff857b39e0368931

Figures and Readme for paper. (#16)

zzaccharieramzi committed 4 years ago
Verified
d2af2b91b6053a9115c18424ace152ee033fda07

Introduced Adjoint Broyden and OPA (#14)

zzaccharieramzi committed 4 years ago
Verified
79b6fd28ec4a42b1fc73987d30db07f871fc6c66

Corrected refine, seeding and adding plot file (#15)

zzaccharieramzi committed 4 years ago

README

The README file for this repository.

MDEQ - SHINE

This is the second part of the code for the paper "SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit models", submitted at the 2022 ICLR conference. The first part of the code to reproduce the Bi-level optimizations experiments is available here. This source code allows to reproduce the experiments on multiscale DEQs, i.e. Figure 3, and Figure E.2. in Appendix. This repo is based on the original mdeq repo by @jerrybai1995.

General instructions

You need Python 3.7 or above to run this code. This code will only run on a computer equipped with a GPU. You can then install the requirements with: pip install -r requirements.txt.

Reproducing Figure 3, DEQ

You can reproduce Figure 3 of the paper with the following sequence of scripts:

# cifar
python paper_trainings.py
python paper_backward_times.py
# imagenet
python paper_trainings.py --dataset imagenet --n_runs 1 --refines 0,5,None
python paper_backward_times.py --dataset imagenet
python paper_plot.py

You can further indicate how many gpus to use in each script with the --n_gpus option (default for training is 4). You can find other options using the --help option. Beware:

  • each CIFAR training is 11hours to 15 hours long (100 of them by default)
  • each ImageNet training is 3 days to 7 days long (6 of them by default)

For a practical reproduction you might want to run those in an HPC (i.e. change line 56-60 to work with e.g. submitit). For a test use, you can use the --n_runs (the number of repetitions for the error bar) and --n_refines (the number of points on the Pareto curve) options.

You can also just do the CIFAR trainings, by simply not running the ImageNet ones. The Figure will still be generated.

Reproducing Figure E.2., Quality of the inversion using OPA in DEQs

You can reproduce Figure E.2. of the paper with the following script:

python mdeq_lib/tests/modules/adj_broyden_correl.py

This should take about 15 mins to run with a single GPU.