Authors: Apavou Clément & Belkada Younes & Zucker Arthur

The kaggle challenge is the following : https://www.kaggle.com/c/altegrad-2021/leaderboard

In this challenge, we are given a large scientific citation graph, with each node corresponding to a certain article. The dataset consists of 138 499 vertices i.e articles, with their associated abstract and list of authors. The goal is to be able to predict whether two nodes are citing each other, given all this information. In the next sections, we will try to elaborate on the various intuitions behind our approaches, and present the obtained results as well as some possible interpretations for each observations. The provided code corresponds to the code that we have used for the best model (i.e the right commit ).

pip3 install requirements.txt

Then,

sh download_data.sh

python3 main.py

The best model can be used using the best-model branch, as it does not use this implementation of the code. This branch is the final code as it allows customization of the various embeddings and corresponds to the latest version of the code.

Model	loss validation	loss test (private leaderboard)	Run
Best model	0.07775	0.07939

All experiments are available on wandb:

Report & Slides

@misc{cohan2020specter,
      title={SPECTER: Document-level Representation Learning using Citation-informed Transformers}, 
      author={Arman Cohan and Sergey Feldman and Iz Beltagy and Doug Downey and Daniel S. Weld},
      year={2020},
      eprint={2004.07180},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}