GitXplorerGitXplorer
b

pytorch-xla-transformer-language-model

public
2 stars
2 forks
0 issues

Commits

List of commits on branch master.
Unverified
76941518d479e3275d17b86df0e7c322687f65e6

Merge branch 'master' of github.com:bmccann/pytorch-xla-transformer-language-model

bbmccann committed 6 years ago
Unverified
ab149786de7c6166c3f99c63738673a2aec2462a

adding data, train, and model

bbmccann committed 6 years ago
Verified
4079c93b9f783a36141bbcd4dc5d5f65af598a1d

Update README.md

bbmccann committed 6 years ago
Unverified
31fdffae2ab8d8d78ab6a444b742e911aad65914

update README

bbmccann committed 6 years ago
Unverified
6e8144290ba4f0f437930467a984cff8294fc807

adding transformer and train code

bbmccann committed 6 years ago
Verified
5f3dac1ffa3ca4cfae86ce81f7f921fa44439c76

Update README.md

bbmccann committed 6 years ago

README

The README file for this repository.

pytorch-xla-transformer-language-model

This repository is an open source test case for pytorch/xla that runs a minimal training loop for a Transformer language model on a single TPU device.

This code is intended to be used as reference for testing the compilation of the model by XLA, and is not intended to be used for training a reasonable language model. During initial runs, this code triggered recompilation far too often, but these issues have now been resolved.

Depends on Docker image gcr.io/tpu-pytorch/xla:r0.1.

export TPU_IP=#YOU MUST SET YOUR TPU IP
export XRT_TPU_CONFIG="tpu_worker;0;$TPU_IP"
export XLA_USE_32BIT_LONG=1
export XLA_IR_DEBUG=1
export XLA_HLO_DEBUG=1
python3 train.py

Output is in run.log.