GitXplorerGitXplorer
n

encoder-decoders

public
1 stars
0 forks
0 issues

Commits

List of commits on branch main.
Unverified
342c9758c77774fba9b71f4dc459437a07872200

basic run file

nnbroad1881 committed a year ago
Unverified
c1b3f8507792c78a095bfe90ce9e9a938c61bbfc

basic llama

nnbroad1881 committed a year ago
Unverified
31d4153ad68fe0ded330d463c408de7f0bbb47fb

readme

nnbroad1881 committed a year ago
Verified
54fd6b8739aa9324ec6763df254d4d3d330e1753

Initial commit

nnbroad1881 committed a year ago

README

The README file for this repository.

encoder-decoders

Use models like Llama as an encoder-decoder

Models

  • bert
  • roberta
  • gpt2
  • [WIP] Llama
  • [WIP] GPTBigCode

Usage

python run_seq2seq.py \
  --encoder_model_name_or_path "bert-base-cased" \
  --decoder_model_name_or_path "bert-base-cased" \
  --output_dir "bert-encoder-decoder" \
  --dataset_name samsum \
  --fp16 \
  --do_train \
  --do_eval \
  --overwrite_output_dir \
  --evaluation_strategy "epoch" \
  --logging_steps 10 \
  --predict_with_generate \
  --max_source_length 512 \
  --save_strategy "epoch" \
  --save_total_limit 1 \
  --per_device_train_batch_size 16