Skip to content

add kwargs for compatibilities - #1

Open
nafkhanzam wants to merge 1 commit into
indobenchmark:masterfrom
nafkhanzam-thesis:master
Open

add kwargs for compatibilities#1
nafkhanzam wants to merge 1 commit into
indobenchmark:masterfrom
nafkhanzam-thesis:master

Conversation

@nafkhanzam

Copy link
Copy Markdown

Prevent this kind of error:

Traceback (most recent call last):
  File "main.py", line 66, in <module>
    main(args)
  File "/home/nafkhanzam/kode/nafkhanzam/thesis/AMRBART/indobart-pre-train/run_multitask_unified_pretraining.py", line 1301, in main
    global_step, tr_loss = train(
  File "/home/nafkhanzam/kode/nafkhanzam/thesis/AMRBART/indobart-pre-train/run_multitask_unified_pretraining.py", line 324, in train
    save_dummy_batch(
  File "/home/nafkhanzam/kode/nafkhanzam/thesis/AMRBART/indobart-pre-train/common/utils.py", line 78, in save_dummy_batch
    ith_tok_dict["input_tokens"] = ids_to_clean_text(tokenizer, input_ids[idx])
  File "/home/nafkhanzam/kode/nafkhanzam/thesis/AMRBART/indobart-pre-train/common/utils.py", line 65, in ids_to_clean_text
    gen_text = tokenizer.batch_decode(generated_ids)
  File "/home/nafkhanzam/.conda/envs/thesis/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 3328, in batch_decode
    return [
  File "/home/nafkhanzam/.conda/envs/thesis/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 3329, in <listcomp>
    self.decode(
TypeError: decode() got an unexpected keyword argument 'clean_up_tokenization_spaces'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant