Skip to content

Commit 897ca31

Browse files
fix missing hf_eval_split option
1 parent 4962bb8 commit 897ca31

1 file changed

Lines changed: 1 addition & 0 deletions

File tree

end_to_end/tpu/test_convergence_1b_params.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ then
5252
gsutil cp -r gs://maxtext-dataset/hf/llama2-tokenizer "${MAXTEXT_ASSETS_ROOT:-${MAXTEXT_PKG_DIR:-${MAXTEXT_REPO_ROOT:-$PWD}/src/MaxText/assets}}"
5353
CMD_DATA=" hf_path=parquet tokenizer_path=${MAXTEXT_ASSETS_ROOT:-${MAXTEXT_PKG_DIR:-${MAXTEXT_REPO_ROOT:-$PWD}/src/MaxText/assets}}/llama2-tokenizer \
5454
hf_train_files=$DATASET_PATH/hf/c4/c4-train-*.parquet \
55+
hf_eval_split=train \
5556
hf_eval_files=$DATASET_PATH/hf/c4/c4-validation-*.parquet "
5657
fi
5758

0 commit comments

Comments
 (0)