Skip to content

Commit a2a2f34

Browse files
Merge pull request #3383 from AI-Hypercomputer:hengtaoguo-uxr
PiperOrigin-RevId: 882200014
2 parents b296e38 + 40f4870 commit a2a2f34

1 file changed

Lines changed: 5 additions & 2 deletions

File tree

docs/guides/checkpointing_solutions/convert_checkpoint.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ Third, setup following environment variables for conversion script
5151

5252
```bash
5353
# -- Model configuration --
54-
export HF_MODEL=<Hugging Face Model to be converted to MaxText> # e.g. 'llama3.1-8b-Instruct'
54+
export MODEL_NAME=<Hugging Face Model to be converted to MaxText> # e.g. 'llama3.1-8b-Instruct'
5555
export HF_TOKEN=<Hugging Face access token> # your token to access gated HF repos
5656

5757
# -- MaxText configuration --
@@ -67,8 +67,11 @@ export LAZY_LOAD_TENSORS=<Flag to lazy load> # True to use lazy load, False to u
6767
Finally, run below command to complete the conversion
6868

6969
```bash
70+
# Optional: If run out of disk space when downloading HuggingFace safetensors,
71+
# customize your "HF_HOME" to redirect the cache to a larger or mounted disk (e.g., on a TPU VM).
72+
# export HF_HOME="/dev/shm/huggingface_tmp"
7073
python3 -m maxtext.checkpoint_conversion.to_maxtext maxtext/configs/base.yml \
71-
model_name=${HF_MODEL?} \
74+
model_name=${MODEL_NAME?} \
7275
hf_access_token=${HF_TOKEN?} \
7376
base_output_directory=${MODEL_CHECKPOINT_DIRECTORY?} \
7477
scan_layers=True \

0 commit comments

Comments
 (0)