Multiscale Pipeline [WIP: DO NOT MERGE]

[Serenagu525]

multiscale text-to-vid pipeline with rectified flow scheduler

Running Instructions

1. Move files here https://github.com/Lightricks/LTX-Video/tree/main/ltx_video/models/autoencoders under src/maxdiffusion/models/ltx_video/models/autoencoders.
2. For ltx_video/models/autoencoders/causal_video_autoencoder.py, change line 22 to where your attention file is
3. Move files here https://github.com/Lightricks/LTX-Video/tree/main/ltx_video/utils to src/maxdiffusion/models/ltx_video/utils
4. Move https://github.com/Lightricks/LTX-Video/blob/main/ltx_video/models/transformers/symmetric_patchifier.py to under src/maxdiffusion/models/ltx_video/transformers
5. Add your safetensor location for ltxvid model to models_dir in src/maxdiffusion/configs/ltx_video.yml
6. Add your jax weights location to ckpt_path in src/maxdiffusion/models/ltx_video/xora_v1.2-13B-balanced-128.json
7. Edit generation paramters in src/maxdiffusion/configs/ltx_video.yml
8. Run text-to-vid generation script using command python src/maxdiffusion/generate_ltx_video.py src/maxdiffusion/configs/ltx_video.yml

TODO

1. Integrate vae
2. Clean diffuser dependencies
3. Dtype, generator coordination
4. Reorg code to make donwloading easier
5. Batching