Skip to content

Commit b53cf74

Browse files
Lora README update (#155)
1 parent d9ce056 commit b53cf74

File tree

1 file changed

+14
-16
lines changed

1 file changed

+14
-16
lines changed

README.md

+14-16
Original file line numberDiff line numberDiff line change
@@ -120,51 +120,49 @@ Then you can run the finetune with:
120120
```
121121
bash scripts/finetune/finetune_mochi.sh # for mochi
122122
```
123-
**Note that for finetuning, we did not tune the hyperparameters in the provided script**
123+
**Note that for finetuning, we did not tune the hyperparameters in the provided script.**
124124
### ⚡ Lora Finetune
125-
Demos and prompts of Black-Myth-Wukong can be found in [here](https://huggingface.co/FastVideo/Hunyuan-Black-Myth-Wukong-lora-weight). You can download the Lora weight through:
125+
126+
Hunyuan supports Lora fine-tuning of videos up to 720p. Demos and prompts of Black-Myth-Wukong can be found in [here](https://huggingface.co/FastVideo/Hunyuan-Black-Myth-Wukong-lora-weight). You can download the Lora weight through:
126127
```bash
127128
python scripts/huggingface/download_hf.py --repo_id=FastVideo/Hunyuan-Black-Myth-Wukong-lora-weight --local_dir=data/Hunyuan-Black-Myth-Wukong-lora-weight --repo_type=model
128129
```
130+
#### Minimum Hardware Requirement
131+
- 40 GB GPU memory each for 2 GPUs with lora.
132+
- 30 GB GPU memory each for 2 GPUs with CPU offload and lora.
133+
134+
129135
Currently, both Mochi and Hunyuan models support Lora finetuning through diffusers. To generate personalized videos from your own dataset, you'll need to follow three main steps: dataset preparation, finetuning, and inference.
130136

131137
#### Dataset Preparation
132138
We provide scripts to better help you get started to train on your own characters!
133-
You can run this to organize your dataset to get the videos2caption.json before preprocess. Specify your video folder and corresponding caption folder(Caption files should be .txt files and have the same name with its video):
139+
You can run this to organize your dataset to get the videos2caption.json before preprocess. Specify your video folder and corresponding caption folder (caption files should be .txt files and have the same name with its video):
134140
```
135141
python scripts/dataset_preparation/prepare_json_file.py --video_dir data/input_videos/ --prompt_dir data/captions/ --output_path data/output_folder/videos2caption.json --verbose
136142
```
137143
Also, we provide script to resize your videos:
138144
```
139-
python scripts/data_preprocess/resize_videos.py \
140-
--input_dir data/raw_videos/ \
141-
--output_dir data/resized_videos/ \
142-
--width 1280 \
143-
--height 720 \
144-
--fps 30
145+
python scripts/data_preprocess/resize_videos.py
145146
```
146147
#### Finetuning
147148
After basic dataset preparation and preprocess, you can start to finetune your model using Lora:
148149
```
149150
bash scripts/finetune/finetune_hunyuan_hf_lora.sh
150-
bash scripts/finetune/finetune_mochi_lora.sh
151151
```
152152
#### Inference
153-
For inference with Lora checkpoint, you can run the following scripts with Additional parameter --lora_checkpoint_dir:
153+
For inference with Lora checkpoint, you can run the following scripts with additional parameter `--lora_checkpoint_dir`:
154154
```
155155
bash scripts/inference/inference_hunyuan_hf.sh
156-
bash scripts/inference/inference_mochi_hf.sh
157156
```
158-
#### Minimum Hardware Requirement
159-
- 40 GB GPU memory each for 2 GPUs with lora
160-
- 30 GB GPU memory each for 2 GPUs with CPU offload and lora.
157+
**We also provide scripts for Mochi in the same directory.**
158+
161159
#### Finetune with Both Image and Video
162160
Our codebase support finetuning with both image and video.
163161
```bash
164162
bash scripts/finetune/finetune_hunyuan.sh
165163
bash scripts/finetune/finetune_mochi_lora_mix.sh
166164
```
167-
For Image-Video Mixture Fine-tuning, make sure to enable the --group_frame option in your script.
165+
For Image-Video Mixture Fine-tuning, make sure to enable the `--group_frame` option in your script.
168166

169167
## 📑 Development Plan
170168

0 commit comments

Comments
 (0)