aigc-apps /VideoX-Fun

About VideoX-Fun

VideoX-Fun makes AI images and videos, and trains your own video models. It runs text-to-video, image-to-video, video-to-video, and control videos from Canny, Pose or Depth. You drive it with Python scripts, a Gradio web UI, or ComfyUI nodes. Weights take about 60GB of disk, and big models need offload or many GPUs.

Best for: Builders and researchers who want to run or fine-tune open video models on their own GPUs.

• Runs text-to-video, image-to-video, video-to-video and control videos
• Uses one script entry for video and image models under examples/{model_name}/
• Saves GPU memory with model_cpu_offload, qfloat8 or sequential_cpu_offload
• Splits work across GPUs with xfuser (ulysses_degree and ring_degree)
• Trains baseline, LoRA (a small add-on model), control and distill models
• Prepares data: splits long videos, cleans them and writes captions

1. 1
   Pull the Docker image the command below and run it with --gpus all.

   docker pull mybigpai-public-registry.cn-beijing.cr.aliyuncs.com/easycv/torch_cuda:cogvideox_fun

   Copy
2. 2
   Clone the repo with the command below, then cd VideoX-Fun.

   git clone https://github.com/aigc-apps/VideoX-Fun.git

   Copy
3. 3
   Make models/Diffusion_Transformer and models/Personalized_Model, then download weights from Hugging Face or ModelScope into them.
4. 4
   Edit the prompt and seed in examples/cogvideox_fun/predict_t2v.py, run it, and find videos in samples/cogvideox-fun-videos.
5. 5
   Or run examples/cogvideox_fun/app.py to open the Gradio interface and generate there.
6. 6
   For multi-GPU, install xfuser==0.4.2 and yunchang==0.6.2, then run the command below.

   torchrun --nproc-per-node=8 examples/wan2.1_fun/predict_t2v.py

   Copy