MCG-NJU /VideoJev

About VideoJev

VideoJev is an open model family (Apache-2.0) that answers questions about video without generating text: it scores candidate time intervals for 'when did X happen' and scores options for multiple-choice QA. Two checkpoints (4B and 35B-A3B) are on Hugging Face. It is fresh research code, so you need a CUDA GPU, FlashAttention 2 and the Qwen base models, and QA only works in multiple-choice form.

Best for: Builders who need fast 'when did X happen' checks or quiz-style verification over long footage — for example reviewing generated drama clips — not open-ended chat about video.

• Finds when something happens in a video (temporal grounding) by scoring candidate time intervals directly
• Answers multiple-choice questions about a video by scoring the options — no autoregressive decoding
• Reuses the video's visual computation across questions, so follow-up queries on the same clip stay cheap
• Ships two open checkpoints: VideoJev-4B (on Qwen3.5-4B) and VideoJev-35B-A3B (on Qwen3.6-35B-A3B), BF16 safetensors
• Paper reports +6.7 / +10.8 mIoU over the base models on temporal grounding, and 100x+ faster QA than Thinking mode
• Runs through Hugging Face Transformers with trust_remote_code; a 10-second demo script is in the README

1. 1
   Clone the repo: the command below then cd VideoJev

   git clone https://github.com/MCG-NJU/VideoJev.git

   Copy
2. 2
   Install the environment (Python 3.12, PyTorch 2.8 / CUDA 12.8, FlashAttention 2): the command below then source scripts/env.sh

   pip install -r requirements.txt

   Copy
3. 3
   Put the base models in models/Qwen3.5-4B and models/Qwen3.6-35B-A3B, or point VIDEOJEV_BASE_4B / VIDEOJEV_BASE_35B at them
4. 4
   Load the 4B checkpoint: the command below, then model.prepare_for_inference()

   AutoModel.from_pretrained("ZhuYuhan/VideoJev-4B", trust_remote_code=True, dtype=torch.bfloat16, attn_implementation="flash_attention_2")

   Copy
5. 5
   Ask questions with the command below — each item is the next command or the next command

   model.predict_video(processor, video, questions, fps=2.0)

   Copy

   {"task": "grounding", "query": ...}

   Copy

   {"task": "choice", "query": ..., "options": [...]}

   Copy
6. 6
   Swap the repo id to ZhuYuhan/VideoJev-35B-A3B for the larger MoE model