InputVideo + natural-language prompt
MMoTEMixture of Task Experts
Pass 1 · observing video
Expert SelectionPrompt-conditioned routing via cosine similarity
P
Waiting for completed prompt
E1Current Action Prediction—
E2Activity Classification—
E3Next Step Prediction—
E4Feedback Generation—
Video ProcessorSampled frames → visual tokens
→
Visual
Encoderfrozen
→Encoderfrozen
MLP
Projectortrainable
→Projectortrainable
◆◆◆
EXPERT ID—
INPUT TOKENS
PROMPT◆◆◆VISUAL
→
Routing in VideoLLM-MoTE
SIMPLIFIED DECODER LAYER
✦MODEL RESPONSE