Ovis-Embedding Advances Universal Omni-Modal Embeddings

Summary

The report introduces Ovis-Embedding, an omni-modal embedding family designed to represent text, images, video, and audio in a shared representation space. Rather than combining separate modality-specific towers, it integrates these modalities through a shared multimodal backbone. The backbone starts from a pretrained Qwen-omni model and is adapted with contrastive training and low-rank initialization. The authors also build a broad training corpus containing single-modality and interleaved multimodal data, and use homogeneous-source sampling to create task-consistent batches with informative in-batch negatives. For embedding-specific optimization, the method applies focal loss to emphasize difficult examples and similarity-based Embedding Distillation to transfer fine-grained similarity structure from complementary experts. During inference, low-rank feature decomposition produces compact embeddings with flexible dimensionality while aiming to preserve performance. The reported family achieves state-of-the-art results on MMEB-v3, MMEB-v2, MVEB, MAEB, and RTEB across text, image, video, and audio evaluations. The report presents these results as evidence that unified omni-modal training can reduce modality fragmentation and support universal any-to-any retrieval, although the abstract does not provide individual benchmark scores or further experimental details.