Valley3: Scaling Omni Foundation Models for E-commerce
Zeyu Chen, Guanghao Zhou, Min Yang, Qixiang Yin, Ziwang Zhao, Huanjin Yao, Pengjiu Xia, Cen Chen, Minghui Qiu
Why It Matters
What makes this one worth your time
This research addresses the growing need for advanced AI models that can handle complex, multimodal tasks in the rapidly evolving e-commerce landscape.
Valley3 is a novel multimodal model tailored for diverse e-commerce applications.
Summary
The paper introduces Valley3, a multimodal large language model designed for e-commerce tasks, which integrates audio, text, images, and video processing capabilities, and employs a four-stage pre-training pipeline to enhance reasoning and domain knowledge.
Key contributions
- Development of a multimodal large language model specifically for e-commerce.
- Implementation of a four-stage pre-training pipeline for enhanced reasoning capabilities.
- Introduction of agentic search capabilities for proactive information retrieval.
Notable insights
- The model's unique four-stage pre-training pipeline allows for progressive learning of audio understanding and cross-modal instruction-following.
- The introduction of controllable reasoning modes enhances the model's adaptability to varying task complexities.
Possible limitations
- Not stated in the abstract.
Abstract
arXiv:2605.01278v3 Announce Type: replace Abstract: In this work, we present Valley3, an omni multimodal large language model (MLLM) developed for diverse global e-commerce tasks, with unified understanding and reasoning capabilities across text, images, video, and audio. A key feature of Valley3 is its native multilingual audio capability for e-commerce, developed by extending vision-language models to better support crucial audio-visual tasks, particularly in short-video scenarios. To achieve this, we carefully design a four-stage omni e-commerce continued pre-training pipeline, through which Valley3 progressively acquires audio understanding, cross-modal instruction-following, e-commerce domain knowledge, and long-context reasoning capabilities, ultimately evolving into an omni model for diverse e-commerce scenarios. Then, we further improve Valley3 through post-training to encourage long-chain reasoning with controllable reasoning modes, enabling one non-thinking mode and three distinct levels of thinking, thereby balancing inference efficiency in simple scenarios with deep reasoning for complex applications. Moreover, we equip Valley3 with agentic search capabilities to proactively invoke search tools and acquire task-relevant information for e-commerce deep research tasks. To comprehensively assess the capabilities of Valley3, we construct an omni e-commerce benchmark spanning 6 tasks. Experimental results show that Valley3 consistently outperforms strong baselines on our in-house and open-source e-commerce benchmarks, while remaining competitive on general-domain benchmarks. Our code and model weights are available at https://github.com/bytedance/Valley.