Multimodal papers

Browse recent papers, open the full write-up, or filter the archive by topic.

58 papers found

ReviewPaperPublishedTopics

7.5MODF-SIR: A Multi-agent Omni-modal Distilled Framework for Social Intelligence ReasoningThis work addresses the challenge of effectively utilizing long-tail events in social intelligence tasks, which is crucial for improving AI's understanding of nuanced social interactions.Jun 11, 2026LLM Agent Reasoning

7.2Information-Theoretic Decomposition for Multimodal Interaction LearningUnderstanding and leveraging sample-specific interactions can significantly enhance the performance of multimodal systems, which are increasingly relevant in AI applications.Jun 11, 2026Multimodal

7.2Multimodal Large Language Models as Synthetic Participants in Video-Based Studies: An EvaluationUnderstanding the limitations of MLLMs in approximating human subjective responses can inform future research and development in AI-driven evaluations.Jun 9, 2026LLM Evaluation Multimodal

6.8GlobeAudio: A Multilingual Multicultural Benchmark for Naturalistic Evaluation of Large Audio-Language ModelsUnderstanding the limitations of current LALMs in real-world, multilingual, and multicultural contexts is crucial for developing more effective and inclusive AI systems.Jun 9, 2026Multimodal Evaluation Benchmark

7.5TokaMind: A Multi-Modal Transformer Foundation Model for Tokamak Plasma DynamicsThis work provides a robust foundation for future research in fusion modeling, potentially accelerating advancements in plasma physics and fusion energy.Jun 8, 2026Multimodal Benchmark

6.8Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language ModelsThis research could reduce the dependency on expensive image-text datasets, making it easier and more cost-effective to train multimodal models at scale.Jun 8, 2026Multimodal Vision Language Alignment

7.5MVCL-DAF++: Enhancing Multimodal Intent Recognition via Prototype-Aware Contrastive Alignment and Coarse-to-Fine Dynamic Attention FusionThis research addresses critical challenges in multimodal intent recognition, making it relevant for applications requiring robust understanding in noisy environments.Jun 8, 2026Multimodal Alignment

7.2A robust PPG foundation model using multimodal physiological supervisionThis approach could lead to more reliable PPG-based applications in consumer devices and clinical settings by improving model performance on noisy data.Jun 8, 2026Multimodal Data

7.5Where Should Knowledge Enter? A Layered Framework for Knowledge Infusion in Multimodal Iterative Generative MoThis research addresses the critical challenge of ensuring reliability in generative models when handling domain-specific knowledge, which is essential for applications in safety-critical areas.Jun 6, 2026Multimodal Alignment Safety

7.2To Be Multimodal or Not to Be: Query-Adaptive Audio-Visual Person Retrieval via Active Modality DetectionThis approach can improve the precision of person retrieval systems in real-world scenarios where modalities may be inconsistently available, offering practical benefits for media archives and surveillance applications.Jun 6, 2026Multimodal Evaluation

7.2TRACE: A Temporal Conditional Estimation for Multimodal Time Series Foundation ModelsThis work is relevant for AI researchers and engineers dealing with real-world multimodal time series data, as it offers a method to improve model robustness and accuracy in the presence of missing or irregular data.Jun 6, 2026Multimodal Data

7.2Revisiting Model Stitching In the Foundation Model EraUnderstanding how to effectively stitch different VFMs can lead to more efficient and powerful multimodal models, which is crucial for advancing AI capabilities in complex tasks.Jun 5, 2026Multimodal Architecture Vision Language

7.5NoRA: Evaluating Grounded Reasonableness in Visual First-person Normative Action ReasoningThis research addresses a critical gap in evaluating AI's normative competence in real-world scenarios, moving beyond simple action selection to deeper reasoning.Jun 5, 2026Agent Reasoning Multimodal

7.5MIRAGE: Mobile Agents with Implicit Reasoning and Generative World ModelsThis work addresses the inefficiencies in mobile agent interactions, potentially streamlining applications that rely on visual and language inputs, which is crucial for real-world deployment.Jun 5, 2026Agent Reasoning Multimodal

7.5VGGSounder: Audio-Visual Evaluations for Foundation ModelsThis work is significant for researchers and engineers developing audio-visual models, as it provides a more reliable benchmark for assessing multi-modal understanding.Jun 5, 2026Multimodal Evaluation Benchmark

6.8CL-DMDF:Dynamic Multimodal Data Fusion Model Based on Contrastive LearningThe approach addresses the challenge of missing or uncertain modality inputs in real-world multimodal data fusion, potentially improving decision-making processes in complex environments.Jun 3, 2026Multimodal Data

6.8Perceive Before Reasoning: A Pre-Reasoning Perception Framework for Efficient and Reliable Proactive Mobile AgentsThis framework could improve the efficiency and reliability of mobile agents by reducing unnecessary interventions and optimizing resource use, which is crucial for real-time applications.Jun 3, 2026Agent Reasoning Multimodal

7.5MIND: Multi-rationale INtegrated Discriminative Reasoning Framework for Multi-modal Large ModelsThis work addresses critical limitations in current multimodal models, potentially leading to more reliable AI systems capable of complex reasoning tasks.Jun 3, 2026LLM Reasoning Multimodal

7.2Cosmos 3: Omnimodal World Models for Physical AIThis work could streamline the development of embodied AI systems by providing a versatile, open-source framework that integrates multiple modalities, potentially accelerating research and application in Physical AI.Jun 3, 2026Multimodal Agent Vision Language

7.2Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information DecompositionUnderstanding modality interactions can enhance the design and deployment of multimodal language models, potentially improving their performance in complex tasks.Jun 2, 2026Multimodal Vision Language Evaluation

7.5Balancing Multimodal Learning through Label Space ReshapingThis approach could enhance the performance of multimodal systems, making them more effective in real-world applications where modality imbalance is a common challenge.May 29, 2026Multimodal

7.5AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked ModelingThis work addresses the critical challenge of generating human motion across multiple modalities, which is essential for advancements in robotics and computer vision applications.May 29, 2026Multimodal Data Robotics

7.5A Conflict-Aware Penalty and Statistical Loss Framework for Balancing Modalities and Enhancing Stability in Multimodal Sentiment AnalysisThis work is significant for researchers and engineers in sentiment analysis and multimodal systems, as it proposes a method to effectively balance different modalities, potentially leading to more robust models.May 28, 2026Multimodal

7.2FLORO: A Multimodal Geospatial Foundation Model for Ecological Remote Sensing Across Sensors and ScalesThis work is relevant for AI researchers and engineers working on remote sensing applications, as it demonstrates effective model performance with smaller datasets, which is crucial for ecological and environmental monitoring.May 28, 2026Multimodal Data Benchmark

6.8Leveraging Foundation Models for Causal Generative ModelingThis research addresses the critical need for reliable AI systems capable of counterfactual reasoning, which is essential for applications in decision-making and interpretability.May 25, 2026Reasoning Multimodal Vision Language

7.0How Far Are We from Generating Missing Modalities with Foundation Models?Understanding and improving missing modality reconstruction can enhance the versatility and applicability of multimodal models in real-world scenarios where incomplete data is common.May 25, 2026Multimodal Evaluation Agent

7.2Chronicle: A Multimodal Foundation Model for Joint Language and Time Series UnderstandingThis work is significant for AI researchers interested in developing models that can seamlessly integrate and process multimodal data, potentially leading to more robust and versatile AI systems.May 22, 2026Multimodal Architecture Evaluation

7.5JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and GenerationThis research could significantly improve applications in vision-language-action systems, making it relevant for engineers and researchers working on multimodal AI.May 22, 2026Multimodal Vision Language Architecture

6.8TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-trainingA unified codebase like TorchUMM can streamline the evaluation process for multimodal models, making it easier for researchers to compare different architectures and improve model development.May 22, 2026Multimodal Evaluation Benchmark

7.2Is VLA Reasoning Faithful? Probing Safety of Chain-of-CausationUnderstanding the limitations of VLA models is crucial for improving their reliability in real-world applications, particularly in safety-critical environments.May 19, 2026Reasoning Evaluation Safety

7.5Seeing is Understanding: Unlocking Causal Attention into Modality-Mutual Attention for Multimodal LLMsThis work addresses a critical challenge in multimodal AI, potentially improving the accuracy and relevance of model outputs in applications that integrate text and images.May 18, 2026Multimodal Vision Language Alignment

7.2Brain-OF: An Omnifunctional Foundation Model for fMRI, EEG and MEGThis work is significant for AI researchers and engineers interested in neuroscience as it offers a unified model for integrating multiple neuroimaging modalities, potentially leading to more comprehensive brain activity analysis.May 18, 2026Multimodal Architecture Other

6.8fMRI-LM: Towards a Universal Foundation Model for Language-Aligned fMRI UnderstandingThis work is significant for researchers interested in the intersection of neuroscience and AI, as it proposes a method to link brain activity with language, potentially advancing cognitive neuroscience and AI applications.May 16, 2026LLM Multimodal Alignment

7.2GuardAD: Safeguarding Autonomous Driving MLLMs via Markovian Safety LogicImproving safety in autonomous driving systems is crucial for real-world deployment, and GuardAD offers a novel approach to address dynamic traffic scenarios.May 13, 2026Multimodal Safety Reasoning

7.2Pan-FM: A Pan-Organ Foundation Model with Saliency-Guided Masking for Missing RobustnessThis research addresses the challenge of missing data in multimodal medical imaging, which is crucial for developing more generalizable and robust AI models in healthcare.May 11, 2026Multimodal Data

8.2NeuroAgent: LLM Agents for Multimodal Neuroimaging Analysis and ResearchThis work addresses significant barriers in neuroimaging analysis, potentially accelerating research and improving reproducibility in a critical area of medical science.May 8, 2026LLM Agent Multimodal

7.5Toward Structural Multimodal Representations: Specialization, Selection, and Sparsification via Mixture-of-ExpertsThis framework provides a practical alternative to existing methods in multimodal learning, potentially enhancing performance and efficiency in various applications.May 7, 2026Multimodal Benchmark

7.2Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecastingThis work is relevant for AI researchers and engineers focused on climate science, as it provides a versatile tool for integrating and forecasting diverse environmental data, potentially improving predictions of extreme weather events.May 6, 2026Multimodal Data Architecture

6.8Quantifying Multimodal Capabilities: Formal Generalization Guarantees in Pairwise Metric LearningUnderstanding the generalization properties of multimodal learning models can lead to improved performance and efficiency in real-world applications where data modalities are often incomplete or redundant.May 6, 2026Multimodal Evaluation

8.2OphMAE: Bridging Volumetric and Planar Imaging with a Foundation Model for Adaptive Ophthalmological DiagnosisThis research addresses a critical gap in ophthalmic AI by enabling effective multi-modality analysis, which is essential for accurate clinical diagnosis and could significantly enhance patient care.May 6, 2026Multimodal Benchmark

6.8How Well Does GPT-4o Understand Vision? Evaluating Multimodal Foundation Models on Standard Computer Vision TasksUnderstanding the capabilities and limitations of multimodal models in vision tasks can guide their application and development in AI systems.May 5, 2026Multimodal Evaluation Benchmark

7.2UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion PriorsThis framework could streamline the development of multimodal video applications by reducing the need for separate models for each task, potentially saving resources and improving cross-modal consistency.May 4, 2026Multimodal Vision Language

6.8Compose and Fuse: Revisiting the Foundational Bottlenecks in Multimodal ReasoningUnderstanding and overcoming the integration challenges in multimodal reasoning can lead to more effective AI systems that leverage diverse data sources.May 1, 2026Reasoning Multimodal Evaluation

7.0Beyond Text-Dominance: Understanding Modality Preference of Omni-modal Large Language ModelsUnderstanding modality preference in OLLMs can lead to more balanced and trustworthy models, improving their performance in multi-modal tasks.Apr 30, 2026LLM Multimodal Vision Language

6.8Source-Modality Monitoring in Vision-Language ModelsUnderstanding source-modality monitoring can enhance the robustness and reliability of multimodal systems, which are increasingly prevalent in AI applications.Apr 27, 2026Multimodal Vision Language Evaluation

6.8Focus Session: Hardware and Software Techniques for Accelerating Multimodal Foundation ModelsThis work is relevant for AI engineers and researchers looking to improve the efficiency and performance of multimodal foundation models, which are increasingly used in diverse applications.Apr 27, 2026Multimodal Infra Architecture

7.0Towards Safe Mobility: A Unified Transportation Foundation Model enabled by Open-Ended Vision-Language DatasetThis work addresses the gap in city-scale traffic analysis by providing a dataset and model that could enhance safety in urban transportation systems.Apr 27, 2026Multimodal Vision Language Safety

7.5The Expense of Seeing: Attaining Trustworthy Multimodal Reasoning Within the Monolithic ParadigmUnderstanding the limitations of VLMs is crucial for developing more reliable AI systems that can effectively integrate and reason across multiple modalities.Apr 23, 2026Multimodal Evaluation Vision Language

7.0SpeechParaling-Bench: A Comprehensive Benchmark for Paralinguistic-Aware Speech GenerationUnderstanding and improving paralinguistic cues in speech generation is crucial for developing more natural and effective human-computer interactions.Apr 23, 2026Benchmark Evaluation Multimodal

7.2MMCORE: MultiModal COnnection with Representation Aligned Latent EmbeddingsThis work is relevant for AI researchers and engineers interested in efficient multimodal image generation and editing, as it promises reduced computational costs and improved synthesis quality.Apr 23, 2026Multimodal Vision Language