Context-specific Credibility-aware Multimodal Fusion with Conditional Probabilistic Circuits
Pranuthi Tenali, Sahil Sidheekh, Saurabh Mathur, Erik Blasch, Kristian Kersting, Sriraam Natarajan
Why It Matters
What makes this one worth your time
This work is relevant for AI engineers and researchers dealing with multimodal data, as it offers a method to improve fusion accuracy in scenarios where data sources may become unreliable due to context-specific factors.
C$^2$MF enhances multimodal fusion by dynamically assessing source reliability with Conditional Probabilistic Circuits.
Summary
The paper introduces C$^2$MF, a framework for context-specific credibility-aware multimodal fusion using Conditional Probabilistic Circuits to model per-instance source reliability. It proposes a new measure, Context-Specific Information Credibility (CSIC), to assess reliability adaptively and evaluates the framework's robustness using a new Conflict benchmark, showing significant improvements in predictive accuracy over static-reliability baselines.
Key contributions
- Development of the C$^2$MF framework for context-specific multimodal fusion.
- Introduction of the Context-Specific Information Credibility (CSIC) measure.
- Proposal of the Conflict benchmark for evaluating cross-modal conflicts.
Notable insights
- The use of Conditional Probabilistic Circuits allows for adaptive modeling of source reliability on a per-instance basis.
- The introduction of the Conflict benchmark provides a novel way to evaluate multimodal fusion robustness under intentionally induced discrepancies.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2603.26629v2 Announce Type: replace Abstract: Multimodal fusion requires integrating information from multiple sources that may conflict depending on context. Existing fusion approaches typically rely on static assumptions about source reliability, limiting their ability to resolve conflicts when a modality becomes unreliable due to situational factors such as sensor degradation or class-specific corruption. We introduce C$^2$MF, a context-specfic credibility-aware multimodal fusion framework that models per-instance source reliability using a Conditional Probabilistic Circuit (CPC). We formalize instance-level reliability through Context-Specific Information Credibility (CSIC), a KL-divergence-based measure computed exactly from the CPC. CSIC generalizes conventional static credibility estimates as a special case, enabling principled and adaptive reliability assessment. To evaluate robustness under cross-modal conflicts, we propose the Conflict benchmark, in which class-specific corruptions deliberately induce discrepancies between different modalities. Experimental results show that C$^2$MF improves predictive accuracy by up to 29% over static-reliability baselines in high-noise settings, while preserving the interpretability advantages of probabilistic circuit-based fusion.