FaCT: Faithful Concept Traces for Explaining Neural Network Decisions

TL;DR

FaCT combines concept-discovery with model-inherent attributions to construct a model that provides faithful concept traces for explaining its decisions, i.e., contributions of pixels to concepts and concepts to the final decision can be faithfully traced. We also propose a novel concept-consistency metric, C2-Score, and show that FaCT yields more consistent and interpretable concepts while retaining competitive performance.

Video

Hosted by YouTube. Pressing play loads the video from Google, subject to their privacy policy.

Abstract

Deep networks have shown remarkable performance across a wide range of tasks, yet getting a global concept-level understanding of how they function remains a key challenge. Many post-hoc concept-based approaches have been introduced to understand their workings, yet they are not always faithful to the model. Further, they make restrictive assumptions on the concepts a model learns, such as class-specificity, small spatial extent, or alignment to human expectations. In this work, we put emphasis on the faithfulness of such concept-based explanations and propose a new model with model-inherent mechanistic concept-explanations. Our concepts are shared across classes and, from any layer, their contribution to the logit and their input-visualization can be faithfully traced. We also leverage foundation models to propose a new concept-consistency metric, C -Score, that can be used to evaluate concept-based methods. We show that, compared to prior work, our concepts are quantitatively more consistent and users find our concepts to be more interpretable, all while retaining competitive ImageNet performance.

Poster

Slides