Distinguished Lecture: Influence-directed explanations for deep networks

Monday, November 12, 2018 -
4:00pm to 5:00pm
CS 1240

Speaker Name: 

Anupam Datta

Speaker Institution: 

Carnegie Mellon University



Cookies Location: 

CS 1240



Deep neural networks are often inscrutable black boxes: they are
effective predictors but lack human-understandable explanations. I
will describe a new paradigm for influence-directed explanations that
addresses this problem. Distinctively, influence-directed explanations
peer inside the network to identify neurons with high influence on a
quantity and distribution of interest, using an axiomatically
justified influence measure, and then providing an interpretation for
the concepts these neurons represent. We evaluate our approach by
demonstrating a number of its unique capabilities on convolutional
neural networks trained on ImageNet. Our evaluation demonstrates that
influence-directed explanations (1) identify influential concepts that
generalize across instances, (2) can be used to extract the “essence”
of what the network learned about a class, and (3) isolate individual
features the network uses to make decisions and distinguish related


Anupam Datta is Professor of Electrical and Computer Engineering and
(by courtesy) Computer Science at Carnegie Mellon University. He is
Director of the Accountable Systems Lab. His work has helped create
foundations and tools for accountable data-driven systems. Specific
results include an accountability tool chain for privacy compliance
deployed in industry, automated discovery of gender bias in the
targeting of job-related online ads, principled tools for explaining
decisions of artificial intelligence systems, and monitoring audit
logs to ensure privacy compliance.

Datta serves on the Steering Committees of the ACM Conference on
Fairness, Accountability, and Transparency and the IEEE Computer
Security Foundations Symposium, and as an Editor-in-Chief of
Foundations and Trends in Privacy and Security. He obtained Ph.D. and
M.S. degrees from Stanford University and a B.Tech. from IIT
Kharagpur, all in Computer Science.