Can We Trust AI Explanations? Dr. Ana Marasović on AI Trustworthiness, Explainability & Faithfulness
In this conversation, Jekaterina Novikova and Malikeh Ehgaghi interview Ana Marasović https://www.anamarasovic.com, an expert in AI trustworthiness, focusing on the complexities of explainability, the realities of academic research, and the dynamics of human-AI collaboration. We discuss the importance of intrinsic and extrinsic trust in AI systems, the challenges of evaluating AI performance, and the implications of synthetic evaluations.
Don't have time for the full episode? Watch it in parts:
Part 1 - The Messy Journey of Research: • Ep 11 Part 1 - The Messy Journey of Resear...
Part 2 - The Role of Explainability and Reasoning Traces in AI Trust: • Ep 11 Part 2 - The Role of Explainability ...
Part 3 - Robustness and Trustworthy AI: • Ep 11 Part 3 - Robustness and Trustworthy ...
CHAPTERS
00:00 Introduction - Ana Marasović
02:15 The Messiness of Research
04:33 The Realities of Academic Life
08:21 Balancing Research and Funding
12:13 Understanding Trust in AI Systems
15:23 Exploring Explainability and Human Decision-Making
17:52 Application-Grounded Evaluations
22:09 The Gap Between Research and Real-World Applications
23:41 Counterintuitive Findings in Human-AI Collaboration
29:16 Balancing Human-AI Collaboration
31:01 Exploring Human-AI Teams
35:53 Understanding Reasoning Traces in AI
39:40 The Relationship Between Model Size and Reasoning
46:54 Evaluating Reasoning Language Models
50:54 Robustness in AI Systems
57:52 The State of AI Evaluation
01:01:31 Synthetic Evaluation: Pros and Cons
01:06:39 Human vs. LLM Annotations
01:08:41 Balancing Efficiency and Robustness in Benchmarking
01:14:55 Trust in AI: Intrinsic vs. Extrinsic
01:19:05 Future Directions in AI Research
REFERENCES:
17:56 On Evaluating Explanation Utility for Human-AI Decision Making in NLP (https://aclanthology.org/2024.finding...)
31:02 Effective Human-AI Teams via Learned Natural Language Rules and Onboarding (https://arxiv.org/abs/2311.01007)
41:10 Chain-of-Thought Unfaithfulness as Disguised Accuracy (https://arxiv.org/abs/2402.14897)
41:52 On Measuring Faithfulness or Self-consistency of Natural Language Explanations (https://arxiv.org/abs/2311.07466)
01:01:51 What Has Been Lost with Synthetic Evaluation? (https://arxiv.org/abs/2505.22830)
🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI.
WiAIR website:
♾️ https://women-in-ai-research.github.io
Follow us at:
♾️ LinkedIn: / women-in-ai-research
♾️ Bluesky: https://bsky.app/profile/wiair.bsky.s...
♾️ X (Twitter): https://x.com/WiAIR_podcast
#AI #airesearch #aiexplainability #aitrust #aifaithfulness #wiair #wiairpodcast