Limits of Transformers, with Dr. Nouha Dziri
Nouha Dziri https://nouhadziri.github.io/ is an AI research scientist at the Allen Institute for AI.
In this episode, we dive deep into the limitations of transformer models with Nouha Dziri, a research scientist at Allen Institute for AI. Nouha shares insights from her research on understanding the capabilities and constraints of LLMs. We discuss the challenges in reasoning, factuality, and ethical considerations that come with deploying these powerful AI systems. This conversation explores both technical aspects and broader implications for the future of AI research.
REFERENCES:
[00:01:17] Noha's google scholar profile
(https://shorturl.at/0dSnc)
[00:19:11] The Generative AI Paradox:" What It Can Create, It May Not Understand"
(https://arxiv.org/abs/2311.00059)
[00:27:11] The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
(https://arxiv.org/abs/2309.12288)
[00:33:30] Faith and Fate: Limits of Transformers on Compositionality
(https://arxiv.org/abs/2305.18654)
ToC:
[00:00:00] Introduction - Nouha Dziri
[00:04:10] Defining Success in AI Research
[00:07:46] Overcoming Challenges and Failures
[00:09:40] Navigating Gender and Cultural Stereotypes in AI
[00:16:02] Understanding the Limits of Language Models
[00:19:06] The Generative AI Paradox Explained
[00:33:17] Compositionality and Reasoning in AI
[00:37:51] Challenges in AI Training and Generalization
[00:43:41] The Future of AI Reasoning
[00:46:33] Collaboration and Safety in AI Research
WiAIR website:
♾️ https://women-in-ai-research.github.io
Follow us at:
♾️ LinkedIn: / women-in-ai-research
♾️ Bluesky: https://bsky.app/profile/wiair.bsky.s...
♾️ X (Twitter): https://x.com/WiAIR_podcast
#WiAIRpodcast #WiAIR