Decentralized AI, with Wanru Zhao
🔍 𝐂𝐚𝐧 𝐭𝐡𝐞 𝐟𝐮𝐭𝐮𝐫𝐞 𝐨𝐟 𝐀𝐈 𝐛𝐞 𝐝𝐞𝐜𝐞𝐧𝐭𝐫𝐚𝐥𝐢𝐳𝐞𝐝? 𝐇𝐨𝐰 𝐝𝐨 𝐰𝐞 𝐬𝐜𝐚𝐥𝐞 𝐛𝐞𝐲𝐨𝐧𝐝 𝐬𝐜𝐚𝐥𝐢𝐧𝐠 𝐥𝐚𝐰𝐬? 𝐀𝐧𝐝 𝐰𝐡𝐚𝐭 𝐝𝐨𝐞𝐬 𝐢𝐭 𝐫𝐞𝐚𝐥𝐥𝐲 𝐭𝐚𝐤𝐞 𝐭𝐨 𝐛𝐮𝐢𝐥𝐝 𝐢𝐧𝐜𝐥𝐮𝐬𝐢𝐯𝐞, 𝐦𝐮𝐥𝐭𝐢𝐥𝐢𝐧𝐠𝐮𝐚𝐥 𝐋𝐋𝐌𝐬?
In this episode of the #WiAIRpodcast, Wanru Zhao discusses decentralized and collaborative AI methods, the limitations of scaling laws, fine-tuning strategies, data attribution challenges in LLMs, and multilingual learning in federated settings—all while reflecting on her experiences across UK and Canadian research ecosystems. She also shares her academic journey, vision for the future of AI, and how she navigates research roadblocks with creativity and curiosity.
🧠 Whether you're building #llms, exploring #FederatedLearning, or just passionate about more inclusive and sustainable AI research—this episode is packed with insights, encouragement, and visionary thinking.
👉 Watch now and be part of the future of AI that’s collaborative, global, and radically inclusive.
ToC:
[00:00] Introduction to Wanru Zhao and Her Journey
[02:56] Academic Pathway: From MPhil to PhD
[06:29] Research Ecosystems: UK vs Canada
[09:56] Collaboration Between Industry and Academia
[12:46] Navigating Challenges in Research
[16:13] Experiences of Being an Outsider in AI
[18:37] Breaking Unspoken Rules in Academia
[21:33] Scaling Laws in Language Models
[25:57] Challenges in Data Attribution for LLMs
[27:17] Leveraging High-Quality Public Data
[30:37] Model Merging Techniques for Fine-Tuning
[32:42] The big picture behind CLUES
[35:17] Future Directions in Collaborative LLM Development
[36:39] Incentivizing Contributions in Collaborative Learning
[38:02] Challenges in Multilingual Fine-Tuning
[45:30] Scaling Collaborative Learning Paradigms
[46:36] Recent advances - Cascade inference
[49:08] Advice for Young Researchers in AI
REFERENCES:
[00:27] Wanru Zhao - Google Scholar profile (https://scholar.google.com/citations?...)
[25:59] CLUES: Collaborative Private-domain High Quality Data Selection for LLMs via Training Dynamics (https://proceedings.neurips.cc/paper_...)
[38:07] Breaking Physical and Linguistic Borders: Multilingual Federated Prompt Tuning for Low-Resource Languages (https://openreview.net/pdf?id=HyRwexERAo)
[45:19] Include: Evaluating Multilingual Language Understanding With Regional Knowledge (https://arxiv.org/abs/2411.19799) , and Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation (https://arxiv.org/abs/2504.07072)
[46:42] CASCADIA: A Cascade Serving System for Large Language Models (https://www.arxiv.org/abs/2506.04203)
🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI.
WiAIR website:
♾️ https://women-in-ai-research.github.io
Follow us at:
♾️ LinkedIn: / women-in-ai-research
♾️ Bluesky: https://bsky.app/profile/wiair.bsky.s...
♾️ X (Twitter): https://x.com/WiAIR_podcast
#FederatedLearning #AIResearch #LLM #WomenInAI #ScalingLaws #DecentralizedAI #MultilingualAI #NeurIPS2024 #ICLR2024 #DataCentricAI #ModelMerging #WiAIR #WiAIRpodcast #TechForInclusion #RepresentationMatters #MachineLearning #NLP #WomenInSTEM #OpenScience #VectorInstitute #UniversityOfCambridge #AWS #MicrosoftResearch