Panel: Data Discovery in Data Mesh; a Data Mesh Learning and Open||Source||Data event
Panel of experts discussing data discovery, especially re data mesh.
Moderator: Paco Nathan (Derwen.ai); Panelists: Shinji Kim (Select Star), Sophie Watson (Red Hat), Mark Grover (Stemma), and Shirshanka Das (Acryl Data)
Co-hosted with Data Mesh Learning and the Open||Source||Data Podcast
The mad scientist himself, Paco Nathan (bio), moderated a panel discussion on data discovery. Data discovery is one of the hardest problems to solve in data management in general and comes up in many discussions as a major pain point in most data mesh discussions.
Our all-star expert panel:
Mark Grover: co-creator of the metadata management solution (Lyft) Amundsen; CEO/Founder of Stemma
Shinji Kim: CEO/Founder of data discovery platform provider Select Star
Shirshanka Das: co-creator of the metadata management solution LinkedIn DataHub and Apache Gobblin; CEO/Co-Founder of Acryl Data (DataHub-as-a-Service)
Sophie Watson: Principal Data Scientist at Red Hat and general bad*ss at large
Community resources:
Meetup Group: https://www.meetup.com/data-mesh-lear...
Community Slack Signup: https://launchpass.com/data-mesh-lear...
Newsletter/Blog: https://datameshlearning.substack.com/
DML Website: https://datameshlearning.com/
LinkedIn: / data-mesh-learning
Free data mesh journey check-ins: / urn:li:activity:6810257742197063680
Open||Source||Data Podcast: https://www.datastax.com/resources/po...
Bigger bios:
Paco Nathan: Known as a "player/coach", with core expertise in data science, natural language, cloud computing; ~40 years tech industry experience, ranging from Bell Labs to early-stage start-ups. Advisor for Amplify Partners, Recognai, KUNGFU.AI, Primer. Lead committer PyTextRank, kglab. Formerly: Director, Community Evangelism @ Databricks and Apache Spark. Cited in 2015 as one of the Top 30 People in Big Data and Analytics by Innovation Enterprise.
Mark Grover is the co-creator of the open source data catalog and metadata engine, Amundsen and the CEO of Stemma. Amundsen is used by data scientists and analysts to discover, understand and trust the data they use. At Lyft, Amundsen has 750+ active users every week, and outside of Lyft, Amundsen is used by 35+ companies like ING, Square, Brex, Instacart and more.
Sophie Watson is a data scientist at Red Hat, where she helps customers to solve business problems using machine learning in the hybrid cloud. She has previously conducted research in the areas of Bayesian Statistics and Recommendation Engines, and is focused on using her data science and statistics skills to inform next-generation infrastructure for intelligent application development.
Shinji Kim is the Founder & CEO of Select Star, intelligent data discovery platform that helps you understand your data. Previously, she was the CEO of Concord Systems, a NYC-based data infrastructure startup acquired by Akamai Technologies in 2016. She led building Akamai’s new IoT data platform for real-time messaging, log processing, and edge computing. Prior to Concord, Shinji was the first Product Manager hired at Yieldmo, where she led the Ad Format Lab, A/B testing, and yield optimization. Before Yieldmo, she was analyzing data and building enterprise applications at Deloitte Consulting, Facebook, Sun Microsystems, and Barclays Capital. Shinji studied Software Engineering at University of Waterloo and General Management at Stanford GSB. She advises early stage startups on product strategy, customer development, and company building.
Shirshanka Das is co-founder and CEO of Acryl Data, the company which is commercializing the open source DataHub project. Prior to founding Acryl, he was the overall architect for Big Data at LinkedIn from 2010 to 2020, and responsible for creating the metadata and data management strategy at the company. As part of this, he founded the DataHub project and shaped its evolution to a metadata platform that powers DataOps, MLOps, productivity and governance use cases at LinkedIn. He is also a PMC and committer on the Apache Gobblin project which manages 100PB+ of data assets at rest at LinkedIn, and is deployed in production at other large companies like Verizon, PayPal etc. Prior to LinkedIn, Shirshanka worked on high performance serving systems at Yahoo and PayPal. Shirshanka has a Ph.D. in Computer Science from UCLA.