Model Monitoring: What, Why, and How
Speaker Bio:
Manasi Vartak, CEO
Verta Inc.
Abstract:
For any organization whose core product or business depends on ML models (think Slack search, Twitter feed ranking, or Tesla Autopilot), ensuring that production ML models are performing with high efficacy is crucial. In fact, according to the McKinsey report on model risk, defective models have led to revenue losses of hundreds of millions of dollars in the financial sector alone. However, in spite of the significant harms of defective models, tools to detect and remedy model performance issues for production ML models are missing.
In this talk, we discuss why model monitoring matters, what do we mean by model monitoring, and considerations when setting up a model monitoring system.