[93] Model Risk Management Best Practices for Data Science (Alejandro Gomez)
Join our Meetup group:
https://www.meetup.com/data-umbrella
Resources
Slides and Colab: https://sites.google.com/adao.tech/po...
About the Event
This is a presentation that aims to explain some of model risk management best practices to data science (split of models between input, methodology and output & model validation). The following Python libraries will be covered:
ydata-profiling
pycaret
altair
ydata-profiling:
is a leading package for data profiling, that automates and standardizes the generation of detailed reports, complete with statistics and visualizations.(https://docs.profiling.ydata.ai/4.6/)
pycaret:
PyCaret is an open-source, low-code machine learning library in Python that automates machine learning workflows. It is an end-to-end machine learning and model management tool that exponentially speeds up the experiment cycle and makes you more productive. PyCaret is essentially a Python wrapper around several machine learning libraries and frameworks, such as scikit-learn, XGBoost, LightGBM, CatBoost, spaCy, Optuna, Hyperopt, Ray, and a few more.(https://pycaret.org/)
altair:
Vega-Altair is a declarative visualization library for Python. Its simple, friendly and consistent API, built on top of the powerful Vega-Lite grammar, empowers you to spend less time writing code and more time exploring your data.(https://altair-viz.github.io/)
Timestamps
00:00 Data Umbrella introduction
04:22 Alejandro begins talk
05:53 Talk agenda / outline
07:07 Part 1: model risk management for data science
08:24 Model risk management in banking, regulations
10:44 -- Input, Processing & Output; Stages: Initiation, Model life cycle
13:09 -- Lines of Defense: 1 (model developers), 2 (model validation), 3 (audit)
13:50 Model Risk for: Data Science, AI / ML; AI regulations, guidance (EU, NIST, UK, Indep)
15:50 NIST: AI Risk Model Framework (RMF); TEVV (test, evaluation, verification, validation)
17:52 Part 2: Story of Tyke and python libraries
20:28 -- python libraries: ydata-profiling
22:35 demo: Jupyter notebook
27:27 --python libraries: pycaret (how the model was created)
29:45 --model comparisons by key metrics
31:30 --SHAP analysis
32:37 --python libraries: altair (graphing)
36:14 Complex altair chart
38:55 -- stack multiple charts
40:39 Part 3: Conclusions
43:54 Q: How does ydata-profiling compare with dabl or lux?
44:15 Q: Is ydata-profiling a good replacement for pandas profiling?
45:03 Q: Are there more opportunities for model risk management as a career?
46:26 Q: For mathematicians / statisticians, should they use Python over R?
49:15 Q: Can you share about your start-up, Adao?
50:30 Q: Is pycaret used in developing cognitive models?
About the Speaker
Alejandro Gomez is a mathematician, who has worked in the model risk management space for over 10 years. He is the founder of ADAO, a start up bringing technology to SMEs.
GitHub: https://github.com/agomezh
X: / a_gomez_h
LinkedIn: / agomezh
#opensource #modelrisk