Open-Source Spotlight - Pandera - Niels Bantilan
Pandera: Typing for dataframes
00:00 Introduction
00:48 Pandera origin story
02:12 Statistical typing
03:20 Data types for DataFrames
05:00 Who is Pandera for?
06:18 Defining DataFrame schemas
07:00 Specifying data types
07:16 Specifying checks
09:20 Pydantic-style schemas
11:45 Validating DataFrames
13:40 Lazy validation
15:47 Type annotations and type checking decorators
16:55 SchemaModel strictness and transformations
20:24 Column aliasing and regex batching
21:25 Scaling data validation with Dask, Fugue, Koalas, and Modin
22:29 Integration goals
23:00 Core contributors
23:50 How to contribute
26:37 Advice: use typing!
Links:
https://pandera.readthedocs.io/en/sta...
https://github.com/pandera-dev/pandera
Scipy Maintainer's Track [2021]: • Maintainers Track Talks | SciPy 2021
Pycon [2021]: • TALK / Niels Bantilan / Statistical T...
Scipy [2020] - Statistical Data Validation of Pandas Dataframes: • Pandera: Statistical Data Validation ... , paper: https://conference.scipy.org/proceedi...
Pandera Blog [2020]: https://blog.pandera.ci/statistical%2...
PyOpenSci Blog [2019]: https://www.pyopensci.org/blog/pander...
Join DataTalks.Club: https://datatalks.club/slack.html
Our events: https://datatalks.club/events.html