Open-source ML tooling · Case study
pandas-eda-check
An installable Python package for repeatable DataFrame profiling and reference-versus-current comparisons across schema, completeness, distributions, dates, and categories.
- Context
- Open-source data-quality tooling
- Role
- Package author and maintainer
- Input
- Reference and current DataFrames
- Distribution
- Installable from PyPI
- Status
- Public package with tests and CI
The problem
Small data changes can quietly become large model and reporting failures
Schema shifts, missing values, duplicate rows, altered distributions, and new categories can pass unnoticed between exploratory analysis and production use. Teams need the same checks to be repeatable in notebooks, tests, and automated workflows.
The system
Profile once, compare consistently, inspect the result
The package converts common exploratory and comparison checks into structured, reusable operations that work with familiar pandas DataFrames.
What I built
Data-quality checks packaged for reuse
Created a concise Python API for profiling a DataFrame and comparing current data with a reference dataset.
Covered schema, missingness, duplicates, numeric distributions, date ranges, and categorical values.
Returned structured results that can be inspected by people or incorporated into automated checks.
Added test coverage for expected behavior and edge cases across supported comparison types.
Prepared the project as an installable package with modern metadata and continuous-integration checks.
Published documentation and examples alongside the source and PyPI distribution.
Quality and evaluation
The package itself is tested like production tooling
Automated tests exercise the public API, comparison behavior, and important edge cases. Packaging checks verify that the same project can be installed and used outside its source directory.
Thresholds and statistical outputs are evidence for review, not universal pass/fail rules. Users remain responsible for selecting checks appropriate to their dataset and application.
Technology
Evidence