Python Interview Questions: PyJanitor, Lux, SQLMesh, ClearML & MLEM! 🚀
1️⃣ PyJanitor for Data Cleaning
PyJanitor extends pandas with convenient methods for common cleaning tasks, such as removing empty rows, converting column names to snake_case, and chaining cleaning steps .
Example:
import pandas as pd
import janitor
df = pd.read_csv("raw.csv")
df = (df
.clean_names() # snake_case column names
.drop_empty() # drop rows with all nulls
.remove_columns(['unwanted'])) # remove unnecessary cols
2️⃣ Lux for Automated Visualization
Lux integrates with pandas to automatically suggest visualizations based on your DataFrame’s structure and variable types, launching an interactive widget in Jupyter .
Example:
import pandas as pd
import lux
df = pd.read_csv("sales.csv")
df # In a Jupyter notebook, Lux prompts charts for you to explore
3️⃣ SQLMesh for Versioned SQL Transformations
SQLMesh provides a framework to author modular SQL models, version changes, apply unit tests, and orchestrate dependencies, treating SQL like code .
Example structure:
-- models/staging_orders.sql
SELECT * FROM raw.orders WHERE order_date v= '{{ start_date }}';
sqlmesh run --model staging_orders --start 2025-01-01
sqlmesh test # Run defined SQL tests
4️⃣ ClearML for Experiment Management
ClearML offers an open-source suite for experiment tracking, data management, and pipeline orchestration, with minimal code changes via decorators or context managers .
Example:
from clearml import Task
task = Task.init(project_name="iris", task_name="train")
Your training code follows; metrics and artifacts get logged automatically
5️⃣ MLEM for Model Packaging and Deployment
MLEM is a lightweight tool to package, version, and serve ML models, generating Docker images and REST APIs from a mlem.yml spec .
Example:
mlem install model.pkl --alias best_model
mlem serve best_model --port 5000