Designing Reliable CDC Pipelines on AWS
A practical architecture for handling full loads, updates, deletes, late-arriving changes, and replay without turning the pipeline into a collection of special cases.
Data Engineering · Cloud Systems · Open Source · Transportation Safety
Janardhan Reddy Kasireddy
I design and write about reliable data systems—distributed processing, change data capture, data quality, cloud architecture, open-source technologies, and transportation safety analytics.
Writing
All articles →Open source
Details →OPEN PR
Apache PyIceberg
Nested field statistics and inferred metrics limits in the Python implementation of Apache Iceberg.
OPEN PR
dlt verified sources
Regression fix for optional defaults in the PostgreSQL replication source.
About
I am a data engineer focused on reliable cloud data systems, distributed processing, and trustworthy analytics. I use this site to publish technical essays, architecture lessons, open-source work, and ideas at the intersection of data engineering, transportation, and public safety.