KSQLDB Streaming Pipeline
Streaming ETL with Kafka + KSQLDB and a custom Kafka Connect sink — Postgres to Postgres.
Architecture
SQL-first streaming ETL: CDC from Postgres into Kafka, transformed in KSQLDB, written back via a custom sink connector.
Sink
Infrastructure
Overview
A streaming ETL pipeline that performs transformations directly in the stream using KSQLDB on top of Kafka.
It reads from PostgreSQL and writes back to PostgreSQL through a custom Kafka Connect sink connector — a SQL-first approach to stream processing.
Highlights
- Stream transformations in KSQLDB (SQL on streams)
- Custom Kafka Connect sink connector
- Postgres → Postgres end to end
Tech stack
More projects
View all →SMILE Platform (UNDP for Kemenkes)
National immunization logistics system in production across all of Indonesia — streaming CDC + batch analytics platform.
LidValid — Unified Data Validation Platform
Full-stack data validation platform with Tiered Validation — cheap aggregate checks across every table, then precise row-level checks only where they fail.
Dataklin — AI-Powered Data Quality & Entity Resolution
Data quality and entity-resolution platform that profiles, validates, deduplicates, and standardizes datasets before handoff — with LLM-generated rules.