Database Validation Tool
Data-validation tool from a real AWS Athena → Alibaba MaxCompute migration — detects missing IDs and value discrepancies across sources.
Architecture
Cross-database validation: fetches both sides in ID chunks, detects missing IDs and value discrepancies, and writes detailed reports.
Databases
Built with
Overview
A data-validation tool built during a real production migration from AWS Athena to Alibaba MaxCompute. It verifies that data landed correctly after migration.
It detects missing IDs and value discrepancies across sources, giving confidence that the migrated data matches the original. Beyond the migration use case, it also supports Oracle and PostgreSQL.
Highlights
- Built for a real AWS Athena → Alibaba MaxCompute migration
- Detects missing IDs and value-level discrepancies across sources
- Extensible to Oracle and PostgreSQL
Role
Data Engineer — built the validation tooling
Context
Real production cloud migration (AWS ↔ Alibaba)
Tech stack
More projects
View all →SMILE Platform (UNDP for Kemenkes)
National immunization logistics system in production across all of Indonesia — streaming CDC + batch analytics platform.
LidValid — Unified Data Validation Platform
Full-stack data validation platform with Tiered Validation — cheap aggregate checks across every table, then precise row-level checks only where they fail.
Dataklin — AI-Powered Data Quality & Entity Resolution
Data quality and entity-resolution platform that profiles, validates, deduplicates, and standardizes datasets before handoff — with LLM-generated rules.