Professional Experience
Data Engineer, Management & Governance Analyst
Accenture · Bengaluru, India
Feb 2024 – Present (Associate → Analyst, effective March 2026)
- Built and validated 90+ production PySpark ETL jobs across 4 supply-chain sub-domains, contributing to 50+ data products through Python, JSON configuration, and DDL.
- Led 12 change requests across data products and completed tag validations in a single day by comparing design documents, source messages, and data across staging, active, and history layers.
- Part of a 7-member E2E supply-chain data engineering team (1 lead, 1 tester, 5 DEs) covering site, supplier, item, and product sub-domains; primarily owned site and supplier pipelines while supporting item and product live monitoring.
- Developed configuration-driven pipeline definitions and reusable Python utilities, cutting deployment cycle time by 40%.
- Achieved 99.5% pipeline uptime through modular PySpark pipelines with error handling, retry logic, schema validation, and data quality checks.
- Maintained 95%+ test coverage across 60+ pytest suites with mocked components and integration patterns.
- Improved data processing latency by 30% via partitioning, caching, and Delta Lake query tuning.
- Owned end-to-end data quality and platform validation for 3–4 sprint releases across source-to-target data layers.
- Used Jira dashboards to track sprint tasks, incidents, and maintenance activities for data products across the supply-chain platform.
- Developed data products from design documents and built downstream datasets from source schemas, applying transformation queries when multiple source data products feed a single dataset; maintained per-schema exception tables to capture invalid records with target table reference, error log, and timestamp.
Selected Projects
Cloud-Native Streaming Data Platform
- Designed modular Terraform templates for multi-environment infrastructure and a PySpark Structured Streaming job with watermark-based dedup and exactly-once Delta Lake writes.
- Automated Terraform plan/apply pipelines through GitHub Actions for dev and prod environments.
- Validated streaming output with schema checks and row-level data quality checks.
Kafka → PySpark → Delta Pipeline
- Ingested JSON events from Kafka, enforced schemas, and wrote exactly-once to Delta Lake using checkpointing and idempotent writes.
- Benchmarked 31k–45k rows/sec on a 4-core laptop for 100k–1M row workloads.
- Maintained 95%+ pytest coverage with an in-memory Spark fixture and continuous integration.
RAG Document QA Chatbot
- Built dense vector retrieval with ChromaDB, FastAPI backend, response caching, and modular LLM interfaces (OpenAI/LLaMA).