Hi, I'm Sudhir 👋
Senior Data Engineer | Dubai, UAE | AWS & Azure | Apache Spark
Result-oriented Senior Data Engineer with 10+ years of experience building optimized data engineering pipelines for ETL processes. Expert in extracting, transforming, and loading data into various storage systems to provide actionable insights for customer acquisition, operational efficiency, and sentiment analysis. Proven track record in big data processing, cloud platforms, and cross-functional team collaboration.
Technical Expertise
Languages & Processing
- • Python, Java, JavaScript
- • PySpark, Spark Streaming
- • SQL, PL/SQL
- • Apache Nifi
Cloud & Data Platforms
- • AWS (Lambda, ECS, S3, Glue)
- • Azure
- • Snowflake
- • Kafka Streaming
Databases & Storage
- • Oracle, PostgreSQL
- • SQL Server, HBase
- • Parquet, AVRO, CSV
- • Data Lakes & Warehouses
DevOps & Tools
- • Docker, Kubernetes
- • Terraform
- • Apache Airflow, Control-M
- • Git, GitLab, Bitbucket
Experience
Senior Data Engineer (Consultant)
Emirates NBD, Dubai | Apr 2025 - Present
Implemented Medallion architecture and real-time streaming pipelines, optimizing micro-batch processing by 58%. Built reusable ETL frameworks with focus on performance optimization
PySpark Batch, PySpark Streaming, Kafka, Performance Optimization, Oracle, Reusable ETL Frameworks, Tableau
Senior Data Engineer
JPMorgan Chase & Co., Mumbai | Oct 2021 - Apr 2025
Designed event-driven pipelines on AWS, migrated data to Snowflake, and achieved 6x performance improvement through Spark optimization
AWS (Glue, Lambda, ECS, S3, CloudWatch, Route53, NLB, ALB, Service Mesh, Multi-region), Snowflake, Spark, Python, Java, Kafka, Terraform, Kubernetes, Docker, Spring Boot
Data Engineer & ETL Automation Developer
BNP Paribas, Mumbai | Mar 2017 - Oct 2021
Built ETL frameworks and automation tools reducing development time by 60-90%, created data lineage and quality frameworks
PySpark, Python, Hive, Kafka, HBase, Nifi, Apache Airflow, Oracle, SQL Server
ETL Developer
Capgemini, Mumbai | Aug 2014 - Feb 2017
Designed and developed complex end-to-end ETL pipelines, automated production support processes
ETL Tools, SQL, Oracle, Automation Frameworks
Let's Connect
Interested in collaborating or discussing data engineering?