6 + YoE - Data Engineer – Big Data / PySpark - any UST Location - Immediate Joiner

Candidates ready to join immediately can share their details via email for quick processing.

CCTC | ECTC | Notice Period | Location Preference

Act fast for immediate attention! ⏳

Must-Have Skills

• 6+ years of overall experience in Data Engineering / Big Data.
• Strong understanding of Big Data concepts and architecture .
• Strong hands-on experience with Apache Spark .
• Expertise in:
• Spark Performance Tuning
• Spark Optimization
• Query/Job Performance Improvement
• Troubleshooting Spark workloads
• Strong hands-on experience with PySpark and Spark .
• Strong programming experience in Python .
• Good experience working with MySQL / SQL .
• Strong experience in designing and developing Data Pipelines .
• Hands-on experience with Apache Airflow for data pipeline orchestration and scheduling.
• Experience working with at least one Cloud Platform .
• GCP experience is preferred.
• Good understanding of CI/CD and DevOps concepts .
• Experience integrating data engineering workloads with CI/CD pipelines.
• Strong debugging, troubleshooting, and problem-solving skills.

Preferred Skills

• Hands-on exposure to relevant GCP data services .
• Experience handling large-scale and high-volume datasets.
• Understanding of distributed data processing and data architecture.
• Experience improving the scalability, reliability, and performance of data pipelines.
• Exposure to Agile development and DevOps practices.

Key Responsibilities

• Design, develop, and maintain scalable Big Data and Data Engineering solutions .
• Develop data processing applications using Python, PySpark, and Apache Spark .
• Perform Spark performance tuning and optimization for large-scale workloads.
• Build, maintain, and monitor robust ETL/ELT data pipelines .
• Develop and manage workflow orchestration using Apache Airflow .
• Work with MySQL/SQL for data extraction, transformation, and validation.
• Deploy and support data engineering solutions in cloud environments, preferably GCP .
• Work with DevOps teams to implement and maintain CI/CD pipelines .
• Troubleshoot production issues and optimize data processing performance.
• Collaborate with engineering and business teams to deliver reliable and scalable data solutions.

Primary Skill Combination

Big Data + Apache Spark + PySpark + Python + Airflow + SQL/MySQL + Cloud (GCP Preferred) + CI/CD

Mandatory Focus: Strong hands-on Apache Spark performance tuning and optimization experience.