ETL Data Engineer - AWS & Spark
- Category: IT Engineer & Developer Jobs
- Location: Sydney, New South Wales
- Job Type: Full Time / Part Time
- Salary: Estimated: $ 20K to 29K
- Published on: 2025/09/26
Job title: ETL Data Engineer - AWS & Spark
Location: Sydney
Role: Contract
Position Overview: We are seeking highly skilled ETL Data Engineer with experience in ETL development and a solid understanding of the AWS ecosystem, particularly EMR, S3, Athena, and SQL. The ideal candidate will have expertise in Spark and Scala, ETL lifecycle management, and job scheduling using Autosys. Experience with framework builds and ETL job migration to AWS environments is highly desirable.
Key Responsibilities:
Execute ETL jobs and processes from Day 1 to ensure smooth data operations.
Reconcile ETL jobs for a minimum of three runs, ensuring data accuracy and consistency.
Triage, debug, and fix issues with ETL jobs while reconciling data discrepancies.
Collaborate with multiple capability teams to debug and resolve job-related challenges effectively.
Conduct knowledge transfer (KT) sessions to ensure offshore teams are equipped to manage and support ETL processes.
Preferred Qualifications and Skills:
5-12 years of strong ETL development experience using Spark and Scala.
Hands-on experience with AWS services, including EMR, S3, Athena, and SQL.
Proficiency in Autosys for job scheduling.
Familiarity with building reusable ETL frameworks.
Prior experience in migrating ETL jobs to AWS environments.
Comprehensive understanding of the ETL lifecycle, including running, debugging, testing, reconciling, and deploying jobs in an AWS environment.
Strong problem-solving skills and ability to work effectively with cross-functional teams.
Proactively run and execute jobs to ensure operational continuity.
Perform reconciliation of jobs for at least three runs to validate data integrity.
Debug, troubleshoot, and resolve issues related to ETL job failures.
Engage and collaborate with multiple capability teams to address challenges and implement fixes.
Provide thorough onsite knowledge transfer to offshore teams for smooth transition and support.
Desired Candidate Profile:
Proactive and self-driven with a strong technical foundation in AWS and ETL processes.
Excellent communication and collaboration skills to work with diverse teams.
Proven ability to work under pressure and deliver high-quality solutions in a dynamic environment
Related jobs
-
Senior Front End Engineer, Lending
Constantinople is the first of its kind: an all-in-one software and operational platform for banks. We provide our banking clients with a cloud-native infrastructure to host and manage their customers. We offer a comprehensive suite of banking produc...
-
CLOUD SOLUTION ARCHITECT- Retails
With over 17,000 employees worldwide, the mission of the Customer Experience & Success (CE&S) organization is to empower customers to accelerate business value through differentiated customer experiences that leverage Microsoft’s products and...
-
Staff Data Scientist
Job Description Join the team redefining how the world experiences design. Hey, g'day, mabuhay, kia ora, 你好, hallo, vítejte! Thanks for stopping by. We know job hunting can be time-consuming, so let’s get straight to the point. Where And How You Can ...