Data Engineer

Job description
The Data Engineer will work on a project that focuses on integrating and processing channel activity data from multiple sources, including Terradata, Hadoop environments, and AWS cloud infrastructure.

Key Responsibilities:

Design, build, and maintain robust data pipelines for processing channel activity data, focusing on digital channels and integrating multiple sources of data such as Hadoop and Terradata.
Work on AWS cloud infrastructure (including services like Pyspark, Glue, EMR, lambda, Redshift, Athena, and other services like Iceberg) and orchestration tools like Airflow to ensure high availability, scalability, and performance.
Leverage Python for building automated data engineering solutions and data transformations.
Collaborate with cross-functional teams to ensure data from various digital channels is effectively ingested and processed.
Contribute to architecture decisions related to data lakes, data warehouses, and big data ecosystems.
Ensure data quality, governance, and integrity across all data systems.
Engage in continuous learning and adapting of new technologies and industry best practices to improve data processes.