Data Engineer Consultant - Spark/Python
- Category: IT Engineer & Developer Jobs
- Location: Mumbai, Maharashtra
- Job Type: Full Time / Part Time
- Salary: Estimated: $ 25K to 28K
- Published on: 2025/09/23
Job Description
Required Skills :
• Must have excellent knowledge in Apache Spark and Python programming experience
• Deep technical understanding of distributed computing and broader awareness of different Spark version
• Strong UNIX operating system concepts and shell scripting knowledge
• Hands-on experience using Spark & Python
• Deep experience in developing data processing tasks using PySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations.
• Externally certified in one of the cloud services (foundational or advanced)- (AWS, GCP, Azure, Snowflake, Databricks)
• Experience in deployment and operationalizing the code, knowledge of scheduling tools like Airflow, Control-M etc. is preferred
• Experience in creating visualizations in either of Tableau, PowerBI, Qlik, Looker or any of the other reporting tools
• Good knowledge of Hadoop, Hive and Cloudera/ Hortonworks Data Platform
• Should have exposure with Jenkins or equivalent CICD tool & Git repository
• Experience handling CDC operations for huge volume of data
• Should understand and have operating experience with Agile delivery model
• Should have experience in Spark related performance tuning
• Should be well versed with understanding of design documents like HLD, TDD etc
• Should be well versed with Data historical load and overall Framework concepts
• Should have participated in different kinds of testing like Unit Testing, System Testing, User Acceptance Testing, etc
Preferred Skills
• Exposure to PySpark, Cloudera/ Hortonworks, Hadoop and Hive.
• Exposure to AWS S3/EC2 and Apache Airflow
(ref:hirist.tech)
Related jobs
-
Data Engineering - Experienced Associate
At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enablin...
-
Product Data Manager
Interested candidates can share updated resumes to shraddha.ambekar@smarte.pro About SMARTe Inc. SMARTe Inc. is a cutting-edge organization focused on providing innovative data solutions to help businesses harness the power of data for enhanced decis...
-
Senior AI Engineer
PristineAI has pioneered integrated optimization solutions for retail pricing, promotions, personalization and assortment. Headquartered in Woburn, MA, with AI Engineering labs in Mumbai and Chennai, we are a results-driven team of innovators, subjec...