Data on cloud L3

Job Summary

Primary Role Function:
• Create and maintain optimal data pipeline architecture,
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• Experience with AWS cloud services: EC2, EMR, RDS, Redshift
• Experience with stream-processing systems: Storm, Spark-Streaming, etc.
• Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Keep our data separated and secure across national boundaries through multiple data centers and AWS regions.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.
• Writes high quality and well-documented code according to accepted standards based on user requirements

Knowledge:
• Thorough in-depth knowledge of design and analysis methodology and application development processes
• Exhibits solid knowledge of databases
• Programming experience with extensive business knowledge
• University degree in Computer Science, Engineering or equivalent industry experience
• Solid understanding of SDLC and QA requirements

Role Based Competencies

Tech - Requirement Gathering

Beh - Assertiveness

Tech - Estimation

Tech - High level design

Tech - Low level design

Tech - Domain

Tech - Code quality and coding standards

Tech - Team Management

Beh - Problem Solving

Tech - Unit Testing

Tech - Agile Methodology

Tech - Systems Design and Architecture

Beh - Communication

Beh - Customer Orientation

Beh - Collaboration

Beh - Innovation

Mandatory Competencies

Python - Python

Big Data - Scala

Big Data - PySpark

Big Data - Hadoop

Cloud - AWS

Data on Cloud - AWS S3

DevOps - CLOUD AWS

Good to Have Competencies