Y&L Consulting

Job description
Skills

Azure Data Factory, Azure Data Lake, Azure SQLData Warehouse, Azure Databricks, Azure Synapse, Spark, Python, Azure AnalysisServices, Azure DevOps

Experience working in ETL process for at least 8 years

Experience developing ETL process using AzureETL tools for at least 5 years

Qualifications

Bachelor's degree in information systems,Computer Science, Engineering, or related field

Proficient in designing and implementing datasourcing and transformation processes (ETL) in a large, distributed environmentusing cloud services e.g., Azure Data Factory,Azure Data Lake, Azure SQL Data Warehouse, Azure Databricks, Azure Synapse,Spark, Python, Azure Analysis Services and Azure DevOps

Hands-on experience with developing complex ETLpipelines that ingest and transform data from various source systems into Azurecloud operational databases and data warehouse environment

Proficient in developing and operationalizingvarious data distribution patterns such as APIs, event based, publish/subscribemodels

Well versed with data models, source targetdefinitions, identify/fixing discrepancies adhering to enterprise datagovernance standards

Adept with Azure DevOps-CI/CD (ContinuousIntegration and Continuous Deployment) and Test-Driven Development practices

Design, develop and implement scalable batch /real-time data pipelines to integrate data from a variety of sources into anAzure database / data warehouse and data lake

Experience migrating data lakes from on-premisesto cloud

Experience with agile delivery using AzureDevOps

Experience monitoring performance of dataworkflows and infrastructure through use of monitoring tools and enablingalerts

Functional knowledge of data visualization tools

Responsibilities

Design and implement data frameworks andpipelines to process data from on-premises and cloud data sources to feed intothe Azure Cloud, monitor data quality and implement related controls

Evaluate existing designs, improve methods, andimplement optimizations

Analyze and validate data sharing requirementswithin and outside data partners

Analyze and interpret collected data;identifying trends; writing reports and recommendations for internal orexternal clients

Use statistical practices to analyze current andhistorical data to make predictions, identify risks, and opportunities enablingbetter decisions on planned/future events

Work with business leadership to understand datarequirements; propose and develop solutions that enable effectivedecision-making and drive business objectives

Prepare project deliverables that are valued bythe business and present them in such a manner that they are readily understoodby project stakeholders

Prepare advanced project implementation plansusing Agile methodologies, which highlight milestones and deliverables,leveraging standards, practices, and work planning tools

Recognize potential issues and risks during theanalytics project implementation and recommend mitigation strategies

Document best practices for data analysis, dataengineering, and evangelize their usages

Work with stakeholders, including the executivesas well as product engineering teams to assist with data-related technicalissues and support their data infrastructure needs

Create data tools for data scientist team toassist them in building and optimizing our product into an innovative industryleader

Work with data and analytics experts to strivefor greater functionality in our data capabilities

Experience inoffshore/ onsite coordination