Recmatrix Consulting

Job Title : Data Architect

Location : Bengaluru or Remote

Role : Contract

Duration : 6 months

Notice Period : Immediate only

Experience : 10 15 years

Role Overview :

As an Azure Data Architect, you will be responsible for developing, designing, and automating large-scale, high-performance data processing systems on Azure that drive business growth and enhance product experiences.

You will lead data engineering projects, contribute to data infrastructure, and collaborate with cross-functional teams to ensure reliable and efficient data pipelines.

Key Responsibilities :
• Develop, design, and automate large-scale batch and streaming data processing systems on Azure.
• Evangelize high-quality software engineering practices for building scalable data infrastructure and pipelines.
• Lead data engineering projects to ensure pipelines are reliable, efficient, testable, and maintainable.
• Design optimal data models for storage and retrieval to meet critical business and product requirements.
• Influence and architect logging best practices to support data flow and enhance infrastructure.
• Contribute to shared data engineering tooling and standards to improve productivity across the company.
• Collaborate with leadership, engineers, program managers, and data scientists to understand and address data needs.
• Educate stakeholders on data engineering practices, identifying and addressing gaps in logging and processes.
• Build and maintain data lineage, governance, and cataloging systems.
• Lead projects using agile methodologies and ensure effective communication at all levels.
• Recruit, retain, and develop team members for future responsibilities and challenges.

Experience & Skills Required :
• 10+ years of relevant industry experience in data architecture and engineering.
• Expertise in custom ETL design, implementation, and maintenance.
• Proven experience in designing data models for reporting systems.
• Hands-on experience with Azure Databricks, including SQL workloads.
• Proficiency in data ingestion techniques using file formats like CSV, TXT, and JSON.
• Experience setting up observability for data pipelines and Databricks in Azure.
• Strong knowledge of relational databases and SQL query authoring.
• Familiarity with Java, Scala, Spark, PySpark, Python, and Databricks technologies.
• Cloud experience, with a preference for Azure.
• Proven ability to work with large-scale data and design robust, high-performance data pipelines.
• Experience with CI / CD for ETL processes and deploying data pipelines with reliable monitoring and logging practices.
• Ability to work across team boundaries and provide overarching data architecture guidance.
• Expertise in minimizing data transfer and storage costs and optimizing ETL pipelines.
• Experience with Databricks SQL unified governance model (using Unity Catalog) across clouds, open formats, and APIs.
Technical Stack :
• Azure
• Azure
• Databricks SQL Serverless
• Databricks workspaces and notebooks
• Delta Lake Architecture
• Lakehouse Concepts
• Spark Structured / Streaming
• File Formats (CSV, Avro, Parquet)
• CI / CD for ETL
• Databricks SQL unified governance model (using Unity Catalog)