Software Engineer II | Data Practice

We're looking for a highly skilled software engineer(s) to join our application practice and technology team. Our ideal candidate will have expert knowledge of software development processes and solid experience testing and deploying applications in the cloud. If writing high-quality, elegant, beautiful, and meticulous code is among the talents that make you tick, we'd like to hear from you.

Responsibilities
• Design data storage systems, collect, process, and transform data for storage.
• Involvement in data collection decisions, applying best practices in managing data, make choices based on the type of data for appropriate storage, processing, and retrieval.
• Ensure infrastructure availability and optimization.
• Build/Design or Create Data Warehouse or Data Lakes, manage heterogeneous data across various sources, analyze data, uncover insights and enable the use of data for further processing, identify missing data, and take suitable action.
• Classify, clean and process data for use in downstream processes and applications.
• Ensure data quality.
• Perform tests and troubleshoot Data Warehouses or Data Lakes, and take action to continuously improve data collection, data processing, data transformation, and data storage required for business.
• Perform exploratory data analysis as required.
• Create data visualizations for the defined audience, enable sense-making and augment the use of data for business decision-making, and identify data monetization opportunities.
• Contribute to the development and engineering of existing products, and deliver mission-critical software across multiple industry verticals.
• Work with partners, collaborators, and cross-functional, distributed teams in an agile setting.
• Champion innovation for our products and customers through participation in ideation, prototyping, and validating product fit.
• Contribute to Data Architecting applications and solutions, involve in data architecture decisions, embrace and implement data quality, and other best practices, and contribute to multiple areas under data practice including supporting/mentoring junior resources and championing Limendo culture.
• Design, develop and modify data systems, using scientific analysis and mathematical models to predict and measure outcomes and consequences of design
• Develop and direct data system testing and validation procedures, programming, and documentation, analyze information to determine, recommend, and plan data system reports and layouts, and modifications.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS big data technologies.
• Writing Complex SQL queries, sub-queries, and complex joins.
• Azure/AWS Pipeline knowledge to develop ETL for data movement, experience to map the source to target rules and fields (Redshift or Snowflake) Implement natural keys, surrogate keys, SCD, and SCD2 into the data warehouse to support history, automated triggers to take snapshots, history maintenance, automated triggers to take backup and restore in case of failures.
• Translate business requirements, design jobs for data migration from various data sources, RDBMS design, development, and performance tuning, experience with MS SQL Server 2014 database technologies and ETL best practices with MS SQL Server Using Business Intelligence Development Studio to build SSIS packages.
• Excellent troubleshooting & optimization skills: interpreting ETL logs, performing data validation, dissecting SSIS code, understanding the benefits and drawbacks of parallelism, experience with change data capture, using expressions, scoping of variables, commonly used transforms, event handlers, and logging providers, ability to understand and optimize the surrogate key generation and inconsistent data type handling.

Requirements
• Bachelor's degree in Computer / Software engineering or Information Technology.
• 4 - + years of experience as a Software Engineer with ETL/DWH, Databases, and Storage Technologies. (Azure Data Pipeline, Kafka Apache Data Stream).
• Experience with rapid development cycles in an agile setting.
• Strong scripting and test automation abilities.
• Experience in cloud settings is preferred, AWS, Azure or GCP Server and Databases: MSQL, MySQL, PL/SQL, Oracle, NodeJS Server, PostgreSQL, MongoDB, Firebird Business Intelligence Solutions: SSIS, SSAS, SSRS, MDX, OLAP, experience in Hadoop technologies, Data Modeling.
• Tools: MS Visio, Erwin, Java, and/or. NET experience.
• Programming Language/another framework: Python, SQL, PySpark, Libraries: Pandas, sci-kit.
• Data visualization: MS Power BI or any other