Data Engineer - Azure

• Overall, 7 to 9 years of experience in cloud data and analytics platforms such as AWS, Azure, or GCP

• Including 3+ years’ experience with Azure cloud Analytical tools is a must

• Including 5+ years of experience working with data & analytics concepts such as SQL, ETL, ELT, reporting and report building, data visualization, data lineage, data importing & exporting, and data warehousing

• Including 3+ years of experience working with general IT concepts such as integrations, encryption, authentication & authorization, batch processing, real-time processing, CI/CD, automation

•Advanced knowledge of cloud technologies and services, specifically around Azure Data Analytics tools

• Strong coding skills in languages such as

• Experience in data streaming technologies such as Kafka or Azure Event Hubs

• Experience in handling unstructured streaming data is highly desired

• Knowledge of Business Intelligence Dimensional Modelling, Star Schemas, Slowly Changing Dimensions

• Broad understanding of data engineering methodologies and tools, including Data warehousing, DevOps/DataOps, Data ingestion, ELT/ETL and Data visualization tools

• Knowledge of database management systems, data modelling, and data warehousing best practices

• Experience in software development on a team using Agile methodology

• Knowledge of data governance and security practices

• Develop and implement strategies for processing and analysing large volumes of IoT data, ensuring scalability and performance.

• Design, develop, and maintain streaming and batch data pipelines in Azure cloud environment

• Participate in the planning and prioritization of development tasks, ensuring efficient use of resources and timely delivery of projects

• Collaborate with data analysts and data scientists to understand their data requirements and develop solutions to meet their needs.

• Optimize and tune data pipelines for performance and scalability.

• Develop and maintain monitoring and alerting solutions to ensure data quality and availability.

• Collaborate with our Platform team to ensure data analytics infrastructure is secure, reliable, and highly available.

• Continuously research and evaluate new tools and technologies to improve data processing and analytics capabilities.

• Conduct code reviews to ensure adherence to coding standards and best practices.

• Capture and document data lineage, metadata, and data dictionaries to ensure comprehensive data governance.