Azure Data Engineer - London, United Kingdom - Venturi

Description

Role - Azure Data EngineerRate - £500 Per Day Outside IR35Remote work with an openness to travel to London if requiredDuration - 6 monthsMy client is seeking experienced Azure Data Engineers for a stakeholder-facing project.

This role will have particular emphasis on Azure, Databricks and DevOps.

Our client is in the consultancy sector but does not have a preference for previous experience within a consultancy role.

There is also a key element of delivery within the project, focusing on delivering technically but also ensuring the quality of delivery within a wider team of Data Engineers.

The ideal candidate needs experience and confidence to speak up when several teams can be involved in delivering the project.

There is a massive emphasis on Pyspark and Databricks for this particular role.


Technical Skills Required:
Azure (ADF, Functions, Blob Storage, Data Lake Storage, Logic Apps, DevOps)

DatabricksSparkDelta LakeSQLPythonPySparkADLSSoft Skills Required:
Stakeholder ManagementExperience in a Senior Data Engineering roleProject Delivery ExperienceFace-to-face communication skills

Day To Day Responsibilities:

Extensive experience in designing, developing, and managing end-to-end data pipelines, ETL (Extract, Transform, Load), and ELT (Extract, Load, Transform) solutions.

Maintains a proactive approach to staying updated with emerging technologies and a strong desire to continuously learn and adapt to new tools, languages and methodologies in the ever-evolving field of data engineering and analytics.

Possesses expert knowledge of the contemporary data engineering and analytics landscape, including current tools, pertinent approaches and likely challenges.

A strong history of designing and constructing databases, analytics systems and Business Intelligence (BI) platforms within the context of Big Data.

A demonstrable ability to mentor and coach junior data engineers, sharing knowledge and expertise to help them grow in their roles.

The capability to collaborate efficiently with business stakeholders to understand their requirements and translate them into effective data solutions.
Strong support for Continuous Integration/Continuous Deployment (CI/CD) practices and an understanding of agile working methodologies.
Knowledge and experience working with Delta Lake, a storage layer that brings ACID transactions to data lakes.
Understanding of the Azure Platform setup including Subscriptions, Resource Group, IAM, Networking and DevOps.
Experience in setting up processes for cost/performance monitoring Spark jobs to improve maintainability.
Have deep knowledge of providing technical governance to enhance ways of working to improve the SDLC process.