Data Engineer - City of London, United Kingdom - N P Associates

N P Associates

Verified Company

City of London, United Kingdom

4 weeks ago

Posted by:

Tom O´Connor

beBee Recruiter

Description

Data Engineer

This is an exciting opportunity for a Data Engineer to join a team of other highly skilled data professionals responsible for managing and enhancing the data estate.

Purpose of Role:

You will be part of a team responsible for data as a whole within the organisation, including data pipelines moving data from various on premise and cloud based source systems into the Azure based data platform for BI and analytics.

The Azure data platform consists of a number of Azure services including Data Factory, Data Lake Storage, Synapse Analytics, and Power BI.

This role also offers the opportunity to work with other database/data platforms like MongoDB and Splunk.

Responsibilities:

Designing and implementing data pipelines using Azure Data Factory, Spark, SQL feeding data from source systems through to the data lake, data warehouse and Power BI
Monitoring and supporting the existing data pipelines
Design and evolution of the Data Lake
Data Warehouse Schema modelling (Kimball/Star)
ELT Data transformation into the Data Warehouse Schema
Proficient in database performance analysis and tuning, particularly with Synapse SQL Pools.
Developing 'best practice' standards within the data team.
Continual learning about latest technologies in the data space.
Maintaining a consistently high level of service delivery for incidents, requests, and changes.
Maintain security standards across the data platform.

Skills & Experienced Required:

Good communication skills with technology staff and end users.
Strong technical knowledge of the Azure Data Platform including Data Factory, Data Lake, Synapse Analytics
Strong knowledge of Azure Synapse Analytics SQL Pools and how it differs to SQL Server for performance reasons
Strong SQL skills (SQL, T-SQL, PL/SQL)
Scripting skills (Python, PowerShell)
Data Lake Design optimal layout for performance, security etc
Experience in managing and migrating large volumes of data. Compression, partitioning and ETL
Experience of other data cloud technologies
AWS, Azure, Oracle Cloud.

Desirable:

Spark (Databricks/Synapse Spark Pools)
MongoDB
Splunk
Experience of Oracle Cloud (OCI)
Delta Lake
Experience of working in an Agile/SCRUM environment
A desire to learn and understand more about the Business environment in which you work.
Technical knowledge of SQL Server to 2019 and Oracle to 19c.