Data Engineer - City of London, United Kingdom - N P Associates

Tom O´Connor

Posted by:

Tom O´Connor

beBee Recruiter


Description

Data Engineer


This is an exciting opportunity for a Data Engineer to join a team of other highly skilled data professionals responsible for managing and enhancing the data estate.


Purpose of Role:


You will be part of a team responsible for data as a whole within the organisation, including data pipelines moving data from various on premise and cloud based source systems into the Azure based data platform for BI and analytics.


The Azure data platform consists of a number of Azure services including Data Factory, Data Lake Storage, Synapse Analytics, and Power BI.

This role also offers the opportunity to work with other database/data platforms like MongoDB and Splunk.


Responsibilities:


  • Designing and implementing data pipelines using Azure Data Factory, Spark, SQL feeding data from source systems through to the data lake, data warehouse and Power BI
  • Monitoring and supporting the existing data pipelines
  • Design and evolution of the Data Lake
  • Data Warehouse Schema modelling (Kimball/Star)
  • ELT Data transformation into the Data Warehouse Schema
  • Proficient in database performance analysis and tuning, particularly with Synapse SQL Pools.
  • Developing 'best practice' standards within the data team.
  • Continual learning about latest technologies in the data space.
  • Maintaining a consistently high level of service delivery for incidents, requests, and changes.
  • Maintain security standards across the data platform.

Skills & Experienced Required:


  • Good communication skills with technology staff and end users.
  • Strong technical knowledge of the Azure Data Platform including Data Factory, Data Lake, Synapse Analytics
  • Strong knowledge of Azure Synapse Analytics SQL Pools and how it differs to SQL Server for performance reasons
  • Strong SQL skills (SQL, T-SQL, PL/SQL)
  • Scripting skills (Python, PowerShell)
  • Data Lake Design optimal layout for performance, security etc
  • Experience in managing and migrating large volumes of data. Compression, partitioning and ETL
  • Experience of other data cloud technologies
  • AWS, Azure, Oracle Cloud.

Desirable:


  • Spark (Databricks/Synapse Spark Pools)
  • MongoDB
  • Splunk
  • Experience of Oracle Cloud (OCI)
  • Delta Lake
  • Experience of working in an Agile/SCRUM environment
  • A desire to learn and understand more about the Business environment in which you work.
  • Technical knowledge of SQL Server to 2019 and Oracle to 19c.

More jobs from N P Associates