Data Engineer - London, United Kingdom - eFinancialCareers

eFinancialCareers
eFinancialCareers
Verified Company
London, United Kingdom

2 weeks ago

Tom O´Connor

Posted by:

Tom O´Connor

beBee Recruiter
Description

You will be part of a team responsible for data as a whole, including data pipelines moving data from various on premise and cloud based source systems into the Azure based data platform for BI and analytics.


The Azure data platform consists of a number of Azure services including Data Factory, Data Lake Storage, Synapse Analytics, and Power BI.

This role also offers the opportunity to work with other database/data platforms like MongoDB and Splunk.


Responsibilities:


  • Designing and implementing data pipelines using Azure Data Factory, Spark, SQL feeding data from source systems through to the data lake, data warehouse and Power BI
  • Monitoring and supporting the existing data pipelines
  • Design and evolution of the Data Lake
  • Data Warehouse Schema modelling (Kimball/Star)
  • ELT Data transformation into the Data Warehouse Schema
  • Proficient in database performance analysis and tuning, particularly with Synapse SQL Pools.
  • Developing 'best practice' standards within the data team.
  • Continual learning about latest technologies in the data space.
  • Maintaining a consistently high level of service delivery for incidents, requests, and changes.
  • Maintain security standards across the data platform.

Skills & Experienced Required:
Essential

  • Good communication skills with technology staff and end users.
  • Strong technical knowledge of the Azure Data Platform including Data Factory, Data Lake, Synapse Analytics
  • Strong knowledge of Azure Synapse Analytics SQL Pools and how it differs to SQL Server for performance reasons
  • Strong SQL skills (SQL, T-SQL, PL/SQL)
  • Scripting skills (Python, PowerShell)
  • Data Lake Design optimal layout for performance, security etc
  • Experience in managing and migrating large volumes of data. Compression, partitioning and ETL
  • Experience of other data cloud technologies
  • AWS, Azure, Oracle Cloud.
Desirable

  • Spark (Databricks/Synapse Spark Pools)
  • MongoDB
  • Splunk
  • Experience of Oracle Cloud (OCI)
  • Delta Lake
  • Experience of working in an Agile/SCRUM environment
  • A desire to learn and understand more about the Business environment in which you work.
  • Technical knowledge of SQL Server to 2019 and Oracle to 19c.
More jobs from eFinancialCareers
See all jobs of eFinancialCareers