Data Specialist Engineer - London, United Kingdom - N Consulting Ltd

    N Consulting Ltd
    N Consulting Ltd London, United Kingdom

    2 weeks ago

    Default job background
    Contract
    Description

    Role Title:
    Data Specialist Engineer


    Possibility of remote work:
    NO | Hybrid 3 days in the office

    Contract duration: 6 months


    Location:
    London


    Required Core Skills:

    • Databricks
    • Azure Data Factory
    • Spark, Scala
    • Azure DevOps/GIT Repo
    • Azure Artifact and CI/CD using Azure Pipelines
    • End to End Release and Branching strategies
    Minimum years of experience: 6 years


    Areas of responsibility:

    • Stake holder management skills
    • Nice to have energy trading experience
    • Team Handling

    Detailed Job Description:

    • Write, and maintain build pipelines for enabling CI for seamless code builds, packaging, and unit tests.
    • Develop, deploy and maintain data pipelines and custom libraries required for Common Data Model (CDM) conformance.
    • Enhance the PR review process by implementing automatic build validation checks, automatic approval process, etc.
    • Enhance the code and test coverage reports as part of build pipelines to ensure continuous improvement of code quality.
    • Design, develop and deliver large scale data pre-ingestion, ingestion and data transformation use-cases on Azure Data Hub.
    • Deploy and maintain reliable and scalable data infrastructure needed in data ingestion pipelines development stage.
    • Implement data transformation pipelines in line with provided mapping document, using Azure Data Factory and Databricks to deliver data in enriched layer to a target CDM model.
    • Build and deploy enriched and curated views for delivering analytical and business value out of various transactional and reference data sources.
    • Assist with the configuration and setup of Power BI reports