Jobs
>
Sheffield

    Data Engineer - Sheffield, United Kingdom - Job N Job (UK) (US) (CA) (PK)

    Job N Job (UK) (US) (CA) (PK)
    Job N Job (UK) (US) (CA) (PK) Sheffield, United Kingdom

    3 weeks ago

    Default job background
    Description

    Job Title: bCLEARer Data Engineer

    Location: Remote (UK, India, US, Switzerland)

    Contract Duration: 15 months to 2 years, with possibility of extension

    Country of Work Location: Any of 3 IT Hubs.

    flexible, need for resources to support all regions and need to work with our architects in US, UK, and India.

    Job Purpose

    The data engineers are required to develop, operate, and maintain data pipelines for modernizing legacy data based on a semantic methodology (bCLEARer). Knowledge of this methodology is not required, however, some background in semantics and ontology will be preferred.

    These pipelines will support the digitalization, cleansing and transition of legacy data into the target systems as well as support ongoing integration and governance. The data pipelines need to be developed based on a core semantic data model that will be used to define and validate data quality and consistency rules across source systems and to transformation and transition data to the target system.

    Special Challenges:

    Technical Skills and Knowledge

    Specific SNOW Skillsets: data analysis, data platforms, database management and design, and Azure PaaS

    Additional skillsets: data pipeline development, ETL, stream analytics, data visualization, graph databases and graph visualization

    Skills & Requirements:

    Key Skills and Experience

    • understanding of data engineering principles and tools at least two years of professional databricks / python experience required.
    • of software development standards including clean coding and clean architecture principles
    • knowledge of data structures and data transformation logic development
    • knowledge of pattern matching and heuristic algorithms (including as a minimum regex-based solutions) of using object oriented and procedural languages for data engineering operations (Python required)
    • knowledge of agile practices.
    • knowledge of test-driven development.
    • knowledge of azure dev board (previously VSTS).
    • knowledge of azure CI / CD PowerShell and azure cli. of cloud stack (Azure) for cloud-based components of the process. Experience in some of the cloud-based capabilities:
    • storage infrastructure including Blob stores
    • data processing platform: Databricks with Databricks Dataframe and http APIs.
    • understanding of Databricks RDD API
    • understanding of Databricks query optimization techniques.

    What would be a plus.

    • understanding of graph operations, e.g. tree traversal, and network analysis.
    • of semantics and ontological modelling
    • of object-oriented data models.
    • of graph and ontology stores.
    • with engineering data platforms specifically an Engineering Data Warehouse.
    • Engineering and Facilities domain knowledge/experience.
  • Harnham - Data & Analytics Recruitment

    Data Engineer

    6 days ago

    Harnham - Data & Analytics Recruitment Sheffield, United Kingdom

    **Data Engineer** · **Fully remote** · **£45, 000** · **Permanent** · An exciting opportunity for a Data Engineer looking to make an impact in the public health sector, in a rapidly growing start-up company. · **The Company** · A great opportunity for a Data Engineer to join a nu ...

  • Harnham - Data & Analytics Recruitment

    Data Engineer

    1 week ago

    Harnham - Data & Analytics Recruitment Sheffield, United Kingdom

    **Data Engineer** · **Fully remote** · **£45, 000** · **Permanent** · An exciting opportunity for a Data Engineer looking to make an impact in the public health sector, in a rapidly growing start-up company. · **The Company** · A great opportunity for a Data Engineer to join a nu ...

  • Airship Services Ltd

    Cloud Data Engineer

    6 days ago

    Airship Services Ltd Sheffield, United Kingdom

    Airship have an exciting opportunity for a **Cloud Data Engineer** to join the team. · Location: Sheffield · **Salary**: £40-70k depending on experience. · **About Us**: · Airship builds products that serve the best and most exciting brands in the UK hospitality industry - think ...

  • Harnham - Data & Analytics Recruitment

    Senior Data Engineer

    2 weeks ago

    Harnham - Data & Analytics Recruitment Sheffield, United Kingdom

    **Senior Data Engineer** · **Remote** · **£125, 000** · **Permanent** · An exciting opportunity for a Senior Data Engineer to join a fintech start-up and set the infrastructure for a modern data platform to aid decision-making. · **The Company** · A fintech start-up is looking fo ...

  • In Technology Group Limited

    AI Data Modelling Engineer

    1 week ago

    In Technology Group Limited Sheffield, United Kingdom Full time

    AI Data Modelling Engineer - £45K - Sheffield (Hybrid) · A Sheffield based, forward thinking, and award winning business are looking to add to their existing team with the acquisition of a new AI Data Modelling Engineer. A role that will see the successful candidate developing m ...

  • The National College

    Data Engineer

    2 weeks ago

    The National College Sheffield, United Kingdom Full time

    We have implemented a modern cloud-based analytics tech stack, bringing together data from our various systems into a data warehouse platform which will provide real-time accurate and critical insights and analytics for the business. · This is where you come in. As a Data Engine ...

  • Pearson Carter

    Data Engineer

    1 week ago

    Pearson Carter Sheffield, United Kingdom

    Job Description · Data Engineer - Sheffield Hybrid - Up to £60,000 · Tech Stack : AWS/Python/SQL · Location: Hybrid in Sheffield. · Benefits: Flexible office hours, work from home days, a fun and relaxed environment as well as a casual dress code. · Company: Here at Pearson Cart ...

  • Pearson Carter

    data engineer

    1 week ago

    Pearson Carter Sheffield, United Kingdom

    Job Description · DATA ENGINEER – UP TO £60,000 · Location: Sheffield · Benefits: 25 Days + Bank Holidays Life Assurance Flexible Working · Company: · A chance to have a positive societal impact if you launch yourself into a fast growing, but firmly established, company in the mu ...

  • UK Civil Service

    Data Engineer

    1 week ago

    UK Civil Service Sheffield, United Kingdom

    Job summary · Please note�this role requires clearance. For further information, please see 'Selection process details'. · Do you want to utilise best-in-class technologies to deliver data products? · Are you looking for ongoing professional development while helping design and b ...

  • Pearson Carter

    Data Engineer

    5 days ago

    Pearson Carter Sheffield, United Kingdom

    Data Engineer · Pearson Carter are currently working with a Education consultancy based in Sheffield who are looking for a Data Engineer who has strong AWS cloud, SQL and Python experience. · They offer a base salary of £50,000 and £60,000 · Get in touch if you are looking for yo ...

  • Pearson Carter

    Data Engineer

    3 days ago

    Pearson Carter Sheffield, United Kingdom

    Data Engineer £50,000 - £60,000 - Hybrid · My client, industry leading EdTech organisation are currently looking to expand their Data team · become a Data Engineer with vast progression opportunities, where your work is all for a great cause · Requirements · Thrive in a fast pace ...

  • Pearson Carter

    Data Engineer

    4 days ago

    Pearson Carter sheffield, south yorkshire, United Kingdom

    Job DescriptionData Engineer · Pearson Carter are currently working with a Education consultancy based in Sheffield who are looking for a Data Engineer who has strong AWS cloud, SQL and Python experience. · They offer a base salary of £50,000 and £60,000 · Get in touch if you are ...

  • BJSS

    Data Engineer

    2 weeks ago

    BJSS Sheffield, United Kingdom

    Job Description · About Us · We're an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we've been finding better, more sustainable ways to solve complex technology problems for some of the world's leading organisations and delivered sol ...

  • Pearson Carter

    Data Engineer

    5 days ago

    Pearson Carter Sheffield, United Kingdom

    Data Engineer · Pearson Carter are currently working with a Education consultancy based in Sheffield who are looking for a Data Engineer who has strong AWS cloud, SQL and Python experience. · They offer a base salary of £50,000 and £60,000 · Get in touch if you are looking fo ...

  • The National College

    Data Engineer

    3 days ago

    The National College Sheffield, United Kingdom Full time

    We have implemented a modern cloud-based analytics tech stack, bringing together data from our various systems into a data warehouse platform which will provide real-time accurate and critical insights and analytics for the business. · This is where you come in. As a Data Engine ...

  • Pearson Carter

    Data Engineer

    4 days ago

    Pearson Carter Sheffield, United Kingdom £50,000 - £60,000

    Job Description Data Engineer · Pearson Carter are currently working with a Education consultancy based in Sheffield who are looking for a Data Engineer who has strong AWS cloud, SQL and Python experience. · They offer a base salary of £50,000 and £60,000 · Get in touch if you ...

  • Pearson Carter

    Data Engineer

    1 day ago

    Pearson Carter Sheffield, United Kingdom £50,000 - £60,000

    Data Engineer · Pearson Carter are currently working with a Education consultancy based in Sheffield who are looking for a Data Engineer who has strong AWS cloud, SQL and Python experience. · They offer a base salary of £50,000 and £60,000 · Get in touch if you are looking for ...

  • Pearson Carter

    Data Engineer

    1 day ago

    Pearson Carter Sheffield, United Kingdom £60,000

    DATA ENGINEER – UP TO £60,000 Location : Sheffield Benefits: 25 Days + Bank Holidays Life Assurance Flexible Working Company: A chance to have a positive societal impact if you launch yourself into a fast growing, but firmly established, company in the much sought after EdTech se ...

  • Pearson Carter

    Data Engineer

    3 days ago

    Pearson Carter Sheffield, United Kingdom

    Data Engineer · Pearson Carter are currently working with a Education consultancy based in Sheffield who are looking for a Data Engineer who has strong AWS cloud, SQL and Python experience. · They offer a base salary of £50,000 and £60,000 · Get in touch if you are looking for yo ...

  • UK Civil Service

    Senior Data Engineer

    5 days ago

    UK Civil Service Sheffield, United Kingdom

    Job summary · Please note�this role requires you to pass clearance. For further information, please see 'Selection process details'. · DWP is the UK's largest government department. We help people into work and make payments worth over �195bn a year to support some of the most vu ...