Jobs
>
London

    Senior Data Engineer - London, United Kingdom - Lorien

    Lorien
    Lorien London, United Kingdom

    1 week ago

    Default job background
    Description

    Our client within the Education sector are looking for a Senior Data Engineer to join them on a contract basis.

    Inside IR35

    6 month contract

    Hybrid 1 – 2 days per week onsite

    Technical Skills:

    • In depth knowledge of relevant Azure services such as Azure Data Lake Storage, Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Azure SQL Database, Azure Stream Analytics, etc.
    • Strong Python or Scala with Spark, PySpark experience
    • Experience with relational databases and NoSQL databases
    • Significant experience and in-depth knowledge of creating data pipelines and associated design principles, standards, Data modelling concepts, ETL procedures, and all steps of data production process
    • Experience with unit and integration testing, and data quality frameworks such as Deequ, Great Expectations or Delta Live
    • Experience working with continuous integration
    • Good understanding of data modelling (medallion architectures, Kimball architectures), data schemas (Avro Schema, JSON Schema) and data serialisation (Parquet, ORC, Avro)

    Azure Services Knowledge:

    • Azure Data Factory: Advanced knowledge of Azure Data Factory for orchestrating and automating data workflows.
    • Azure SQL Database: Expertise in SQL and T-SQL for data querying and manipulation.
    • Azure Synapse Analytics: Ability to design and implement data warehousing solutions using Azure Synapse Analytics.
    • Azure Databricks: Proficiency in using Azure Databricks for data processing and analytics.
    • Apache Spark: Deep understanding of Apache Spark for large-scale data processing.
    • Azure Blob Storage and Azure Data Lake Storage: Expertise in setting up and optimizing Azure Data Lake Storage for storing and managing large volumes of data.

    Cloud Experience:

    • Resource Manager (ARM): Understanding how to deploy, manage, and monitor Azure resources using ARM templates.
    • Azure Portal and CLI: Proficiency in using the Azure Portal and Command-Line Interface for managing Azure resources.
    • Azure Security and Compliance: Knowledge of Azure security features and compliance standards.
    • Monitoring and Logging: Familiarity with Azure Monitor and Azure Log Analytics for monitoring and logging.
    • Ways of working:
    • Experience in Data Migration, Data Profiling and Data Quality enhancement.
    • Experience in Agile (Scrum) way of working
    • Understanding of CI/CD process & ways of working including Git
    • Excellent Problem-solving skills
    • Excellent Communication skills
    • Strong understanding of software development life cycle


    • Knowledge of software engineering best practices for the full software development life cycle, including Agile methodologies, coding standards, code reviews, source control management, build processes, testing, etc

    If you believe you have the right skills for this role please submit your CV.

  • Understanding Recruitment

    Data Engineer

    1 week ago

    Understanding Recruitment London, United Kingdom

    **Data Engineer** · A brand new and exciting opportunity has recently arisen for a Data Engineer with an extensive knowledge in designing, developing and managing ETL processes and writing SQL code to join the UK's Largest Government department based remote. As the Data Engineery ...

  • eFinancialCareers

    Data Engineer

    5 days ago

    eFinancialCareers London, United Kingdom

    Join us as a Data Engineer · - This is an exciting opportunity to use your technical expertise to collaborate with colleagues and build effortless, digital first customer experiences · - You'll be simplifying the bank through developing innovative data driven solutions, inspiring ...

  • eFinancialCareers

    Data Engineer

    1 week ago

    eFinancialCareers London, United Kingdom

    An exciting Data Engineering opportunity has become available within a multinational oil and gas company. The end client has a requirement for multiple engineering positions, to be part of a cross-disciplinary team working closely with other Data Engineers,Software Engineers, Dat ...

  • Harnham

    Data Engineer

    5 days ago

    Harnham London, United Kingdom

    **Data Engineer** · **London - Hybrid - Remote** · **£65,000** · Do you want to own the migration to a cloud platform? Are you looking to modernize a market-leading company to properly utilize its data and analytics? If so, this could be the role for you. · This insurance broker ...

  • Involved Solutions

    Data Engineer

    4 days ago

    Involved Solutions London, United Kingdom

    **Role: Data Engineer** · **Location: Remote / Central London** · **Rate: £300 to £500 per day** · **Determination: Inside IR35** · **Duration: 6 Months (initially)** · My client, an exciting leading organisation based in central London, are currently looking for an Data Engineer ...

  • Morgan Fraser Group Limited

    Data Engineer

    4 days ago

    Morgan Fraser Group Limited London, United Kingdom

    This is a long term contract role working both inside and outside IR35. · This is an exciting contract position whereby you'll be performing as a Data Engineer, delivering services for a prestigious financial company based in London. · This particular project requires you to be a ...

  • Harnham

    Data Engineer

    4 days ago

    Harnham London, United Kingdom

    **Data Engineer** · **London** · **Up to £45,000 + Competitive Bonus and Benefits** · **The Company** · This is an exciting opportunity to join a disruptive challenger bank. You will work in a dedicated engineering function with the chance to learn and pick up the newest Big Data ...

  • Anson McCade Ltd - IT and Finance Recruitment

    Data Engineer

    19 hours ago

    Anson McCade Ltd - IT and Finance Recruitment London, United Kingdom

    **Data Engineer** · **_London, Bristol, Gloucester_** · **_£55-75k_** · Attention all Data Engineers Are you looking for an exciting and challenging opportunity to work on large scale projects and help shape specific solutions for clients? Then we want you · We are currently seek ...

  • Circle Recruitment

    Data Engineer

    4 days ago

    Circle Recruitment London, United Kingdom

    **Data Engineer** · **£600 - £700/Day Inside IR35** · **London, Hybrid** · **6 Months** · We are seeking a Data Engineer for our Government client based in London. This role will be office based 2 days per week and the remaining remote. The Data Engineer role will be and initial ...

  • ADLIB

    Data Engineer

    19 hours ago

    ADLIB London, United Kingdom

    **Leading financial services data & analytics consulting arm.**: · - **Work with Azure, AWS and GCP services.**: · - **Client-facing role working alongside data architects and scientists.** · A market leading consultancy is growing out its Financial Services Data & Analytics prac ...

  • eFinancialCareers

    Data Engineer

    1 week ago

    eFinancialCareers London, United Kingdom

    **Permanent Role** · **London Based (Hybrid)** · **No Sponsorship** · **Role** · - Test/evaluate new alternative datasets in collaboration with quant researchers and data teams · - Once identified with machine learning and statistical methods, automate the extraction of predictiv ...

  • Harnham

    Data Engineer

    4 days ago

    Harnham London, United Kingdom

    **DATA ENGINEER** · **LONDON BASED** · **£65,000-75,000 PER ANNUM** · This UK-based performance agency are searching for a new Data Engineer, to join their expanding Data and Analytics team. You will be responsible for building and maintaining data pipelines using Python, and col ...

  • Data Idols

    Data Engineer

    5 days ago

    Data Idols London, United Kingdom

    **Senior Data Engineer** Data Idols are proud to be working with an iconic UK brand to assist them in hiring a Senior Data Engineer as they undertake a digital and data-driven journey that will transform the business. Data is at the heart of the organisation and forms a key parto ...

  • Harnham

    Data Engineer

    4 days ago

    Harnham London, United Kingdom

    **Data Engineer** · **Birmingham** · **Up to £70,000 + Competitive bonus & Benefits** · **The Company** · Be part of a global organisation that will help you develop your career and progress through the company. This reputable Footsie-100 company operates in over 180 countries an ...

  • Experis LTD

    Data Engineer

    19 hours ago

    Experis LTD London, United Kingdom

    **Role: Data Engineer** · **Duration:6th months initially(Extension Expected)** · **Location: London (2 TO 4 Days a month need to go onsite)** · **Rate: £550 per day inside IR35** · **RESPONSIBILITIES**: · - Create and maintain optimal data pipeline architecture · - Assemble larg ...

  • Data Idols

    Data Engineer

    1 week ago

    Data Idols London, United Kingdom

    **Data Engineer** Data Idols are excited to be working with one of the top 100 companies in the world to recruit a Data Engineer A company which owns multiple brands and who are investing heavily into 'all things data' to revolutionise their extraordinary team. · **Skills and Exp ...

  • Jefferson Frank

    Data Engineer

    2 days ago

    Jefferson Frank London, United Kingdom

    **Data Engineer** · **Hybrid/London** · **Up to £120k DOE** · **No Sponsorship Available** · Jefferson Frank have been enlisted to work with a global analytics and digital solutions company. They partner with organisations to build out their capabilities in analytics, cloud and A ...

  • Method Resourcing

    Data Engineer

    1 week ago

    Method Resourcing London, United Kingdom

    **Data Engineer | Python | SQL | GCP | BigQuery | Data Studio | ETL | DBT | Stitch | Fivetran | Singer | £40,000-£50,00** · Method Resourcing are currently working with a Media Publication company that report on the world of data and provide publications to over 300,000 data prof ...

  • Harnham

    Data Engineer

    5 days ago

    Harnham London, United Kingdom

    **Data Engineer** · **London, Flexible, Remote** · **£70,000** · Are you wanting progression? Do you want to work in an environment where you can develop technical skills whilst also being involved in innovative projects? If so, this could be the role for you · This leading media ...

  • VALLUM ASSOCIATES LIMITED

    Data Engineer

    4 days ago

    VALLUM ASSOCIATES LIMITED London, United Kingdom

    **Mid-Senior Data Engineer - Energy & Financial Solutions (Commodities/Trading) - Permanent** · **Location: London or Swindon (Hybrid remote working)** · **Salary : Mid 70-85k, Snr 85-100k + bonus** · Are you an experienced or passionate Data Engineer looking to broaden your hori ...