Senior Engineer, Data Engineering - City of London, Greater London, United Kingdom - Aventum Group

Aventum Group City of London, Greater London, United Kingdom

2 weeks ago

Description

Job Title - Data Engineer

Reports to - Head of Data

Location - London/Hybrid

A family that dreams big and always works together to make insurance better.

We offer our employees a unique benefits package that includes personal training sessions, well-being provisions, and an additional day off for your birthday You will work in various settings to build systems that collect, manage, and convert raw data into usable information for data scientists and business analysts to interpret.

Your ultimate goal is to make data accessible so that Aventum stakeholders can use it to evaluate and optimise their performance.

Technical and business processes are used to combine data from multiple sources to provide a unified, single view of the data.

Partly responsible for accessing, validating, and querying data from various repositories using available tools.

Build and maintain data integration processes using SQL Services and other ETL / ELT processes and scripting tools as well as ongoing requests and projects related to the data warehouse, MI, or fast-moving financial data.

Designing the infrastructure/architecture of the big data platform.
Evaluating, comparing and improving the different approaches including design patterns innovation, data lifecycle design, data ontology alignment, annotated datasets, and elastic search approaches
Developing, creating and maintaining a reliable data pipeline and schemas that feed other data processes; includes both technical processes and business logic to transform data from disparate sources into cohesive meaningful and valuable data with quality, governance and compliance considerations.
Customising and managing integration tools, databases, warehouses, and analytical systems.
Design and implement the management, monitoring, security, and privacy of data using the full stack of Azure data services to satisfy business needs.
Ensuring non-functional system characteristics such as such as security, maintainability, quality, performance, and reliability are captured, prioritized, and incorporated into products.
Leverage Agile, CI / CD and DevOps methodologies to deliver high-quality products on time.
Architecting, building, testing, and maintaining data platform. Develop and support a wide range of data transformations and migrations for the whole business.
Design and implement data pipelines, data marts and schemas, access versatile data sources and apply data quality measures.
Monitoring the complete process and applying necessary infrastructure changes to speed up the query execution and analyse data more efficiently; this includes Database optimisation techniques (data partitioning, database indexing and denormalisation) & efficient data ingestion (data mining techniques and different data ingestion APIs).
Responding to errors and alerts to correct and re-process data. Investigate data mismatches and apply solutions. Data scrubbing and analysis to troubleshoot data integration issues and determine root cause.
Bachelor's degree or equivalent in an engineering/numerate subject (e.g. Engineering, Stats, Maths, Computer Sciences)
Experience in full-stack development and applying it to build science products (E.g. could include some or all Python/R, Linux scripting, SQL, Docker coupled with front ends such as Javascript)
Some experience as a developer building data pipelines and schemas, with data WH implementation, with SQL database development
Stored Procedures, ADF, NoSQL Databases, JSON/XML data formats)
Hands-on experience with Azure Functions, Azure service bus, Azure Data Factory data integration techniques
Knowledge of Data Modelling concepts, monitoring, designs and techniques
Knowledge of Data Warehouse project lifecycle, tools, technologies, and best practices
Experience using Cloud Computing platforms (ADLS Gen2), Microsoft Stack (Synapse, DataBricks, Fabric, Profisee), Snowflake Data Integration, Azure Service Bus, Deltalake, BigQuery, Azure DevOps, Azure Monitor, Azure Data Factory, SQL Server, Azure DataLake Storage, Azure App Service, Apache Airflow, Apache Iceberg, Apache Spark, Apache Hudi, Apache Kafka, Power BI, BigQuery, Azure ML is a plus
Experience with Azure SQL Database, Cosmos DB, NoSQL, MongoDB
Experience with Agile, DevOps methodologies
Awareness and knowledge of ELT/ETL, DWH, APIs (RESTful), Spark APIs, FTP protocols, SSL, SFTP, PKI (Public Key Infrastructure) and Integration testing

Knowledge of Python, SQL, SSIS, and Spark languages.

Demonstrative ability to develop complex SQL queries and Stored Procedures
Relationship-building and stakeholder management

We value applicants from all backgrounds and foster a culture of inclusivity. We understand the need for flexibility, so work in a hybrid model.

Data Engineer

3 weeks ago

ADLIB City of London, United Kingdom

**Luxury Fashion House**: · - End-to-end BI development in a multi-cloud environment. · - Path to Lead Data Engineer. · - Great benefits package and remote working. · My client is a luxury fashion brand who are currently on the hunt for a Data Engineer to join their BI Platform T ...
Data Engineer

3 weeks ago

Avanti City of London, United Kingdom

Mid - Senior Data Engineer · I'm working with a £1Bn European fintech who are looking to break into the UK market and have big plans to expand their UK team during the near year. They create innovative banking as a service platforms already used by some of the biggest names in Fi ...
Data Engineer

3 weeks ago

Lutine Bell City of London, United Kingdom

**Job description** · **Responsibilities** · - Collaborate and work with stakeholders to identify requirements for complex business problems that may be loosely defined · - Collaborate with a team of data scientists, data engineers and analysts with the aim of exploratory data di ...
Data Engineer

2 weeks ago

IGA Talent Solutions City of London, United Kingdom

Data Engineer - 6 month contract - Remote - Inside IR35 -£400-£425 Per day · I am currently looking for a Data Engineer for a client of mine for a 6 month fully remote contract. This role is paying £400-£425 per day and is inside IR35. · To be considered for this role you need th ...
Data Engineer

1 week ago

ARCUS SEARCH LIMITED City of London, United Kingdom

**Data Engineer** · This is an exciting opportunity to grow, develop your skills and evolve. · **Experience**: · - **Technical**: Excellent knowledge and use of SQL. Snowflake experience is an advantage · - Knowledge and competent use of one or more of python, R, Tableau, and GIS ...
Data Engineer

2 weeks ago

Levin Group City of London, United Kingdom

**Job Description: Data Engineer - £50-65k + 15% Bonus DOE (Remote)** · **Levin** · **Purpose**: · Define, build, maintain and oversee Levin Groups Data Architecture and underlining cloud data services (MS Azure). Deliver ETL & DWH capabilities to provide the business enhanced re ...
Data Engineer

21 hours ago

Data Idols City of London, United Kingdom

**Data Engineer - Biotechnology** Data Idols are working with an exciting and innovative biotechnology startup who are working to assist biopharma teams to find better medicines and drugs quicker and more efficiently. · **The Opportunity** This is an opportunity for a Data Engine ...
Data Engineer

3 days ago

Comtecs Ltd City of London, United Kingdom

Lead Data Engineer / Data Architect - ETL/ELT Pipelines, Data Architecture, ETL Transformation, Cloud Engineering, AWS, Kubernetes, Terraform, SQL, Technical Advisory & Consultancy, Implementation. Insurance/FinTech. London/Remote (One Day PER MONTH Onsite).12 Month Contract - OU ...
Data Engineer

2 days ago

BLUE WOLF DIGITAL LTD City of London, United Kingdom

**Data Engineer** · **UK Remote** · **up to £550 per day (awaiting IR35 outcome)** · **3 months with extension throughout 2023** · I have an urgent resource requirement for an experienced Data Engineer who has experience and knowledge on data extraction using APIs. · - Different ...
Data Engineer

1 week ago

N P Associates City of London, United Kingdom

**Data Engineer** · This is an exciting opportunity for a Data Engineer to join a team of other highly skilled data professionals responsible for managing and enhancing the data estate. · **Purpose of Role**: · You will be part of a team responsible for data as a whole within the ...
Data Engineer

1 week ago

Colossus Associates City of London, United Kingdom

Senior Data Engineer · Hybrid · About us · Hybrid role - Based in London · About the role · We are seeking a committed and passionate Quantitative Data Engineer to work with us on our climate technology platform · The impact you'll own · - Ability to utilise Natural Language Proc ...
Data Engineer

3 days ago

Career Legal City of London, United Kingdom

3+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems, or another quantitative field. · - Advanced working knowledge and experience of relational (SQL) databases, as well as working fam ...
Data Engineer

21 hours ago

Xpertise Recruitment Ltd City of London, United Kingdom

**Role **: Data Engineer · **Location **: London · **Salary **: £90,000-100,000, with additional benefits · We have a great opportunity with one of our financial services clients looking for someone to join their IT team, providing data expertise. You'll be working closely with t ...
Gcp Data Engineer

3 weeks ago

BLUE WOLF DIGITAL LTD City of London, United Kingdom

**GCP Data Engineer** · **Remote (UK) with occassional client travel** · **£80,000 - £110,000** · I'm currently working with a London-based consultancy to find a Senior GCP Data Engineer to add to their team. The role is new and is due to continued success and growth. The role is ...
Lead Data Engineer

2 weeks ago

Data Idols City of London, United Kingdom

**Lead Data Engineer** Data Idols are proud to be working with an iconic UK brand to assist them in hiring a Lead Data Engineer as they undertake a digital and data-driven journey that will transform the business. Data is at the heart of the organisation and forms a key part ofth ...
Snowflake Data Engineer

1 week ago

BLUE WOLF DIGITAL LTD City of London, United Kingdom

**Snowflake Data Engineer** · **Remote with occasional London office visits** · **up to £450-650pd (OUTSIDE IR35)** · - At least 12 months' experience using Snowflake as a primary technology · - Experience of setting up the security model, data anonymisation and classification wi ...
Data Engineer Snowflake

2 weeks ago

Trust In Soda City of London, United Kingdom

Are you looking to work for a global consultancy that is ranked in the top 5 in the world and top 4 most diverse? They offer curated career development to make sure that can develop your career the way you want, and they have an amazing mentorship programto help you navigate to y ...
Senior Data Engineer

3 days ago

Premier IT City of London, United Kingdom

A fantastic new opportunity which has just landed on my desk working for one of the UK's biggest real estate agencies based in their Head Office in London - They are looking to employ an experience and skilled Senior Data Engineer to join their Engineeringteam who work within the ...
Azure Data Engineer

1 week ago

BCT Resourcing City of London, United Kingdom

Azure Data Engineer | Financial Services Client · City of London · £60,000 - £80,000 per annum · Company client is working with a renowned Financial Services business looking to hire a Senior Data Engineer to join the Data team. · This is an opportunity to work with a technology ...
Lead Data Engineer

21 hours ago

Harnham - Data & Analytics Recruitment City of London, United Kingdom

Job Title: Lead Data Engineer · Location: London · **Salary**: Up to £100K + 10 % - 20 % · **Job Description**: · We are seeking a highly motivated and experienced Lead Data Engineer to join a data-centric insurance company. As the Lead Data Engineer, you will be responsible for ...