- It is imperative to have worked as a Principal or Lead Data Engineer in a previous business and can manage your own deliverables.
- Prior experience working with large datasets and a proficiency in SQL, Python and PySpark is required.
- Experience of managing a team of Engineers with varying degrees of experience.
- Experience of pipeline deployment within Azure Databricks in line with the medallion architecture framework.
- It would be highly beneficial to understand the London Insurance Market, with a particular preference given to Lloyds.
Lead Data Engineer - London Area, United Kingdom - Cornwallis Elt
Description
Lead Data Engineer – SQL, Python, PySpark, Azure, Databricks, Lloyds, Insurance, Permanent – London
A fast-growing global Insurance broker situated in the City of London is searching for a Lead Data Engineer to join their newly formed Data & Analytics team and play a pivotal role in their data transformation. At present, they're looking to transition from a program-based structure into a balanced operation through embracing agile methodologies to augment efficiency.
The Lead Data Engineer will be tasked with creating a Lakehouse platform in Azure by owning, managing, and implementing data engineering pipelines within Azure Databricks to ingest both structured and unstructured data from the global business. Furthermore, you will be responsible for managing a small team of engineers to help grow their careers and enable them to succeed.
Skills Required
This is an excellent opportunity to join a well-respected fast-growing Insurance business as they look to double their size in the next three years.