- Ownership and delivery of the design, development, and maintenance of scalable data pipelines, infrastructure and data products using Databricks. Including implementation of efficient and reliable data ingestion, processing, and storage solutions at scale and prototyping new approaches.
- Develop and optimise extract, transform, load (ELT) processes ensuring smooth flow of data from various sources into the data platform. Perform data cleansing, validation, and enrichment to support accurate and reliable data analysis.
- Ensure adherence to data architecture principles, best practices and processes, collaborating with data scientists, analysts and stakeholders capturing and defining data requirements, designing appropriate data models and architectures whilst optimising data storage formats and partitioning for efficient data retrieval and analysis.
- Monitor and optimise data pipelines and workflows to ensure high performance and reliability. Identify and resolve performance bottlenecks, data quality issues, and data integration challenges.
- Define the data quality characteristics of your data products and work with the QA team to implement automated quality processes to measure data correctness and report outcomes through our observability infrastructure
- Implement data governance practices and ensure compliance with ITV data privacy and security regulations. Establish data access controls, encryption mechanisms, and data retention policies. Ensure all data products conform to all observability requirements and suitable dashboards are in place.
- Work closely with cross-functional teams, including data scientists, analysts, and software engineers, to understand their requirements and provide data engineering support. Document data pipelines, workflows, data presentation implementations and technical specifications for future reference.
- This will include writing and giving presentations to the wider team
- Extensive experience leading small teams of data engineers
-
Data & BI Engineer
1 week ago
Arthur London, United KingdomData Engineer / BI Engineer / Data / Power BI - Azure, Power BI, Acturis, ETL, SQL, Data, Python, data governance, quality, migration, modelling, Data warehousing · This role will report directly into the Head of Architecture & Engineering for the international business. · We are ...
-
BI Data Engineer
10 hours ago
VALLUM ASSOCIATES LIMITED London, United Kingdom**One of our energy client is looking for BI Data Engineer for a 3 months rolling contract** · **BI Data Engineer** · **3 months rolling** · **Remote** · - Design and implement data pipelines from a variety of sources into AWS Datalake warehouse (Snowflake) · - Stakeholder engage ...
-
BI Data Engineer
1 week ago
Sentient Talent London, United Kingdom**BI Data Engineer** · **London (Hybrid)** · **About the Role** · **Responsibilities** · - Design, build, and maintain the data infrastructure and systems that support our company's data needs, including data pipelines and data warehouse. · - Work with stakeholders to gather and ...
-
BI Data Engineer
3 days ago
Sentient Talent London, United Kingdom**About the Role** · **Responsibilities** · - Design, build, and maintain the data infrastructure and systems that support our company's data needs, including data pipelines and data warehouse. · - Work with stakeholders to gather and understand business and data requirements. · ...
-
BI Data Analyst
1 week ago
Executive Facilities London, United KingdomOur client is a global fintech company with UK headquarters in central London and corporate HQ in Silicon Valley, California. · They are now seeking a BI Data Analyst with 3 years+ experience preferably within a corporate environment for a 6 month contract. You will be using a va ...
-
Head of Data and Bi
1 day ago
Saga London, United KingdomJob Introduction · This role will ensure that the Data & BI function supports the business by producing accurate, complete, timely and relevant MI and supporting analysis for Saga Travel Group. · Responsible for the creation and ongoing management of Centre of Excellence team, th ...
-
Senior Data Engineer/ BI Developer
1 week ago
Harnham London, United Kingdom**Data Engineer · London/Flexible/Remote · £55,000-£75,000** · **THE COMPANY**: · The company is a rapidly growing UK-based Online Estate Agency. They are disrupting the traditional high street agencies by eradicating the unnecessary cost of high street shop fronts and delivering ...
-
BI & Data Delivery Lead
4 days ago
Maxwell Bond London, United Kingdom**Lead BI Delivery Manager** · **Hybrid - 2 days in Office in London** · **£70,000 - £80,000** · **No Visa Sponsorship Provided** · I am currently working with a leading financial services company committed to delivering innovative solutions and exceptional service to their clien ...
-
BI & Data Management Expert
1 week ago
Allen Lane London, United KingdomAllen Lane are delighted to be supporting our client in their search for an expereinced warehousing and data management specialist. · As the BI and Data Management expert you will have experience in: · - Ability to maintain and develop complex data warehouses · - ETL techniques ( ...
-
Power BI Engineer
10 hours ago
eFinancialCareers London, United KingdomYou will create and manage BI and analytics solutions that turn data into business knowledge to help facilitate the decision-making process within the organisation. · **Responsibilities** · - Designing, building, and deploying BI solutions · - Building Analysis Services reporting ...
-
Data BI Engineer
2 weeks ago
Methods Analytics England, United KingdomMethods Analytics are looking to hire a team of SC or DV cleared data engineers with strong experiences in Power BI.Location: Southwest England (5days Full time on site)Daily rate: £450 to £550 outside IR35Duration: 3 months with possible extensionWho you are:Must have live SC cl ...
-
BI/ Data Consultant
4 days ago
Montreal Associates London, United KingdomMicrosoft Data & Power Bi Consultant - RAL up to 45k + AUTO · Il nostro cliente è una società di consulenza multinazionale, leader nel settore Microsoft. · Chi cerchiamo? · In un'ottica di ampliamento del nostro team, stiamo cercando una persona proattiva che abbia esperienza ...
-
Data Engineer
1 week ago
Nigel Frank International London, United Kingdom**Data Engineer - London - £65k DoE** · I am currently working with a market-leading organisation who are looking to for a Data Engineer to join their team and help them in their goal of enhancing the way data is used in their specialist industry. In this role you will use data f ...
-
Data Engineer
6 days ago
Involved Solutions London, United Kingdom**Role: Data Engineer** · **Location: Remote / Central London** · **Rate: £300 to £500 per day** · **Determination: Inside IR35** · **Duration: 6 Months (initially)** · My client, an exciting leading organisation based in central London, are currently looking for an Data Engineer ...
-
Data Engineer
3 days ago
Nigel Frank International London, United KingdomData Engineer - Azure Databricks - London - £60K · I'm looking for a Data Engineer to work on developing products and solutions in Azure. you will have the opportunity to upskill in complex areas of cloud based engineering solutions in the very latest tech. · My client sit within ...
-
Data Engineer
2 days ago
eFinancialCareers London, United Kingdomcer Financial are working alongside a bank who are based in the City of London, they are looking for a Data Engineer to join them on a permanent basis · **The Responsibilities of a Data Engineer Will Include**: · - Design, develop, and hold accountability for data engineering ser ...
-
Data Engineer
2 days ago
Lorien London, United Kingdom**Data Engineer - Insurance** · We are currently recruiting for a Data Engineer with Insurance Industry experience to join one of our Insurance Clients on a 6 month initial contract, remote working. · Please note this role is **Inside IR35.** · You will support the team with on-g ...
-
Data Engineer
6 days ago
Harnham London, United Kingdom**DATA ENGINEER** · **LONDON - HYBRID** · **£55, ,000** · **THE COMPANY**: · A top insurance broker that is looking for a skilled Data Engineer with expertise in SQL server and ETL development. The company is looking to orchestrate a migration project from legacy systems to Azure ...
-
Data Engineer
2 days ago
Harnham London, United Kingdom**DATA ENGINEER** · **LONDON - HYBRID** · **£60, ,000** · **THE COMPANY**: · One of the UK's leading independent (re)insurance broker firms is looking for a skilled Data Engineer to have ownership of the cloud migration project they will be piloting in the New Year. The company h ...
-
Data Engineering
1 week ago
eFinancialCareers London, United KingdomThey are seeking an experienced Azure Data engineer to come onboard in a permanent capacity. · The role will require you to work in a well-developed BI and Analytics team where you will utilise your Azure skill set and build data solutions. The business has reported year on year ...
Lead Data/BI Engineer - London, United Kingdom - Entertainment Services Alliance, Inc.
Description
Shaping culture is in the DNA of ITV.So, it's not surprising that you'll find us in every home in the UK, our productions are famous all over the world and we're at the forefront of the digital streaming revolution.
With opportunities to learn, to grow and make a real difference. Our data team is responsible for helping us realise the value of data. That could mean anything from up-skilling and training our people, to increasing our revenue through experimentation and personalisation. The Innovation strand consists of Data Scientists, Operational Researchers and Behavioural Scientists.As we enter the fourth industrial revolution, anyone who joins the team will have the opportunity to work on genuinely groundbreaking innovations so that we can continue to blend science into a creative business.
As a Lead Data Engineer, both directly and via your team, you will be responsible for developing, maintaining, and optimising our data pipeline infrastructure using our proprietary central data platform (Symphony), which is based on Databricks.
You will collaborate with cross-functional teams to design and implement scalable data solutions, ensuring efficient data ingestion, transformation, storage, and analysis.
This is a hands-on leadership role where you will be responsible for a small team of data engineers.You will work in partnership with your agile delivery manager and product manager to size up and breakdown work, and distribute that across your team, and be proactive in bringing that activity to closure.
Deliver technical leadership monitoring productivity and output; contributing to reviews and performance assessment of a small number of data engineersStrong programming skills in languages such as Python (PySpark preferred), Scala, or SQL
Experience in owning, designing and implementing data pipelines ingesting enterprise levels of data volume with strong knowledge of data engineering concepts, data integration, and ETL processes
Ability to write production-grade code including automated testing and able to deploy code via CI/CD platforms (e.g. Experience in Proficiency in working with distributed computing frameworks, such as Apache Spark and data modelling, database systems, and SQL optimisation techniques
AWS, Azure, Google Cloud) and associated services (e.g., Knowledge of the UK broadcast industry & broadcast/OTT advertising market and digital marketing and advertising industry
Data design techniques for classification and regression as well as knowledge of A/B testing, experimental design, and general statistical modelling
Architecture disciplines such as Data Mesh Architecture, Data Architecture, BI Architecture and Enterprise Architecture
Data governance, data privacy, and security principles.
Tools across other data management capabilities – data engineering, data governance, data quality, metadata management and master data management
Designing and building out data visualisation solutions based on technology such as: Tableau, AWS QuickSight, Looker, ThoughtSpot