- Design, develop, and maintain robust data pipelines using DLT on Databricks.
- Collaborate with software engineers, data scientists and platform engineers to understand data requirements and deliver high-quality solutions.
- Implement ETL/ELT processes to ingest, transform, and store data from various sources (structured and unstructured).
- Optimize performance and cost-efficiency of data workflows on Databricks.
- Ensure data quality, integrity, and governance through validation, monitoring, and documentation.
- Develop reusable components and frameworks to accelerate data engineering efforts.
- Support CI/CD practices and automation for data pipeline deployment.
- Stay current with Databricks features and best practices, and advocate for their adoption.
- Solid understanding of data modelling, warehousing concepts, and distributed computing.
- Familiarity with Delta Lake and Unity Catalog.
- Knowledge of data governance frameworks and compliance standards (eg, GDPR, HIPAA).
- Strong programming skills in Python and SQL.
- Experience with version control (eg, Git) and CI/CD tools.
- Excellent problem-solving and communication skills, both written and oral, with proven experience as a Data Engineer with hands-on expertise in Databricks and DLT.
- Experience with cloud data platforms, ideally Azure but experience with AWS or Google would be an advantage.
- Exposure to machine learning workflows and integration with ML models.
- Delivering results working in a distributed, cross-functional team.
- Databricks certification (eg, Databricks Certified Data Engineer Associate/Professional).
Data Engineer 24 Month Fixed Term Contract - Hunton Bridge - RES Limited
Description
Overview
We are seeking a skilled Data Engineer with expertise in Databricks to join our asset performance management software team, within our Digital Solutions business. Working with other data engineers, and our platform team, you will be responsible for designing, building, and optimizing scalable data pipelines using the Databricks platform. This role is ideal for someone passionate about big data technologies, cloud platforms, and enabling data-driven analytics and ML to report and improve on the performance of renewable assets.
Responsibilities
Qualifications
About the Role
We're the world's largest independent renewable energy company. We're driven by a simple yet powerful vision: to create a future where everyone has access to affordable, zero carbon energy. We know that achieving our ambitions would be impossible without our people. RES is a family with a diverse workforce, and we are dedicated to the personal professional growth of our people, no matter what stage of their career they're at. We can promise you rewarding work which makes a real impact, the chance to learn from inspiring colleagues from across a growing, global network and opportunities to grow personally and professionally. Our competitive package offers rewards and benefits including pension schemes, flexible working, and top-down emphasis on better work-life balance. We also offer private healthcare, discounted green travel, 25 days holiday with options to buy/sell days, enhanced family leave and four volunteering days per year so you can make a difference somewhere else. Please note this position is a 24 month fixed term contract.
#J-18808-Ljbffr