- Developing and maintaining robust Databricks notebooks and workflows.
- Working with Databricks Unity Catalog for Governance related tasks
- Driving the delivery of data to support new product features in line with strategic roadmaps.
- Upholding coding standards across various teams contributing to our data platforms.
- Collaborating with Software, Data, and Business teams to enhance our customer-centric culture.
- Proven track record in big data engineering with a solid understanding of ETL pipelines and data system projections.
- Proficiency in Python, PySpark, SQL, and familiarity with data science tools like R, ML, AI.
- Strong foundation in database management (SQL and No-SQL databases such as Aurora, MS SQL Server, MySQL).
- Experience with agile methodologies, TDD, BDD, pairing, and CI/CD practices.
- Strong knowledge of cloud platforms (AWS, Azure) and infrastructure as code tools (Terraform, AWS CDK).
Data Engineer - London, United Kingdom - Primus Connect
Description
Job Description
An excellent Data Product company have need of Data Engineers for very Databricks heavy roles to join their team.
They specifically like people with Databricks Certifications, but other Cloud Data stack certs (AWS, Azure, GCP etc) are also of interest to them.
These roles would support talented and driven Juniors through Mid-Level Data Engineers who want to continue learning and developing. Salaries between circa £50,000 - £60,000 + excellent benefits including a good bonus and the roles can be fully remote (so long as you are based in the UK).
Role Overview:
In this vital role, you will develop and maintain enterprise-grade software systems leveraging your expertise in Databricks, Python, Spark, R, and SQL. You will collaborate closely with our architecture team to design scalable, clean solutions that support continuous delivery and improvement. Your contributions will be crucial in automating tests, enhancing our continuous integration and delivery practices, and maintaining high standards in data processing and analytics.
Responsibilities Include:
Ideal Candidate Profile: