Data Engineer - Newcastle upon Tyne, United Kingdom - Jefferson Frank

Jefferson Frank

Verified Company

Newcastle upon Tyne, United Kingdom

3 weeks ago

Posted by:

Tom O´Connor

beBee Recruiter

Description

Data Engineer - Salary to £60,000 - Newcastle/Hybrid
Our client is a large online retailer working in a very specific industry.

Over the last 12 months they have seen unprecedented growth and with that in mind are looking to expand their data function.

With investment currently focused on new products and technologies they are fast becoming the only competitor within their market share that is growing at this rate.

The role itself will be working on quite an exciting transformation project. Moving from monolithic Azure based systems to an new AWS based infrastructure.

If you have exposure to working on similar projects whilst being involved with toolings such as Databricks, Python and strong ETL knowledge this could be a great position for you.

Responsibilities:

Data Ingestion: Build and optimize data ingestion pipelines using AWS services such as Lambda, Kinesis, and Glue.
Data Transformation: Develop, maintain, and optimize transformation pipelines within our Delta Lake on Databricks.
Data Modelling: Design and implement efficient and reliable data models to store and retrieve data.
Collaboration: Work closely with data scientists, analysts, and stakeholders to understand data requirements and deliver optimal solutions.
Data Quality: Ensure the reliability, integrity, and timeliness of data by implementing best practices and quality checks.
Performance Optimization: Optimize queries and improve performance of data retrieval operations.
Technology Stewardship: Stay updated with the latest advancements in AWS, Databricks, and the data engineering field in general. Provide recommendations on the adoption of new tools and technologies.

Requirements:

AWS Services: Proficiency in AWS services like Lambda, Kinesis, and Glue.
Databricks &

Delta Lake:
Hands-on experience with Databricks and Delta Lakehouse architectures.

Programming: Proficient in Python, Scala, and SQL for data engineering tasks.
ETL Processes: Demonstrated expertise in designing, building, and maintaining ETL processes.
Data Modelling: Experience in logical and physical data modelling with large datasets.
Problem Solving: Strong analytical and problemsolving skills, with a focus on troubleshooting data issues

Benefits: