Senior Data Engineer - London, United Kingdom - Jefferson Frank
Description
Senior Data Engineer
London
Global Telco Group
Hybrid Remote, East London - DataLakes, PySpark, SQL, Azure, Python, AWS, Databricks, Agile
Role Overview
Role & Responsibilities
- Collaborate with Big Data Solution Architects to design, prototype, implement, and optimize data ingestion pipelines so that data is shared effectively across various business systems.
- Build ETL/ELT and Ingestion pipelines and design optimal data storage and analytics solutions using cloud and onperm technologies.
- Ensure the design, code and procedural aspects of the solution are production ready, in terms of operational, security and compliance standards.
- Participate in daytoday project and product delivery status meetings, and provide technical support for faster resolution of issues.
Skills and Experience
- Demonstrable
design & development experience and experience with
big data technologies like
Spark/Flink and Kafka - Proficient in
Python, PySpark, or Java/Scala.
Hands-on experience with some of the following technologies:
- Azure/AWS
- Data Lake Projects
- SQL
- ETL/ELT
- Databricks
- Spring/Guice or any other DI framework,
- RESTful Web Services.
- Proficient in querying and manipulating data from various DB (relational and big data).
- Experience of writing effective and maintainable unit and integration tests for ingestion pipelines.
- Experience of using static analysis and code quality tools and building CI/CD pipelines.
More jobs from Jefferson Frank
-
DevOps Engineer
Bristol, United Kingdom - 3 days ago
-
Java Backend Developer
Finland, United Kingdom - 2 days ago
-
Senior Frontend Developer
Reigate, Surrey, United Kingdom - 4 hours ago
-
Senior Frontend Developer
Reigate, Surrey, United Kingdom - 13 hours ago
-
Experienced Recruitment Consultant
Newcastle upon Tyne, United Kingdom - 1 week ago
-
Lead Full Stack Engineer, Hybrid
London, United Kingdom - 1 week ago