Data Engineer - London, United Kingdom - Salt Search

Salt Search

Verified Company

London, United Kingdom

1 week ago

Posted by:

Tom O´Connor

beBee Recruiter

Description

One of my key clients is looking for a Data Engineer to join the team at a very exciting time.

Working closely with the 1st line of defence to ensure that the firm's collection strategy and forbearance solution are effective.

Data Engineer Responsibilities/Experience:

Work as part of a Data Engineering team in data department
In depth working knowledge of JSON structures with exposure to Java, python, XML.
Experienced in working with AWS resources including Athena, Quicksight, Redshift, RDS, Lambda, EC2, Kinesis, etc.
Experience with AWS orchestration, data engineering, and serverless compute services
Experience designing, developing and delivering data engineering pipelines on AWS platforms for data science and machine learning products
Indepth knowledge for Machine Learning, Analytics/BI platforms, Metadata/Governance tooling, stream processing, ELT/ETL Data Integration.
Skilled in extracting, transforming, and loading data from a number of sources eg ETL using Glue, Lambda
Solid knowledge and experience of NoSQL, RDBMS, object stores and SQL
Knowledge of software solutions (Databricks, Snowflake, Hive, SQL, Apache)
Interface with other technology teams to extract, transform, and load data from a wide variety of data sources.
Optimization, and continually improve ongoing analysis processes, automating or simplifying selfservice support for customers.
Extensive experience using Python and similar language for data processing and analysis
Good knowledge and experience of Python and associated data frameworks (PySpark, Pandas, numpy)
Experience in metadata and indepth working knowledge in Complex data environment
Must have worked in Banking and finance data and full understanding of handling, mapping and analysing transactional data.
Must have experience in complex and unstructured data including multilayered, nested data.
Experience with DevOps/DataOps practices and tooling (eg: product releases, versioning, testing and monitoring)
Understanding of Agile principles and frameworks and experience in productoriented development
Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, and operations.
Design patterns for eventbased and microservices architectures
Github experience is desirable
Open source project contributions are also desirable
Proven success in communicating with business users, other technical teams, and senior management.