Senior Data Engineer - Edinburgh, United Kingdom - Methods
Description
Job Description
Senior Data Engineer (MongoDB / Postgres)
Methods are looking for a Senior Data Engineer to work on an initial 6 month programme of work with a client of ours.
The role will be completed with a hybrid way of working, with the successful candidate needing to be on site in the Edinburgh based office location.
The role has been confirmed to fall outside IR35.
Role Overview
Database Engineer with solid commercial database administration and development experience is required to join the Database Development (DD) team and accelerate our move from legacy on-prem databases to new AWS cloud-native solutions but also maintain and support the existing on-prem DB estate that runs on RHEL. This might be a developer with strong database design and administration skills, or vice-versa.
We operate multiple database technologies – MongoDB, Postgres, Aurora Postgres RDS, Oracle & Informix and are looking for candidates that can work across all of these applications with the team focus on providing DBAAS (Database as a Service) to multiple development teams. Database Development put a strong emphasis on using IaC tools and our on-prem estate is maintained by Ansible, with AWS deployments using AWS CDK. A key focus of the team is developing shared CDK constructs (Typescript) that are consumed by multiple development teams across the organisation for new and migrated databases, so experience with the AWS Developer toolset (CodePipeline, CodeBuild, CodeCommit and CloudFormation) will be useful.
The constructs will embody practical standards for database architecture and quality, allowing centralised assurance. In addition to the listed database technologies – we are currently involved in rolling out a new centralised backup solution using the CommVault application and for monitoring we utilise the TIG stack (Telegraf, InfluxDB & Grafana). The team Grafana instance is deployed using RedHat OpenShift – so experience with containers would be beneficial.
Main Objectives
Major version upgrade of the current on-prem MongoDB Estate
Major version upgrades to multiple on-prem Postgres databases
Support development teams with on-prem migrations to AWS RDS
-Help migrate our current on-prem Grafana instance to containers deployed on OpenShift
Support the Database team with ongoing BAU tasks such as upgrading, patching, automation, and monitoring improvements etc.
Help fixing the Support Requests raised by our stakeholders.
Additional Services
Collaborate with colleagues, to define, refine and prioritise the database roadmap and backlog.
Actively participate in all team events, leading where specialist knowledge is required and supporting the team to improve their process through inspection and adaptation.
Engage with the wider communities of practice and interest to share knowledge, technique, and experience.
Knowledge share with other team members as necessary.
Developing appropriate documentation and artefacts, e.g., data standards, local work instructions.
Support updating, installing, and testing new versions and upgrades to databases.
Troubleshooting and resolving database related issues.
Ensure existing on-prem estate continues to meet service standards until migration to AWS has been completed.
Ensure CDK constructs embody agreed standards for resiliency, security, and backups, and meet agreed standards for quality.
Support and deliver the disaster recover assurance of digital services, striving towards a sustainable Recovery Time Objective of 2hrs and Recovery Point objective of zero.
Essential Experience
Significant commercial experience with the following technology:
MongoDB
Postgres
AWS IAM, S3, EC2, RDS
Ansible
Typescript
CDK and AWS Development tools including Cloud Formation
SQL
Monitoring solutions (e.g., CloudWatch, Grafana)
Significant and demonstrable experience in the following areas:
Making excellent quality software with other people.
Working with business stakeholders, to understand the context and to focus on delivering value.
Design and implementation of solutions using a service-based and serverless architecture.
Excellent collaboration, communication, and teamwork skills
Experience working in the Agile delivery models - such as Scrum and/or Kanban frameworks.
Cloud database monitoring, telemetry, intelligence tools and processes
Defining infrastructure-as-code
Testing of disaster recovery procedures
Data replication and high-availability technology
Desirable Skills
Experience delivering new software that consumes services from a centralised database management team.
Commercial experience of the following:
Informix
Oracle
OpenSearch
Serverless (Lambda)