Senior Engineer, Data Engineering - London, United Kingdom - Algolia

Description

We're on a mission to make Algolia a data-driven organisation, and we're looking for a Senior Data Engineer to join our Data team and help us get there.

Our Data team aims at helping other teams make better decisions by providing relevant data while ensuring its integrity and its consistency.

This team has a significant impact as we work closely with business analysts on our Operations, Marketing, Product and Infrastructure teams to help them discover meaningful trends.

Our data team is composed of a Data Engineering team (building pipelines and scaling the Lake & Warehouse), an Analytics Engineering team (building standardised data models) and Data Analysts.

As a member of the Data Engineering team , your daily responsibilities will include:

Developing data pipelines and ETL workflows mainly with Python, Airflow (AWS MWAA), Spark (AWS Glue, EMR), other AWS services managed with Terraform.

Improving and maintaining our data warehouse (AWS Redshift).

Support the Analytics Engineering team to build data models with dbt to be used by Analysts on business reports.

Interacting with Engineering and Business teams to understand requirements.

We maintain an open-minded approach and eagerly embrace experimentation with new technologies to achieve our goals and we try to continuously challenge our choices on that topic.

You will have the opportunity to make some decisions going in that direction.

As a senior joining the team, your goal will also be to mentor junior engineers and help them grow to be successful at Algolia.

With Algolia's rapid growth, there will be many data-related challenges that will need to be tackled
Are you ready for the challenge?
Key responsibilities...
Design, build, enrich, scale up our data pipelines.
Work with engineers, data analysts, business analysts to capture and model data.
Monitor data integrity and growth.
Help improve our data lake and data warehouse architecture, to increase performance, simplicity and autonomy of the users.
Help analysts industrialise reports and dashboards to improve company productivity.
Ensure every product is released with a data-driven approach.
You might be a good fit if you have ...
Strong experience designing and building data pipelines.
Experience orchestrating pipelines with Airflow.
Professional knowledge of Python.
Strong experience working with cloud platforms and architecting them.
Strong experience with data warehouses.
Experience about infrastructure topics specific to data engineering.
Interest in Big Data challenges (~600Tb of data, +14Tb per week, ~700M files, +1,2M per week, ~300 dbt models, ~70 Airflow DAGs)
Interest in understanding the data and business requirements.
Excellent spoken and written English skills.
You are humble, curious, proactive, and strike a balance between creativity, resourcefulness, and pragmatism.
Nice to have...
Experience using and managing AWS Redshift.
Experience with a parallel data processing framework such as Apache Spark.
Experience analysing data quality using dbt.

Experience at our current stage and beyond ($50-200M ARR range, high growth, lots of change and building internal infrastructure).

#LI-Remote
#J-18808-Ljbffr