Lead Kafka Architect - London, United Kingdom - Next Ventures

    Next Ventures
    Next Ventures London, United Kingdom

    2 weeks ago

    Default job background
    Description

    Job Description

    Job Description:

    Primary responsibility: Lead Kafka Architect will design, perform POC where needed and develop enterprise's Apache Kafka Distributed Messaging and Integration Ecosystem.

    Solid experience and knowledge in the deployment of Kafka(Apache/Confluent

    Physical Deployment across multiple environments

    Optimization and Tuning based on performance metrics

    Setup best practices, Standards, Patterns Development and automate process for onboarding.

    Developed playbooks for troubleshooting and support teams

    Designed monitoring solutions and baseline statistics reporting to support the implementation

    Requirements

    Individual needs strong Java, SPARK and KAFKA experience.

    Experience with medium-to-large data pipelines: implementing, testing and deploying

    Experience with stream processing using Kafka

    Experience with automated unit and integration testing

    Experience with Kafka Connect.

    Champion

    good engineering practices and help teams to define and setup frameworks for "Big Data As A Service".

    technology strategy and engineering roadmaps around Big Data and

    execute strategic engineering proof of concepts.

    Develop monitoring strategies for infrastructure, platforms and applications aligning with enterprise strategy and overall industry trends.

    Big Data Technologies such as Hadoop, Spark, Kafka, etc.

    Hadoop: 5+ years

    Kafka: 3+ years

    Spark: 4+ years

    PySpark: 3+ years