- Design and implement comprehensive testing strategies for GenAI features including conversational AI, agentic systems, and LLM-powered workflows
- Develop automated test suites for prompt testing, including regression tests that detect unintended changes in model behaviour
- Create evaluation frameworks to measure GenAI quality across multiple dimensions (accuracy, relevance, safety, consistency, latency)
- Build and maintain test datasets and golden examples that represent diverse user scenarios and edge cases
- Implement monitoring and alerting systems to detect quality degradation in production GenAI features
- Perform adversarial testing to identify potential failures, hallucinations, biases, or security vulnerabilities in AI systems
- Develop tools and frameworks that make it easy for engineers to test their GenAI implementations
- Conduct user acceptance testing and gather feedback on AI feature performance from internal users
- Document testing procedures, known issues, and quality metrics in clear, accessible formats
- Strong QA or test engineering experience, preferably with AI/ML systems but not essential.
- Understanding of GenAI technologies including LLMs, prompt engineering, and AI application patterns
- Knowledge of software testing methodologies (functional, integration, regression, performance, security testing)
- Ability to design test cases and evaluation criteria for non‑deterministic systems
- Strong analytical and problem‑solving skills with attention to detail
- Experience with API testing tools (Postman, REST Assured) and backend testing
- Familiarity with CI/CD pipelines and automated testing integration
- Excellent communication skills for documenting issues and collaboration
- Extend offers to candidates without an extensive interview process with a member of our recruitment team and a hiring manager via video or in person.
- Send job offers via email. All offers are first extended verbally by a member of our internal recruitment team whenever possible and then followed up via written communication.
-
Software Engineering
1 month ago
Only for registered members ManchesterFitch Ratings is currently seeking a Associate Director of Software Engineering - Full-Stack Applications based out of our Manchester office.This role will oversee the strategic direction, development, and delivery of our Java, REACT, and Python-based applications with a strong e ...
-
Software Engineer
1 month ago
Only for registered members ManchesterWe're looking for an enthusiastic and experienced Android Software Engineer to join our growing Engineering team based in Wigan.This is an exciting opportunity to work on customer-facing mobile products that are central to the Tote consumer experience. · You'll be part of a colla ...
-
Software Engineering
1 month ago
Only for registered members Manchester Full time+p>Fitch Ratings es un proveedor líder de calificaciones crediticias, comentarios y investigación. Estamos buscando a un Líder de equipo de desarrollo de software basado en Manchester. Esta función supervisará la dirección estratégica, el desarrollo y la entrega de nuestras aplic ...
-
Software Engineer
4 weeks ago
Only for registered members ManchesterWe are seeking a talented Software Engineer with experience in web applications to join our growing development team. · ...
-
Software Engineer
1 month ago
Only for registered members ManchesterAs a Software Engineer, you will be motivated in becoming a Full Stack Software Engineer. You'll enjoy delivering mission critical solutions to National Security customers. · ...
-
Software Engineer
1 month ago
Only for registered members ManchesterWe are looking for a talented and motivated Backend Software Engineer to join our Engineering team in Wigan. · ...
-
Software Engineer
1 month ago
Only for registered members ManchesterThe Role · We're looking for a Software Engineer who enjoys building reliable, scalable systems and wants real ownership of their work. · ...
-
Software Engineering
1 month ago
Only for registered members ManchesterFitch Ratings is currently seeking a Associate Director of Software Engineering - Full-Stack Applications based out of our Manchester office. · ...
-
Software Engineer
4 weeks ago
Only for registered members ManchesterWe are looking for a Software Engineer to join our Real Time Pricing (RTP) technology team. · ...
-
Software Engineer
2 weeks ago
Only for registered members ManchesterWe're looking for a Software Engineer with strong experience in Python, FastAPI, and React to help us build reliable, · scalable products.Building and maintaining backend services using Python and FastAPI · Developing responsive, · user-friendly interfaces with React · Designing ...
-
Software Engineer
2 weeks ago
Only for registered members ManchesterWe are a collaborative engineering-led technology team building scalable high-performance software used by a broad and growing user base. Quality automation and continuous improvement sit at the heart of how we work. · ...
-
Software Engineer
1 week ago
Only for registered members Manchester Full timeWe're On the Beach Since 2003 we've been rewriting the rules of how people discover book and experience their perfect getaway. · About The Role As a Software Engineer Level 1 at On The Beach you'll help build products that anticipates customer needs simplify complexity and make h ...
-
Software Engineer
3 weeks ago
Only for registered members Manchester £40,000 - £95,000 (GBP)We make software that makes a difference. · Your skills and experience: · ...
-
Software Engineer
2 days ago
Only for registered members ManchesterGreat opportunity for a Software Engineer to join a private equity backed tech company in Manchester as they scale in · 2026. · You'd join a product-based squad within a larger engineering function and would be given the opportunity to be involved in all aspects of the product y ...
-
Software Engineer
1 month ago
Only for registered members ManchesterWe're looking for a Software Engineer with strong Angular experience to join a growing product-focused team building predominantly application-based solutions. · Commercial experience as a Software Engineer / Angular Developer · Strong knowledge of Angular and TypeScript · ...
-
Software Engineering
1 month ago
Only for registered members Manchester, EnglandFitch Group is seeking a Associate Director of Software Engineering - Full-Stack Applications based out of our Manchester office. This role will oversee the strategic direction, development, and delivery of our Java, REACT, and Python-based applications with a strong emphasis on ...
-
Software Engineer
1 month ago
Only for registered members ManchesterThe Opportunity · Join a major UK financial institution undergoing a huge digital transformation,With over 20 million digital users,the engineering work you'll contribute to will directly impact customer experience at national scale. · ...
-
Software Engineer
2 weeks ago
Only for registered members ManchesterWe're On the Beach Since 2003 we've been rewriting the rules of how people discover book and experience their perfect getaway. · Build and enhance scalable backend services and applications that process large volumes of data. · Develop and maintain services with a focus on perfor ...
-
Software Engineer
4 weeks ago
Only for registered members ManchesterGreat opportunity for a Software Engineer to join a private equity backed tech company in Manchester as they scale in 2025. · You'd join a product-based squad within a larger engineering function and would be given the opportunity to be involved in all aspects of the product you ...
-
Software Engineer
1 month ago
Only for registered members ManchesterWe're looking for an experienced Full-Stack Engineer to help redefine our core analytical platforms. · Develop end-to-end features using Java, React, and Python. · Take the lead on complex data migrations and high-performance database design (PostgreSQL). · ...
-
Software Engineer
1 week ago
Only for registered members ManchesterWe're On the Beach Since 2003 we've been rewriting the rules of how people discover book and experience their perfect getaway. · ...
Senior Manager, Software Engineering - Manchester - Anaplan Inc
Description
Location
Manchester, United Kingdom
About the Role
We're pioneering a new role focused exclusively on quality assurance for GenAI systems. As our AI QA Engineer, you'll develop testing strategies, evaluation frameworks, and quality metrics specifically designed for LLM-powered applications. This role requires a unique blend of QA expertise, understanding of GenAI behaviour, and automation skills to ensure our AI features are reliable, accurate, and trustworthy.
Your Impact
Your Qualifications
Our Commitment to Diversity, Equity, Inclusionand Belonging (DEIB)
We believe attracting and retaining the best talent and fostering an inclusive culture strengthens our business. DEIB improves our workforce, enhances trust with our partners and customers, and drives business success. Build your career in a place where diversity, equity, inclusion and belonging aren't just words on paper – this is what drives our innovation, it's how we connect, and it contributes to what makes us a market leader. We believe in a hiring and working environment where all people are respected and valued, regardless of gender identity or expression, sexual orientation, religion, ethnicity, age, neurodiversity, disability status, citizenship, or any other aspect which makes people unique. We hire you for who you are, and we want you to bring your authentic self to work every day We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform essential job functions, and receive equitable benefits and all privileges of employment. Please contact us to request accommodation.
Fraud Recruitment Disclaimer
It has come to our attention that fraudulent and fictitious job opportunities are being circulated on the Internet. Prospective candidates are being contacted by certain individuals, mainly through telephone calls, emails and correspondence, claiming they are representatives of Anaplan. The main purpose of these correspondences and announcements is to obtain privileged information from individuals.
All emails from Anaplan would come from an email address. Should you have any doubts about the authenticity of an email, letter or telephone communication purportedly from, for, or on behalf of Anaplan, please send an email to before taking any further action in relation to the correspondence.
#J-18808-Ljbffr
-
Software Engineering
Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester
-
Software Engineering
Full time Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester
-
Software Engineering
Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester
-
Software Engineer
Full time Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester
-
Software Engineering
Only for registered members Manchester, England
-
Software Engineer
Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester
-
Software Engineer
Only for registered members Manchester