This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
We're looking for an experienced and curious QA Analyst to lead the testing of our AI and GenAI-powered features. This role goes beyond traditional SaaS QA — it requires a mindset for dealing with uncertainty, edge cases, model behaviour, and testing for quality when outcomes are not always predictable. If you're passionate about both automation and understanding how intelligent systems behave in the wild, we want to hear from you. You’ll be embedded in a cross-functional scrum team, working closely with engineers, data scientists, and product managers to define and execute test strategies for AI features. You'll bring an automation-first mindset but be comfortable with exploratory and manual testing when required — especially when testing model-driven features. Your focus will include not just functional correctness, but also trust, safety, performance, and the overall quality of the AI experience.
Job Responsibility:
Design, develop, and maintain test plans and automated test suites for AI/GenAI features across frontend and backend
Test complex AI-driven systems where responses may vary — validate logic, relevance, completeness, tone, and safety
Build automation frameworks using Playwright and Cucumber with a focus on maintainability and scalability
Develop tools/scripts to validate model outputs and compare results across model versions (e.g. regression drift, hallucination checks)
Define and execute non-functional tests (e.g. performance under load, prompt injection resilience, fairness/bias testing)
Log and manage defects effectively
collaborate in root cause analysis and prevention
Actively contribute to CI/CD testing pipelines to ensure fast, reliable releases
Stay current on trends in AI/ML testing and suggest improvements to our QA practices
Requirements:
Proven experience testing AI/ML-powered systems, ideally including GenAI features (e.g. LLMs, chatbots, classifiers)
Strong proficiency with Playwright, Cucumber, and automation-first test strategies
Understanding of common GenAI testing challenges: non-determinism, bias, hallucinations, injection attacks
Experience with prompt testing, output validation, or model comparison tools
Familiarity with REST APIs, JSON, and integrating model endpoints into test frameworks
Inquisitive and detail-oriented mindset — able to challenge assumptions and uncover subtle issues
Experience in Agile/Scrum environments and collaborative cross-functional teams
Familiarity with CI/CD tooling (e.g. Azure DevOps, GitHub Actions, or similar)
Strong written and verbal communication skills
Nice to have:
Background in machine learning or data science
Experience testing ethical/guardrail functionality in AI systems
Knowledge of data labelling, synthetic data generation, or test data management in AI contexts
Familiarity with tools like OpenAI’s evals, LangChain testing modules, or similar frameworks
What we offer:
Share Options (EMI) scheme
25 days annual leave, plus flexible bank holidays and the opportunity to buy additional days