This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
We're seeking a technically-focused Data Analyst - QA role to drive the Quality of our AI features and various LLM models. This role involves evaluating AI outputs, identifying and tagging patterns in errors, and ensuring that responses align with accuracy, coherence, and ethical guidelines.
Job Responsibility:
Triage Bad Query responses and categorisation of the issues found
Analyse win/loss patterns across different LLM Models and AI features
Conduct qualitative analysis to detect inconsistencies, biases, retrieval errors, or hallucinations in AI outputs
Develop quality metrics and benchmarks to measure AI response performance over time
Collaborate with data scientists and engineers to provide feedback and improve AI models
Identify trends and recurring issues in AI responses and suggest refinements
Design and maintain quality assurance guidelines for AI-generated content evaluation
Perform manual audits on AI outputs to ensure compliance with linguistic and ethical standards
Requirements:
Bachelor's or Master's degree in Data Science, Computer Science Engineering, AI/ML Engineering, Information Technology, Compute Applications
3 -5 years of experience in quality assurance, data analysis, content moderation, or NLP-related roles
Excellent vocabulary and communication skills
Experience in Search Engine Optimization (SEO) or analysing AI response quality
Strong ability to analyze qualitative data, identify patterns, and synthesize insights
Ability to communicate findings effectively to both technical and non-technical stakeholders
Excellent attention to detail and critical thinking skills
Experience working with AI-generated content, NLP models, or chatbots
Nice to have:
Familiarity with data annotation tools and evaluation frameworks for AI models
Familiarity with qualitative research methodologies and user behavior analysis
Basic knowledge on SQL retrieval
Experience working on JIRA is a plus
Ability to work in a fast-paced environment and adapt to evolving AI technologies