Senior Test Engineer- AI Testing Job at Cyclad Sp. z o.o. (Rabat)

Job Responsibility

Design and execute test strategies for machine learning models (classification, regression, NLP, CV, etc.)
Validate model accuracy, fairness, bias detection, explainability, and robustness under varying datasets
Perform adversarial testing and edge-case validation for AI systems
Test autonomous agents, multi-step reasoning paths, and state transitions
Validate correctness, grounding, consistency, and safety of LLM outputs
Evaluate prompt robustness and behavioral variations across scenarios
Validate retrieval accuracy, grounding quality, and hallucination reduction
Test vector store behavior, document chunking logic, and retriever configurations
Validate fallback behaviors when tools or external services fail
Execute adversarial, prompt-injection, and red team style testing
Validate compliance with data privacy, business rules, and insurance guidelines
Benchmark LLM latency, throughput, and multi-agent performance
Validate concurrency handling, degradation logic, and retry mechanisms
Document test cases, behavior maps, prompt variations, and evaluation reports
Work closely with AI engineers and SMEs to refine agent workflows
Conduct API testing for AI services and model endpoints
Validate performance, scalability, and latency of AI inference services
Perform security testing for data pipelines and model-serving endpoints
Ensure compliance with AI ethics, data privacy, and regulatory standards
Establish KPIs for model quality (precision, recall, F1-score, drift detection)
Document test strategies, results, and provide audit-ready evidence
Work closely with data scientists, ML engineers, and DevOps teams to integrate testing into the AI lifecycle
Mentor junior testers and contribute to building reusable test accelerators
Represent the testing function in client discussions, providing insights on AI quality assurance
Develop automated test frameworks for AI/ML pipelines
Implement synthetic data generation for edge-case testing
Conduct bias and fairness testing across diverse datasets
Validate monitoring dashboards for model drift and performance degradation
Perform regression testing for retrained models
Create reusable test assets for AI projects (scripts, datasets, frameworks)
Provide detailed defect analysis and SLA-driven closure

Requirements

Experience: 6 - 8 years
Strong experience in software testing (manual + automation) with focus on AI systems
Hands-on expertise in Python for test automation, data validation, and AI testing scripts
Knowledge of Gen AI frameworks (Langchain, Langgraph, CrewAI)
Familiarity with CI/CD pipelines (Jenkins, GitHub Actions, GitLab CI)
Strong understanding of data validation, model evaluation metrics, and drift detection
API testing experience (REST, GraphQL) with tools like Postman or REST Assured
Solid knowledge of cloud platforms (AWS, Azure, GCP) for AI deployments
Exposure to AI ethics and compliance testing
Knowledge of containerization & orchestration (Docker, Kubernetes)
Experience with Big Data ecosystems (Spark, Hadoop) for large-scale AI testing
Familiarity with observability tools (Prometheus, Grafana, ELK stack)
Understanding of security testing for AI pipelines (data poisoning, adversarial attacks)

Cyclad Sp. z o.o. - All Job Offers

Select Country

Senior Test Engineer- AI Testing

Job Responsibility

Requirements

Looking for more opportunities?

Senior Test Engineer- AI Testing

Senior Staff Software AI Test Engineer- Prisma SASE

Senior Software Development Engineer in Test (SDET) - AI Cluster Networking and Security

Senior Software Development Engineer in Test (SDET) - AI Cluster

Senior Lead AI Engineer, AI Foundations

Senior Ai Engineer (Agentic & Generative Ai)

Senior AI Engineer - Teams Messaging AI

Senior Lead AI Engineer (Gen AI Platform Services)

Senior Lead AI Engineer (Gen AI Platform Services)

Our AI answers in your language