CrawlJobs Logo

Senior Test Engineer- AI Testing

cyclad.pl Logo

Cyclad Sp. z o.o.

Location Icon

Location:
Morocco , Rabat

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided
Save Job
Save Icon
Apply Position

Job Responsibility:

  • Design and execute test strategies for machine learning models (classification, regression, NLP, CV, etc.)
  • Validate model accuracy, fairness, bias detection, explainability, and robustness under varying datasets
  • Perform adversarial testing and edge-case validation for AI systems
  • Test autonomous agents, multi-step reasoning paths, and state transitions
  • Validate correctness, grounding, consistency, and safety of LLM outputs
  • Evaluate prompt robustness and behavioral variations across scenarios
  • Validate retrieval accuracy, grounding quality, and hallucination reduction
  • Test vector store behavior, document chunking logic, and retriever configurations
  • Validate fallback behaviors when tools or external services fail
  • Execute adversarial, prompt-injection, and red team style testing
  • Validate compliance with data privacy, business rules, and insurance guidelines
  • Benchmark LLM latency, throughput, and multi-agent performance
  • Validate concurrency handling, degradation logic, and retry mechanisms
  • Document test cases, behavior maps, prompt variations, and evaluation reports
  • Work closely with AI engineers and SMEs to refine agent workflows
  • Conduct API testing for AI services and model endpoints
  • Validate performance, scalability, and latency of AI inference services
  • Perform security testing for data pipelines and model-serving endpoints
  • Ensure compliance with AI ethics, data privacy, and regulatory standards
  • Establish KPIs for model quality (precision, recall, F1-score, drift detection)
  • Document test strategies, results, and provide audit-ready evidence
  • Work closely with data scientists, ML engineers, and DevOps teams to integrate testing into the AI lifecycle
  • Mentor junior testers and contribute to building reusable test accelerators
  • Represent the testing function in client discussions, providing insights on AI quality assurance
  • Develop automated test frameworks for AI/ML pipelines
  • Implement synthetic data generation for edge-case testing
  • Conduct bias and fairness testing across diverse datasets
  • Validate monitoring dashboards for model drift and performance degradation
  • Perform regression testing for retrained models
  • Create reusable test assets for AI projects (scripts, datasets, frameworks)
  • Provide detailed defect analysis and SLA-driven closure

Requirements:

  • Experience: 6 - 8 years
  • Strong experience in software testing (manual + automation) with focus on AI systems
  • Hands-on expertise in Python for test automation, data validation, and AI testing scripts
  • Knowledge of Gen AI frameworks (Langchain, Langgraph, CrewAI)
  • Familiarity with CI/CD pipelines (Jenkins, GitHub Actions, GitLab CI)
  • Strong understanding of data validation, model evaluation metrics, and drift detection
  • API testing experience (REST, GraphQL) with tools like Postman or REST Assured
  • Solid knowledge of cloud platforms (AWS, Azure, GCP) for AI deployments
  • Exposure to AI ethics and compliance testing
  • Knowledge of containerization & orchestration (Docker, Kubernetes)
  • Experience with Big Data ecosystems (Spark, Hadoop) for large-scale AI testing
  • Familiarity with observability tools (Prometheus, Grafana, ELK stack)
  • Understanding of security testing for AI pipelines (data poisoning, adversarial attacks)

Additional Information:

Job Posted:
March 22, 2026

Icon
Cyclad Sp. z o.o. - All Job Offers
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Senior Test Engineer- AI Testing

Senior AI Engineer

We are seeking an innovative AI Engineer to join a brand new team focused on pro...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven experience as an AI Engineer with a significant delivery history
  • Strong expertise in multiple programming languages & frameworks
  • Experience and proven experience in using quantitative testing practice applied to the field of AI/ML for actionable Go/No-Go decisions of delivering software to production
  • Demonstrated expertise of developing on a range of architectures, ideally up to and including container-based micro-services with focus on scalability, reliability, maintainability, and high performance
  • Good understanding of SQL and NoSQL databases
  • Excellent communication and collaboration skills
  • A growth mindset and willingness to learn and adapt in a fast-paced environment
  • Passion about site reliability engineering and its impact on product development
  • Being connected to latest technologies, like Generative AI, and keen to put them in practice.
Job Responsibility
Job Responsibility
  • Understand the landscape, tooling and procedures used by developers at Citi and look for opportunities to reduce toil and aid simplification using Gen AI based solutions
  • Apply classic AI and novel Gen AI evaluation methodology to raise the quality and reliability bar for the software that you will deliver, as well to manage and mitigate risks that are specific/inherent to this field
  • Advice on Evaluation metrics, devise and implement Quantitative Testing Plans, and help evolve the existing approaches to AI evaluation
  • Work with a wide variety of Citi technology teams and help them drive towards everything-as-code and a codified controls environment
  • Collaborate with product and engineering teams to design, build and maintain scalable and reliable web applications and services
  • Be hands-on with coding and software design to ensure adherence to high quality standards and best practices
  • Mentor and nurture other engineers to help them grow their skills and expertise
  • Support and drive cultural change, including instigating critical thinking about controls and processes and encouraging a culture of continuous improvement.
What we offer
What we offer
  • 27 days annual leave (plus bank holidays)
  • A discretional annual performance-related bonus
  • Private Medical Care & Life Insurance
  • Employee Assistance Program
  • Pension Plan
  • Paid Parental Leave
  • Special discounts for employees, family, and friends
  • Access to an array of learning and development resources.
  • Fulltime
Read More
Arrow Right

Senior AI Engineer

As a Senior AI Engineer on our AI Engineering team, you will be responsible for ...
Location
Location
Canada; United States
Salary
Salary:
160000.00 - 260000.00 USD / Year
apollo.io Logo
Apollo.io
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of software engineering experience with a focus on production systems
  • 1.5+ years of hands-on LLM experience (2023-present) building real applications with GPT, Claude, Llama, or other modern LLMs
  • Production LLM Applications: Demonstrated experience building customer-facing, scalable LLM-powered products with real user usage (not just POCs or internal tools)
  • Agent Development: Experience building multi-step AI agents, LLM chaining, and complex workflow automation
  • Prompt Engineering Expertise: Deep understanding of prompting strategies, few-shot learning, chain-of-thought reasoning, and prompt optimization techniques
  • Python Proficiency: Expert-level Python skills for production AI systems
  • Backend Engineering: Strong experience building scalable backend systems, APIs, and distributed architectures
  • LangChain or Similar Frameworks: Experience with LangChain, LlamaIndex, or other LLM application frameworks
  • API Integration: Proven ability to integrate multiple APIs and services to create advanced AI capabilities
  • Production Deployment: Experience deploying and managing AI models in cloud environments (AWS, GCP, Azure)
Job Responsibility
Job Responsibility
  • Design and Deploy Production LLM Systems: Build scalable, reliable AI systems that serve millions of users with high availability and performance requirements
  • Agent Development: Create sophisticated AI agents that can chain multiple LLM calls, integrate with external APIs, and maintain state across complex workflows
  • Prompt Engineering Excellence: Develop and optimize prompting strategies, understand trade-offs between prompt engineering vs fine-tuning, and implement advanced prompting techniques
  • System Integration: Build robust APIs and integrate AI capabilities with existing Apollo infrastructure and external services
  • Evaluation & Quality Assurance: Implement comprehensive evaluation frameworks, A/B testing, and monitoring systems to ensure AI systems meet accuracy, safety, and reliability standards
  • Performance Optimization: Optimize for cost, latency, and scalability across different LLM providers and deployment scenarios
  • Cross-functional Collaboration: Work closely with product teams, backend engineers, and stakeholders to translate business requirements into technical AI solutions
What we offer
What we offer
  • equity
  • company bonus or sales commissions/bonuses
  • 401(k) plan
  • at least 10 paid holidays per year, flex PTO, and parental leave
  • employee assistance program and wellbeing benefits
  • global travel coverage
  • life/AD&D/STD/LTD insurance
  • FSA/HSA and medical, dental, and vision benefits
  • Fulltime
Read More
Arrow Right

Senior AI Engineer

Elsewhen, a London-based consultancy, designs and builds technology solutions fo...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
elsewhen.com Logo
Elsewhen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Professional AI engineering experience
  • Background in Software Engineering with Python
  • Solid understanding of the Python standard library and modern Python coding, testing, debugging and automation techniques
  • Hands-on experience building solutions using LLMs and Agentic architectures with ADK, LlamaIndex, or LangGraph
  • Working with vector databases for embedding and indexing
  • Strong experience with cloud platforms
  • Strong experience with API design and frameworks like FastAPI or Flask
  • Solid experience with relational databases and SQL
  • Interest in expanding your knowledge into GenAI and machine learning
  • Excellent communication skills and the ability to work well in a collaborative team environment
Job Responsibility
Job Responsibility
  • Experiment with POCs to find solutions for real-world problems using Large Language Models
  • Collaborate on AI-driven projects, working alongside engineers, product managers and AI specialists while maintaining clear documentation
  • Build and deploy Agentic LLM-based solutions with LangGraph
  • Familiar with different multi agent system patterns
  • Build and deploy LLM-based solutions using RAG
  • Familiar with different types of databases: Relational, Graph etc
  • Design and optimise APIs using Python and FastAPI to serve AI solutions
  • Familiar with GCP ecosystem and Cloudrun
  • Build and optimise data pipelines for vector search and knowledge retrieval using Vector databases and embedding models
What we offer
What we offer
  • Private Health Insurance: Comprehensive coverage for both physical and mental health
  • Flexible and Remote-First Work Environment: Choose how and where you work, with the option for weekly team meet-ups in central London
  • Generous Leave Policy: 27 days of holiday plus bank holidays
  • Family-friendly policies, including enhanced maternity, paternity and shared
  • Learning and Development: Individual annual budget of £2,000 for learning and development, with dedicated learning days
  • Feel Better Fund: £500 to help set up your remote office
  • Social Events: Monthly and quarterly team events, an annual team trip, and half-yearly social events
  • Gym Membership Contribution: Support for maintaining your physical health
  • Pension Contribution: Enhanced employer pension contribution of 6%
  • Bonus Opportunities: Potential to receive a discretionary (non-contractual) bonus based on business and personal achievements
Read More
Arrow Right

Senior Software Test Engineer

You are passionate about building rock solid software. Now is your chance to do ...
Location
Location
Germany
Salary
Salary:
80000.00 - 90000.00 EUR / Year
linuxrecruit.co.uk Logo
Linux Recruit
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong background in software engineering
  • Ideally with a JVM based language
  • Ability to both write and evaluate clean, maintainable code
  • Solid understanding of modern testing and QA practices and frameworks
  • Experience in automation
  • Real curiosity about AI tools and their testing challenges
  • Collaborative mindset
Job Responsibility
Job Responsibility
  • Take ownership of the end to end of software testing projects within a platform team
  • Support dozens of other engineering teams across the organisation
  • Bring software engineering excellence into testing
  • Work with backend languages such as Kotlin, Java, or Go
  • Drive full automation across the stack
  • Collaborate closely with product managers and engineering leaders
  • Shape solutions that will have an impact far beyond your own team
  • Fulltime
Read More
Arrow Right

Senior AI Engineer

We are building an AI platform that accelerates the Software Development Lifecyc...
Location
Location
Uzbekistan , Tashkent
Salary
Salary:
Not provided
ventionteams.com Logo
Vention
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience in software development (backend or full-stack)
  • Strong proficiency in .NET and/or Python. Ability to connect classical software engineering practices with modern AI capabilities
  • Experience with multi-agent orchestration frameworks (LangChain, CrewAI, AutoGen, or similar tools used to build and orchestrate AI agents)
  • Hands-on experience with AI-assisted development tools (Cursor, GitHub Copilot, OpenAI Codex, and other AI solutions that enhance productivity and code quality)
  • Experience embedding AI into coding, testing, documentation, and review processes to automate and accelerate delivery
  • Expertise in automation and workflow design
Job Responsibility
Job Responsibility
  • Design and implement AI-driven workflows across the SDLC
  • Enhance documentation processes
  • Accelerate development with AI-assisted coding
  • Improve PR analysis and code review
  • Optimize test documentation and testing workflows
  • Support deployment workflows
  • Enhance support and operations
  • Develop and orchestrate AI agents
What we offer
What we offer
  • EDU corporate community (300+ members): tech communities, interest clubs, events, a small R&D lab, a knowledge base, and a dedicated AI track
  • Licenses for AI tools: GitHub Copilot, Cursor, and others
  • Expanded medical support for employees in Tashkent
  • 19 working days of vacation per year, 21 after two years in the company
  • Corporate getaway & teambuilding activities
  • Support for the significant events in your life
  • Referral bonuses for bringing in new talent
  • Fulltime
Read More
Arrow Right

Senior AI Engineer

As a Senior AI Engineer on our AI Engineering team, you will be responsible for ...
Location
Location
India
Salary
Salary:
Not provided
apollo.io Logo
Apollo.io
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7+ years of software engineering experience with a focus on production systems
  • 1.5+ years of hands-on LLM experience (2023-present) building real applications with GPT, Claude, Llama, or other modern LLMs
  • Demonstrated experience building customer-facing, scalable LLM-powered products with real user usage (not just POCs or internal tools)
  • Experience building multi-step AI agents, LLM chaining, and complex workflow automation
  • Deep understanding of prompting strategies, few-shot learning, chain-of-thought reasoning, and prompt optimization techniques
  • Expert-level Python skills for production AI systems
  • Strong experience building scalable backend systems, APIs, and distributed architectures
  • Experience with LangChain, LlamaIndex, or other LLM application frameworks
  • Proven ability to integrate multiple APIs and services to create advanced AI capabilities
  • Experience deploying and managing AI models in cloud environments (AWS, GCP, Azure)
Job Responsibility
Job Responsibility
  • Design and Deploy Production LLM Systems: Build scalable, reliable AI systems that serve millions of users with high availability and performance requirements
  • Agent Development: Create sophisticated AI agents that can chain multiple LLM calls, integrate with external APIs, and maintain state across complex workflows
  • Prompt Engineering Excellence: Develop and optimize prompting strategies, understand trade-offs between prompt engineering vs fine-tuning, and implement advanced prompting techniques
  • System Integration: Build robust APIs and integrate AI capabilities with existing Apollo infrastructure and external services
  • Evaluation & Quality Assurance: Implement comprehensive evaluation frameworks, A/B testing, and monitoring systems to ensure AI systems meet accuracy, safety, and reliability standards
  • Performance Optimization: Optimize for cost, latency, and scalability across different LLM providers and deployment scenarios
  • Cross-functional Collaboration: Work closely with product teams, backend engineers, and stakeholders to translate business requirements into technical AI solutions
What we offer
What we offer
  • Invest deeply in your growth, ensuring you have the resources, support, and autonomy to own your role and make a real impact
  • Collaboration is at our core—we’re all for one, meaning you’ll have a team across departments ready to help you succeed
  • We encourage bold ideas and courageous action, giving you the freedom to experiment, take smart risks, and drive big wins
Read More
Arrow Right

Senior Engineering Manager - AI

We are seeking a Senior Engineering Manager (Level 5) to lead a high-performing ...
Location
Location
India , Chennai
Salary
Salary:
Not provided
arcadia.com Logo
Arcadia
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 15+ years of professional experience in software engineering
  • At least 4+ years in engineering leadership roles
  • Strong technical background in AI/ML systems, large-scale data pipelines, and cloud-native platforms
  • Hands-on experience with Python (preferred), modern ML frameworks (PyTorch/TensorFlow), and cloud services (AWS)
  • Proven success in managing teams of 4–6 engineers, scaling processes, and building diverse, high-performance teams
  • Strong architectural design and system-thinking abilities
  • Excellent communication skills with ability to influence cross-functional stakeholders
  • Passion for sustainability, decarbonization, and using technology to create positive climate impact
  • Experienced with building agentic pipelines with the latest models from Anthropic, Google, OpenAI, and more
Job Responsibility
Job Responsibility
  • Lead and grow a team of engineers focused on building AI-driven and data-intensive systems for the Arcadia platform
  • Design and train ML/AI models (forecasting, NLP, graph learning, generative AI) to improve data quality, cost effectiveness, and system scalability
  • Build true agentic workflows with multi-step processing incorporating RAG pipelines and MCPs
  • Balance management responsibilities (hiring, coaching, performance reviews, career growth) with technical leadership (architecture, system design, technical strategy)
  • Drive end-to-end delivery of complex projects in partnership with Product, Data, and Infrastructure teams
  • Guide the adoption of modern AI/ML technologies, ensuring practical, scalable use in production
  • Foster a culture of high performance, ownership, and technical excellence
  • Establish engineering best practices in testing, observability, reliability, and CI/CD
  • Partner with leadership to define roadmaps, set priorities, and align execution with Arcadia’s strategic goals
  • Represent AI across the company, articulating technical trade-offs and championing innovation
What we offer
What we offer
  • Competitive compensation and employee stock options
  • Hybrid/remote-first working model (India-based role, with global collaboration)
  • Flexible leave policy
  • Comprehensive medical insurance (self + family members)
  • Annual performance cycle + quarterly recognition awards
  • A supportive, diverse engineering culture grounded in empathy, teamwork, and innovation
  • Fulltime
Read More
Arrow Right

Senior Platform Engineer, AI Evaluation

We’re looking for an AI Platform Engineer to evolve and extend our internal eval...
Location
Location
United States , Mountain View
Salary
Salary:
137871.00 - 172339.00 USD / Year
khanacademy.org Logo
Khan Academy
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field
  • 5 years of Software Engineering experience with 2+ of those years working on the evaluation of generative AI systems
  • Strong programming skills in Go, Python, SQL, and at least one data pipeline framework (e.g., Airflow, Dagster, Prefect)
  • Familiarity with the architecture of large language models and their industry-standard APIs
Job Responsibility
Job Responsibility
  • Evolve and extend our internal evaluation framework for assessing the quality of our AI-driven experiences
  • Work closely with ML data engineers and platform developers to help internal teams adopt an eval-driven development process incorporating offline benchmark tests and online experiments
  • Gather internal requirements, getting buy-in for changes, and then developing documentation and training materials
What we offer
What we offer
  • Competitive salaries
  • Ample paid time off as needed
  • 8 pre-scheduled Wellness Days in 2026
  • Remote-first culture
  • Generous parental leave
  • 401(k) + 4% matching
  • Comprehensive insurance, including medical, dental, vision, and life
  • Fulltime
Read More
Arrow Right
Jobs by Category
Art and Entertainment Jobs Banking Jobs Call Center Jobs Construction Jobs Consulting Jobs Customer Service Jobs Education Jobs Energy Jobs Finance Jobs Gastronomy Jobs Health and Beauty Jobs Hospitality and Tourism Jobs Human Resources Jobs Insurance Jobs IT - Administration Jobs IT - Software Development Jobs Legal Jobs Logistics Jobs Manufacturing Jobs Marketing, Digital Marketing, SEO, SEM Jobs Media Jobs Nursing Jobs Office Administration Jobs Pharmacy Jobs Physical Work Jobs Public Relations Jobs Public Sector Jobs Purchasing Jobs Quality Control Jobs Real Estate Jobs Research and Development Jobs Sales Jobs
Job Positions Jobs by Country
United States Jobs United Kingdom Jobs India Jobs Canada Jobs Australia Jobs Germany Jobs Ireland Jobs Poland Jobs
Company
Blog About Us Contact
Follow Us
Facebook
X (Twitter)
LinkedIn
Instagram
CrawlJobs Group
ITFlashcards AllDevBlogs
© 2026 CrawlJobs Ltd. All Rights Reserved
Terms & Conditions Privacy Policy