CrawlJobs Logo
C
Codvo AI · IT - Software Development

Qa Lead – Ai/Ml Systems

India, Pune · Job Posted June 03, 2026
Apply Position
Job Link Share

Job Description

We are looking for an experienced QA Lead – AI Systems to lead the validation and quality assurance of AI-powered products, including LLM/RAG systems, Software as a Medical Device (SaMD), and other AI-driven digital solutions. This is a key leadership role responsible for establishing end-to-end QA frameworks, validation strategies, and compliance processes for AI/ML models across development and deployment pipelines. The ideal candidate will bridge data science, software engineering, and regulatory QA to ensure AI systems meet performance, safety, and compliance standards.

Job Responsibility

  • Own and drive QA strategy for AI systems across the model, data, and product lifecycle
  • Lead AI model validation efforts, covering LLM/RAG testing, bias analysis, and performance evaluation
  • Define and implement AI Evaluation Metrics (accuracy, fairness, drift, explainability) aligned with business and regulatory expectations
  • Establish frameworks for Explainability Testing (SHAP, LIME, XAI) and ensure interpretability of AI outcomes
  • Collaborate with Data Science and MLOps teams to validate models within cloud environments (Azure ML, AWS Sagemaker, Vertex AI)
  • Drive verification and validation (V&V) for AI models and applications under FDA and SaMD compliance frameworks
  • Ensure test traceability, documentation, and audit readiness in line with ISO 13485, IEC 62304, and ISO 14971
  • Develop Python-based automation for AI testing, data validation, and model evaluation pipelines
  • Provide technical leadership to QA teams, ensuring alignment with AI/ML development best practices
  • Collaborate with cross-functional teams (Product, Data, Regulatory, Engineering) to identify risks, gaps, and opportunities for test optimization
  • Present validation findings, risks, and recommendations to senior stakeholders and regulatory reviewers

Requirements

  • Bachelor’s or Master’s degree in Computer Science, Data Engineering, Biomedical Engineering, or related field
  • 8–12 years of total QA experience, with 2–3 years directly in AI/ML or GenAI QA
  • Proven experience in AI Model Validation, LLM/RAG Testing, and AI Evaluation Metrics
  • Strong knowledge of MLOps concepts and cloud platforms (Azure ML, AWS Sagemaker, Vertex AI)
  • Understanding of FDA AI/ML Compliance, SaMD testing, and regulated software QA
  • Hands-on expertise in Python automation and testing AI/ML pipelines
  • Excellent documentation and communication skills with ability to produce traceable validation artifacts
Codvo AI - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Qa Lead – Ai/Ml Systems

8 matching positions

Sr Manager, Digital Products & Platforms

This role leads engineering teams that design, build, and operate enterprise-gra...
Location
Location
United States , Frisco; Overland Park
Salary
Salary:
160000.00 - 288500.00 USD / Year
https://www.t-mobile.com Logo
T-Mobile
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree and 7 years of related work experience or a combination of education and experience deemed equivalent Acceptable areas of study include Computer Science, Engineering, IT or equivalent experience
  • 7-10 years experience of developing large scale business systems applications in an agile product development environment with an engineering background across the modern stack (Java, Python, JavaScript/TypeScript, APIs, databases)
  • 2-4 years People leadership managing cross-functional engineering teams (5+ engineers in direct reporting relationships)—including hiring, performance management, career development, and org design
  • At least 18 years of age
  • Legally authorized to work in the United States
Job Responsibility
Job Responsibility
  • Lead and manage cross-functional engineering teams (software engineers, QA, DevOps, data engineers, product managers) to deliver enterprise applications—providing technical direction, architecture guidance, coaching, and performance management
  • Build and evolve an agentic AI capability within the engineering org—standing up AI-assisted workflows (automated code generation, test creation, Jira-to-MR pipelines) that reduce cycle time and free engineers for high-judgment work
  • Define and drive continuous improvement and engineering excellence—CI/CD pipeline optimization, automated quality gates, tech debt management, and measurable productivity frameworks (DORA metrics, automation coverage)
  • Own the technical roadmap in collaboration with architecture, platform engineering, cybersecurity, and design teams—balancing feature delivery, platform modernization, and automation investment
  • Manage technical vendor relationships and evaluate emerging technology (AI/ML tooling, cloud-native platforms, data infrastructure) to keep the engineering stack current and competitive
  • Recruit, hire, and develop engineering talent who bring strong full-stack fundamentals, a bias toward automation, and the ability to work effectively alongside AI-augmented toolchains
What we offer
What we offer
  • competitive base salary and compensation package
  • annual stock grant
  • employee stock purchase plan
  • 401(k)
  • access to free, year-round money coaches
  • medical, dental and vision insurance
  • flexible spending account
  • paid time off and up to 12 paid holidays
  • paid parental and family leave
  • family building benefits
  • Fulltime
Read More
Arrow Right

Ai Gpu Product Planning Lead - Embedded Software

AMD's AECG group is looking for a AI GPU Product Planning Lead - Embedded Softwa...
Location
Location
Canada , Markham
Salary
Salary:
147680.00 - 221520.00 CAD / Year
amd.com Logo
AMD
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong understanding of GPU architectures and GPU software stacks, including graphics, compute, and AI/ML workloads, across Linux and Windows environments
  • Experience with virtualization and embedded systems is highly desirable
  • Strong cross-functional communication and dependency management skills
  • Experience in software integration, program planning, or technical project coordination
  • BS/MS in Computer Science, Computer Engineering, or Electrical Engineering
Job Responsibility
Job Responsibility
  • Work with market segment leads, product line managers, and solution planning team to translate key embedded customer needs into actionable engineering requirements
  • Work with engineering teams to establish and maintain deliverable roadmaps with a focus on early enablement for key embedded customers
  • Define and align requirements for all solution components: Virtualization, Yocto Project Linux for embedded ROCm support, ROCm for AI workloads and specialized stacks (ROS, multimedia analytics, VLM/LLM/CNN's)
  • Work with AECG technical marketing and field support teams to deliver timely demonstrations and examples to promote customer adoption
  • Act as single-point coordinator between ROCm dev, OS enablement, virtualization, QA, software engineering and customer enablement teams
  • Fulltime
Read More
Arrow Right

Ai And Application Development Engineer

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Core Java: Strong understanding of Java (JDK 8+, preferably Java 11/17), including multithreading, collections, garbage collection, and JVM internals
  • Frameworks: Extensive experience with Spring Framework (Spring Boot, Spring MVC, Spring Data JPA, Spring Security)
  • Middleware: Proven experience in designing and developing RESTful APIs and microservices
  • Database Technologies (Must): Relational Databases: Strong proficiency in SQL and experience with Oracle databases, including schema design, query optimization, and stored procedures
  • NoSQL Databases: Experience with MongoDB, including data modeling, querying, and performance tuning
  • CI/CD & DevOps: Hands-on experience with CI/CD tools and practices (e.g., Jenkins, GitLab CI, GitHub Actions, Maven/Gradle, Docker, Kubernetes)
  • Version Control: Proficiency with Git and standard branching strategies (e.g., Gitflow)
  • Testing: Experience with unit testing frameworks (JUnit, Mockito) and integration testing
  • Web Technologies (Beneficial): Familiarity with web services (SOAP/REST), XML, JSON
  • AI Tools & Methodologies (Must): Demonstrable exposure and practical experience with AI development tools such as Devin, GitHub Copilot, Claude, Anti Gravity, and Codex
Job Responsibility
Job Responsibility
  • Lead the design, development, and implementation of complex middleware applications using Java and Spring Boot: Utilize AI-powered code generation tools (e.g., Devin, Copilot, Codex) to accelerate development, automate boilerplate code, suggest optimal implementations, and enforce architectural patterns
  • Architect and optimize database interactions with Oracle, SQL, and MongoDB, ensuring high performance and data integrity: Employ AI to analyze database query performance, suggest advanced indexing strategies, optimize schema designs, and generate efficient SQL/NoSQL queries
  • Drive the adoption and continuous improvement of CI/CD pipelines to facilitate rapid and reliable software delivery: Integrate AI into CI/CD processes for intelligent test case generation, predictive failure analysis, automated code vulnerability scanning, and optimization of pipeline execution times based on historical data
  • Collaborate with cross-functional teams, including product management, QA, and operations, to define requirements, design solutions, and deliver high-quality software: Use AI-powered communication and summarization tools (e.g., Claude) to streamline requirement gathering, document analysis, and stakeholder communication
  • Mentor and provide technical guidance to junior and mid-level software engineers, fostering a culture of technical excellence and continuous learning: Leverage AI platforms for personalized learning paths, automated code feedback, and explanations of complex technical concepts
  • Actively research and experiment with AI technologies to identify opportunities for enhancing developer productivity, automating tasks, and improving software quality: Continuously explore emerging AI tools and techniques (such as Anti Gravity for complex problem-solving) and assess their applicability to our development ecosystem
  • Participate in code reviews, ensuring adherence to coding standards, best practices, and architectural guidelines: Utilize AI-powered code analysis tools to pre-scan code for potential bugs, security vulnerabilities, performance bottlenecks, and style deviations, allowing human reviewers to focus on higher-level logic and design
  • Troubleshoot and resolve complex technical issues, ensuring the stability and performance of production systems: Implement AI-driven anomaly detection in monitoring systems, leverage AI for rapid log analysis and root cause identification, and automate incident response workflows
  • Contribute to the strategic planning and technical roadmap for our middleware platforms: Employ AI to analyze industry trends, forecast technology evolution, assess the impact of new features, and prioritize roadmap initiatives based on data-driven insights
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, and model development: Utilize AI for data synthesis, predictive modeling for estimations, identification of potential IT risks, and accelerated model prototyping and validation
  • Fulltime
Read More
Arrow Right

Software Engineer IV - Sales Domain (Java / Google Cloud / AI Enablement)

We are seeking a highly experienced Software Engineer IV to join our Sales Domai...
Location
Location
United States , Birmingham
Salary
Salary:
Not provided
genpt.com Logo
Genuine Parts Company
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Expert-level Java proficiency (modern Java versions, concurrency, streams, async patterns)
  • Strong experience with Spring Boot and/or Quarkus
  • Deep understanding of: Microservices architecture, RESTful API design, Event-driven systems, Distributed system patterns
  • Experience with relational databases, JDBC, ORM frameworks
  • Experience with NoSQL/document databases (MongoDB preferred)
  • Experience deploying and managing applications in Google Cloud Platform (preferred) or comparable cloud platforms
  • Experience with containerization (Docker) and CI/CD pipelines
  • Familiarity with Pub/Sub or Kafka-based messaging systems
  • Strong Git-based workflow experience
  • Working knowledge of Python and ability to understand, extend, or refactor Python-based services
Job Responsibility
Job Responsibility
  • Lead the design, development, and deployment of cloud-native Java microservices using Spring Boot and/or Quarkus within Google Cloud Platform
  • Architect scalable, reliable, and secure systems aligned to enterprise modernization strategy
  • Drive integration of AI/ML and generative AI capabilities into existing sales domain applications
  • Collaborate with innovation and data teams to operationalize AI solutions into production-grade systems
  • Provide technical leadership across a portfolio of sales-related applications
  • Guide architectural decisions around API design, event-driven systems, and distributed microservices
  • Mentor and upskill developers in: Cloud-native design patterns, Microservices architecture, AI service integration, Python service interoperability
  • Conduct code reviews and promote high standards for maintainability, testability, and documentation
  • Partner with Product Owners, Architects, DevOps, and QA to deliver high-quality solutions
  • Champion CI/CD, DevSecOps, and modern SDLC best practices
What we offer
What we offer
  • Flexible Working Schedules
  • Wellbeing Programs
  • Learning Opportunities
  • Community Engagement
  • Paid Parental Leave
  • Pet Insurance
  • Healthcare coverage
  • 401(k)
  • Tuition reimbursement
  • Vacation, sick, and holiday pay
  • Fulltime
Read More
Arrow Right

Senior Cloud Developer

Senior Cloud Developer This role has been designed as ‘’Onsite’ with an expecta...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.hpe.com/ Logo
Hewlett Packard Enterprise
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related field from a reputed institution
  • 10–15 years of hands‑on QA testing and automation experience, with a proven track record of delivering high‑quality software on schedule in both waterfall and agile methodologies
  • Demonstrated Cloud architect experience, architecting large-scale test and automation programs
  • Strong expertise in solution testing, including designing test topologies, deploying and validating systems, and debugging/troubleshooting across networking and cloud environments
  • Practical experience with greenfield deployments, brownfield migrations, and production considerations such as scale, performance, and latency for cloud environments
  • Direct experience diagnosing and resolving customer issues in live production environments
  • Solid background in Network Management System (NMS) and core networking technologies and protocols (routing, switching)
  • Areas like IP, VLAN, LACP, STP/MSTP, BGP, OSPF, EVPN, VXLAN, QoS, Multicast, High Availability and Stacking systems
  • Advanced automation and programming skills (Python, Cypress preferred, or equivalent)
  • Experience designing and building automation infrastructure, architecture, and strategy from the ground up
Job Responsibility
Job Responsibility
  • Lead and create test strategies for switch software and cloud deployment
  • Test cloud software systems across various platforms, execute cases covering functionality, compliance, negative, solution, integration, scale, interoperability and performance scenarios
  • Design, build and maintain automated test suites and test infrastructure for hyperscale, multi‑cloud environments
  • Implement test orchestration, environment provisioning and CI/CD integration
  • Review and validate test plans, test cases and QA artifacts against product architecture and quality objectives
  • identify implementation gaps early and collaborate with development teams to mitigate risks
  • Own quality reviews of team deliverables — defects, test reports and automation scripts — and provide timely, actionable feedback to improve test coverage and reliability
  • Establish guidelines and standards for testing programmable networks and APIs, including REST, GraphQL, YANG, gRPC, and AI‑driven test scenarios
  • Drive innovation and adoption of new tools, frameworks and automation technologies across the cloud QA organization to increase efficiency and reduce flakiness
  • Lead strategic QA initiatives that address real customer challenges (e.g., high‑availability failures, brownfield migrations, scale issues) and translate findings into improvements and preventive checks
What we offer
What we offer
  • Health & Wellbeing
  • Personal & Professional Development
  • Unconditional Inclusion
Read More
Arrow Right

Senior Cloud Developer

Senior Cloud Developer This role has been designed as ‘’Onsite’ with an expecta...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.hpe.com/ Logo
Hewlett Packard Enterprise
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related field from a reputed institution
  • 10–15 years of hands‑on QA testing and automation experience, with a proven track record of delivering high‑quality software on schedule in both waterfall and agile methodologies
  • Demonstrated Cloud architect experience, architecting large-scale test and automation programs
  • Strong expertise in solution testing, including designing test topologies, deploying and validating systems, and debugging/troubleshooting across networking and cloud environments. Practical experience with greenfield deployments, brownfield migrations, and production considerations such as scale, performance, and latency for cloud environments
  • Direct experience diagnosing and resolving customer issues in live production environments
  • Solid background in Network Management System (NMS) and core networking technologies and protocols (routing, switching). Areas like IP, VLAN, LACP, STP/MSTP, BGP, OSPF, EVPN, VXLAN, QoS, Multicast, High Availability and Stacking systems
  • Advanced automation and programming skills (Python, Cypress preferred, or equivalent). Experience designing and building automation infrastructure, architecture, and strategy from the ground up
  • Advanced knowledge with at least one public cloud (AWS, Azure, Google Cloud) and common services used for testing and deployment
  • Experience with Kubernetes‑based test orchestration and CI/CD pipelines for networked/clouded systems
  • Strong analytical, problem‑solving, and debugging abilities
Job Responsibility
Job Responsibility
  • Lead and create test strategies for switch software and cloud deployment. Test cloud software systems across various platforms, execute cases covering functionality, compliance, negative, solution, integration, scale, interoperability and performance scenarios
  • Design, build and maintain automated test suites and test infrastructure for hyperscale, multi‑cloud environments. Implement test orchestration, environment provisioning and CI/CD integration
  • Review and validate test plans, test cases and QA artifacts against product architecture and quality objectives
  • identify implementation gaps early and collaborate with development teams to mitigate risks
  • Own quality reviews of team deliverables — defects, test reports and automation scripts — and provide timely, actionable feedback to improve test coverage and reliability
  • Establish guidelines and standards for testing programmable networks and APIs, including REST, GraphQL, YANG, gRPC, and AI‑driven test scenarios
  • Drive innovation and adoption of new tools, frameworks and automation technologies across the cloud QA organization to increase efficiency and reduce flakiness
  • Lead strategic QA initiatives that address real customer challenges (e.g., high‑availability failures, brownfield migrations, scale issues) and translate findings into improvements and preventive checks
  • Troubleshoot complex issues in customer deployments and lab environments using strong debugging, log analysis and diagnostic techniques
  • develop reproducible tests and automated regression checks
What we offer
What we offer
  • Health & Wellbeing
  • Personal & Professional Development
  • Unconditional Inclusion
  • Fulltime
Read More
Arrow Right

Ai Lead Application Architecture - Vice President

We are seeking a highly experienced and technically proficient Sr Applied AI Eng...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 15 years of relevant experience in leading Apps Development or systems design role
  • Extensive experience system analysis and in programming of software applications
  • Demonstrable exposure and practical experience with AI development
  • Knowledge of capital market is plus
  • Experience in managing and implementing successful projects
  • Subject Matter Expert (SME) in at least one area of Applications Development
  • Ability to adjust priorities quickly as circumstances dictate
  • Demonstrated leadership and project management skills
  • Consistently demonstrates clear and concise written and verbal communication
  • Bachelor’s degree/University degree or equivalent experience
Job Responsibility
Job Responsibility
  • Lead the design, development, and implementation of complex middleware applications using Java and Spring Boot: Utilize AI-powered code generation tools (e.g., Devin, Copilot, Codex) to accelerate development, automate boilerplate code, suggest optimal implementations, and enforce architectural patterns. Leverage AI for design pattern identification and complex system architecture validation
  • Architect and optimize database interactions with Oracle, SQL, and MongoDB, ensuring high performance and data integrity: Employ AI to analyze database query performance, suggest advanced indexing strategies, optimize schema designs, and generate efficient SQL/NoSQL queries. AI-driven tools can also assist in predicting database load and recommending scaling solutions
  • Drive the adoption and continuous improvement of CI/CD pipelines to facilitate rapid and reliable software delivery: Integrate AI into CI/CD processes for intelligent test case generation, predictive failure analysis, automated code vulnerability scanning, and optimization of pipeline execution times based on historical data
  • Collaborate with cross-functional teams, including product management, QA, and operations, to define requirements, design solutions, and deliver high-quality software:Use AI-powered communication and summarization tools (e.g., Claude) to streamline requirement gathering, document analysis, and stakeholder communication. AI can also assist in translating technical designs into accessible formats for various audiences
  • Mentor and provide technical guidance to junior and mid-level software engineers, fostering a culture of technical excellence and continuous learning: Leverage AI platforms for personalized learning paths, automated code feedback, and explanations of complex technical concepts. Encourage junior engineers to adopt AI-driven development practices
  • Actively research and experiment with AI technologies to identify opportunities for enhancing developer productivity, automating tasks, and improving software quality:Continuously explore emerging AI tools and techniques (such as Anti Gravity for complex problem-solving) and assess their applicability to our development ecosystem
  • Participate in code reviews, ensuring adherence to coding standards, best practices, and architectural guidelines: Utilize AI-powered code analysis tools to pre-scan code for potential bugs, security vulnerabilities, performance bottlenecks, and style deviations, allowing human reviewers to focus on higher-level logic and design
  • Troubleshoot and resolve complex technical issues, ensuring the stability and performance of production systems: Implement AI-driven anomaly detection in monitoring systems, leverage AI for rapid log analysis and root cause identification, and automate incident response workflows
  • Contribute to the strategic planning and technical roadmap for our middleware platforms: Employ AI to analyze industry trends, forecast technology evolution, assess the impact of new features, and prioritize roadmap initiatives based on data-driven insights
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, and model development: Utilize AI for data synthesis, predictive modeling for estimations, identification of potential IT risks, and accelerated model prototyping and validation
  • Fulltime
Read More
Arrow Right

Principal Software Automation/test Engineer

At Palo Alto Networks, we're not just building products — we're redefining what'...
Location
Location
United States , Santa Clara
Salary
Salary:
147000.00 - 237500.00 USD / Year
paloaltonetworks.com Logo
Palo Alto Networks
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 9+ years in software tooling or infrastructure engineering within large-scale enterprises or high-growth startups
  • Strong programming skills in Python
  • Proven experience leveraging AI/ML technologies (LLMs, agentic frameworks, prompt engineering) to build intelligent engineering tools
  • Deep understanding of public cloud platforms (AWS, GCP, Azure) and building tools that leverage cloud-native services
  • Demonstrated experience designing complex tooling frameworks from scratch with a focus on scalability and extensibility
  • Proficiency in integrating technologies such as Pytest, Selenium, Playwright, Terraform, Docker, and Kubernetes
  • Strong troubleshooting skills and experience supporting tooling for distributed systems and SaaS-based products
  • Bachelor’s or Master’s degree in Computer Science, Electrical Engineering, or a related field
Job Responsibility
Job Responsibility
  • Architect Next-Gen Platforms: Lead the design of tooling platforms and comprehensive strategies for complex, large-scale systems
  • Drive AI-Powered Automation: Leverage AI to design intelligent tools that enhance testing efficiency, code quality, and productivity across the organization
  • Build Agentic Workflows: Integrate LLM-powered assistants and agentic workflows to automate repetitive tasks like test case creation, log analysis, and code review triage
  • Operationalize Emerging Tech: Evaluate and adopt technologies such as Retrieval-Augmented Generation (RAG) and fine-tuned models to push the boundaries of engineering automation
  • Establish AI Guardrails: Define best practices for responsible AI usage, including hallucination mitigation and human-in-the-loop review processes
  • Enhance Developer Experience: Design and implement internal tools, SDKs, and frameworks to boost engineering velocity and productivity
  • Advanced Framework Development: Build robust automation frameworks equipped with detailed reporting, observability, and seamless CI/CD integration
  • Cloud-Native Environments: Manage complex testbeds and infrastructure-as-code solutions to support varied development and testing scenarios
  • Predictive Analytics: Develop AI-driven defect prediction and root cause analysis tools to accelerate the QA lifecycle
  • Influence Strategy: Provide insights into engineering efficiency and tooling gaps to influence the development of cutting-edge solutions
What we offer
What we offer
  • restricted stock units
  • bonus
  • Fulltime
Read More
Arrow Right