Sr. SDET

• Own end-to-end quality for agentic features and workflows, including strategy, development, execution, and release qualification
• Design and build automation tooling and frameworks for AI/LLM-driven systems, including prompt flows, agent orchestration, and tool integrations
• Develop and maintain evaluation frameworks (evals) to measure response quality, accuracy, and hallucination rates
• Drive automation coverage (80%+ for critical AI workflows) using deterministic + probabilistic validation approaches
• Integrate AI quality checks into CI/CD pipelines with fast feedback cycles (<15 minutes for PR validation)
• Build tooling for LLM observability and debugging, including prompt tracing and response analysis
• Partner with Applied AI teams on prompt engineering, model selection, and evaluation strategies
• Design and execute performance and load tests for AI services (latency, throughput, cost efficiency)
• Identify and mitigate risks related to hallucinations, bias, safety, and edge cases
• Define and track AI quality KPIs (task success rates, precision/recall, latency, etc.)
• Participate in design and architecture reviews to ensure systems are testable, observable, and resilient
• Mentor engineers and contribute to raising the bar on AI quality engineering practices

• 5+ years of experience in software engineering or SDET roles with an emphasis on software development
• Strong programming skills in Python (preferred), Java, or JavaScript
• Experience testing distributed, cloud-native SaaS systems and APIs
• Demonstrated proficiency in coding with AI agents to accelerate development and improve code quality
• Hands-on exposure to LLMs or AI/ML systems (e.g., OpenAI, Claude, Gemini, or similar platforms)
• Understanding of non-deterministic systems and probabilistic testing approaches
• Experience building test frameworks and scalable automation systems
• Familiarity with AI evaluation techniques (benchmarking, golden datasets, human-in-the-loop validation)
• Experience with CI/CD pipelines (e.g., Jenkins, GitHub Actions)
• Strong collaboration skills with the ability to work across distributed teams and time zones
• Bachelor’s degree in Computer Science or equivalent practical experience

Competitive salary, comprehensive benefits, and real opportunities for growth