Principal AI Tooling Engineer

Design and develop testing frameworks for AI/ML models and LLM applications. Build automated pipelines for model validation, regression testing, and benchmarking. Create evaluation datasets, synthetic data, and test scenarios for edge cases. Implement metrics to assess accuracy, robustness, latency, and safety. Develop tools for prompt testing, output validation, and hallucination detection. Collaborate with engineers, and product teams to define test strategies. Monitor model performance in production and build alerting systems. Ensure compliance with ethical AI standards, fairness, and bias testing. Debug model behavior and identify root causes of failures.