AI Evaluation Analyst Job at Ema (Bengaluru)

New

IT Business Analyst (Ai Programme)

At Ogier, we are committed to making use of cutting-edge technology to deliver t...

Location

Jersey , Jersey

Salary:

Not provided

Ogier

Expiration Date

Until further notice

Requirements

Proven experience in business analysis / product adoption / digital workplace or AI-enabled change in a professional services and/or legal environment
Hands-on delivery of AI or automation initiatives from discovery through rollout, with measurable value outcomes
Demonstrable contributions to governance, risk, and quality processes (e.g. testing, validation, approvals, audit)
Evidential enthusiasm for continuous learning and proactively embracing new technologies, including AI, to enhance personal and team effectiveness
Change management exposure
Strong business analysis toolkit: requirements elicitation, process mapping, user stories, acceptance criteria
Excellent communication skills—able to translate technical concepts into clear business language and practical guidance
Competence with prompt design and evaluation for business scenarios
Structured problem-solving, stakeholder management, and facilitation skills
Data-literate: comfortable with metrics, dashboards, and value tracking

Job Responsibility

Conduct analysis to assess feasibility, impact and risks of AI and technology opportunities
Facilitate use-case discovery workshops and map processes to AI and technology scenarios (e.g., drafting, summarisation, search, data extraction, analysis)
Drive AI and technology adoption across Legal, OGL, and Business Services by understanding role-specific workflows and identifying AI and technology-enablement opportunities
Provide hands-on support for pilots and early-life adoption, transitioning to sustainable, self-serve use
Build trusted relationships with the business to foster understanding and adoption
Support structured change activities
Participate in user groups such as the Legal AI Working Group to share progress, capture feedback, and act as a conduit to service lines
Participate in ongoing technology reviews to capture outcomes and lessons learned
Design reusable AI prompts, patterns and templates and curate scenario playbooks for fee earners and business teams
Proactively identify opportunities to retire/retrain/update AI prompts, workflows and guidance

Fulltime

AI Analyst (Support Agent)- Cortex

Are you an AI Analyst ready to translate raw model output into a reliable, high-...

Location

Israel , Tel Aviv

Salary:

Not provided

Palo Alto Networks Italia

Expiration Date

Until further notice

Requirements

Minimum of 5+ years of professional experience in a blend of technical and analytical roles (e.g., Automated QA, Support Enablement, Data Analysis, AI Research, Prompt Engineering, MLOps)
Proven track record operating at the critical intersection of customer operations, data management, and AI/ML systems
High-level understanding of AI tools, LLMs, machine learning, and applicative AI principles
Proficient in Python for practical applications, including scripting, data processing, and building automation solutions
Demonstrated experience in customer-facing roles with a strong operational understanding of the Customer Support domain (workflows, knowledge base management, optimization)
Strong skills in translating observed model performance issues and Agentic Action requirements into clear, prioritized technical requirements for R&D teams
Dedicated individual who takes full responsibility for their work, driving projects to successful completion
Experience with data validation, content management, and implementing data governance standards, especially for knowledge bases feeding AI systems

Job Responsibility

Support & Customer Advocacy: Champion the Support Journey in Engineering. Develop in-product support instructions that reflect and address real incoming cases, enhancing the quality and effectiveness of AI feature solutions
Model Quality Validation: Use existing evaluation platforms and methodologies to validate production models. Monitor quality metrics to continuously assess and rank AI answers for accuracy and reliability
Prompt Engineering & CT Loop: Drive the Continuous Training loop through systematic prompt engineering (refining and versioning inputs). Analyze failures to define R&D actions or features needed to close model performance gaps
AI Knowledge Governance: Act as AI Content Governor, implementing controls to verify and ingest compliant knowledge base content, ensuring a quality data source
Cross-Functional SME: Serve as the AI Support Co-pilot Subject Matter Expert, partnering with R&D and Support Enablement to translate quality issues into core model logic improvements and feature development

Fulltime

Ai analyst (support agent)- cortex

Are you an AI Analyst ready to translate raw model output into a reliable, high-...

Location

Israel , Tel Aviv

Salary:

Not provided

Palo Alto Networks

Expiration Date

Until further notice

Requirements

Minimum of 5+ years of professional experience in a blend of technical and analytical roles (e.g., Automated QA, Support Enablement, Data Analysis, AI Research, Prompt Engineering, MLOps), with a proven track record operating at the critical intersection of customer operations, data management, and AI/ML systems
AI/ML/LLM Foundation: Possesses a high-level understanding of AI tools, LLMs, machine learning, and applicative AI principles
Python Proficiency: Proficient in Python for practical applications, including scripting, data processing, and building automation solutions
Customer Domain Mastery: Demonstrated experience in customer-facing roles with a strong operational understanding of the Customer Support domain (workflows, knowledge base management, optimization)
Technical Communication: Strong skills in translating observed model performance issues and Agentic Action requirements into clear, prioritized technical requirements for R&D teams
Ownership: A dedicated individual who takes full responsibility for their work, driving projects to successful completion
Data/Content Governance: Experience with data validation, content management, and implementing data governance standards, especially for knowledge bases feeding AI systems

Job Responsibility

Support & Customer Advocacy: Champion the Support Journey in Engineering. Develop in-product support instructions that reflect and address real incoming cases, enhancing the quality and effectiveness of AI feature solutions
Model Quality Validation: Use existing evaluation platforms and methodologies to validate production models. Monitor quality metrics to continuously assess and rank AI answers for accuracy and reliability
Prompt Engineering & CT Loop: Drive the Continuous Training loop through systematic prompt engineering (refining and versioning inputs). Analyze failures to define R&D actions or features needed to close model performance gaps
AI Knowledge Governance: Act as AI Content Governor, implementing controls to verify and ingest compliant knowledge base content, ensuring a quality data source
Cross-Functional SME: Serve as the AI Support Co-pilot Subject Matter Expert, partnering with R&D and Support Enablement to translate quality issues into core model logic improvements and feature development

Data and AI Analyst

As a Data Analyst, you will play a critical role in driving informed decision-ma...

Location

Ireland , Dublin

Salary:

Not provided

NTT DATA

Expiration Date

Until further notice

Requirements

Minimum of 8+ years of experience in data engineering or a similar role
Strong Python skills for data analysis and software development, with familiarity in libraries/frameworks such as PyTorch or TensorFlow
Hands-on experience with Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), vector databases, and prompt engineering
Understanding of classical ML algorithms, data preprocessing techniques, and model evaluation methods
Experience with agentic and orchestration frameworks such as LangChain, AutoGen, or CrewAI
Experience deploying data/ML solutions on cloud platforms (AWS, Azure, GCP) and working with MLOps tools such as Docker

Job Responsibility

Lead data migration activities and analysis
Work with Data scientists/Business Analysts/Product Owner to understand MI and dashboard requirements
Design Data Solutions: Leverage your analytical skills to design innovative data solutions that address complex business requirements and drive decision-making
Create detailed field level mappings as needed for different MI Dashboards
Develop and maintain comprehensive dashboards to visualize key performance indicators
Conduct market research and trend analysis to inform strategic business decisions
Collaborate with cross-functional teams to ensure data accuracy and integrity
Identify and recommend process improvements based on data analysis findings
Prepare detailed reports and presentations on analytical findings for stakeholders
Stay updated on industry trends, data analysis techniques, and best practices

Staff Finance Analyst - AI Group

WHAT YOU DO AT AMD CHANGES EVERYTHING At AMD, our mission is to build great prod...

Location

United States , San Jose

Salary:

109760.00 - 164640.00 USD / Year

AMD

Expiration Date

Until further notice

Requirements

Proven experience in FP&A, commercial finance, or related roles
Strong financial modeling skills with advanced proficiency in Excel
Experience supporting customer deals, pricing, or contract-related financial analysis
Familiarity with contract review and interpreting financial/commercial terms
Working knowledge of accounting concepts (e.g., revenue recognition, accruals)
Strong communication and presentation skills, including PowerPoint, with the ability to synthesize analysis into clear insights
Semiconductor, cloud, or broader technology industry experience preferred
Bachelor’s degree in Finance, Accounting, Economics, or related field
MBA, CPA, or CFA is a plus, but not required

Job Responsibility

Partner with Business Development and Sales to support Cloud and Hyperscaler deals, including financial modeling, scenario analysis, and business case support
Provide financial support through the deal lifecycle, including pricing analysis, investment tracking, and evaluation of commercial terms
Own and manage the Contract Review Board (CRB) process from a finance perspective, including coordinating reviews, preparing materials, and ensuring financial risks and impacts are clearly understood and communicated
Review and interpret customer contracts and deal terms to assess financial implications and ensure alignment with internal assumptions and policies
Build and maintain financial models to evaluate deal economics, profitability, and key sensitivities
Partner with Accounting and Finance teams to ensure appropriate financial treatment and alignment on assumptions
Develop and deliver reporting and presentations to support deal approvals, business reviews, and leadership updates
Create and improve tools, templates, and processes to drive consistency and scalability in deal analysis and CRB workflows
Perform ad hoc analysis and provide insights to support business decision-making in a fast-paced environment

Fulltime

Medical Image Analyst - AI Trainer

Handshake is seeking experienced 3D Slicer users for a flexible, remote position...

Location

United States

Salary:

125.00 USD / Hour

EdTech Jobs

Expiration Date

Until further notice

Requirements

At least 3+ years of hands-on experience with 3D Slicer
A working knowledge of medical imaging concepts, DICOM standards, and clinical research workflows
Strong written communication skills and attention to detail
The ability to work independently and follow written guidelines

Job Responsibility

Evaluate AI-generated content
Create tool-related questions
Review responses for accuracy related to medical imaging and research workflows

IT Business Analyst (AI Programme)

This role blends business analysis, product & project support, user enablement a...

Location

Ireland

Salary:

Not provided

Ogier

Expiration Date

Until further notice

Requirements

Proven experience in business analysis / product adoption / digital workplace or AI-enabled change in a professional services and/or legal environment
Hands-on delivery of AI or automation initiatives from discovery through rollout, with measurable value outcomes
Demonstrable contributions to governance, risk, and quality processes (e.g. testing, validation, approvals, audit)
Evidential enthusiasm for continuous learning and proactively embracing new technologies, including AI, to enhance personal and team effectiveness
Change management exposure
Strong business analysis toolkit: requirements elicitation, process mapping, user stories, acceptance criteria
Excellent communication skills—able to translate technical concepts into clear business language and practical guidance
Competence with prompt design and evaluation for business scenarios
Structured problem-solving, stakeholder management, and facilitation skills
Data-literate: comfortable with metrics, dashboards, and value tracking

Job Responsibility

Conduct analysis to assess feasibility, impact and risks of AI and technology opportunities
Facilitate use-case discovery workshops and map processes to AI and technology scenarios (e.g., drafting, summarisation, search, data extraction, analysis)
Drive AI and technology adoption across Legal, OGL, and Business Services by understanding role-specific workflows and identifying AI and technology-enablement opportunities
Provide hands-on support for pilots and early-life adoption, transitioning to sustainable, self-serve use
Build trusted relationships with the business to foster understanding and adoption
Support structured change activities
Participate in user groups such as the Legal AI Working Group to share progress, capture feedback, and act as a conduit to service lines
Participate in ongoing technology reviews to capture outcomes and lessons learned
Design reusable AI prompts, patterns and templates and curate scenario playbooks for fee earners and business teams
Proactively identify opportunities to retire/retrain/update AI prompts, workflows and guidance

Fulltime

IT Business Analyst (AI Programme)

At Ogier, we are committed to making use of cutting-edge technology to deliver t...

Location

United Kingdom , London

Salary:

Not provided

Ogier

Expiration Date

Until further notice

Requirements

Proven experience in business analysis / product adoption / digital workplace or AI-enabled change in a professional services and/or legal environment
Hands-on delivery of AI or automation initiatives from discovery through rollout, with measurable value outcomes
Demonstrable contributions to governance, risk, and quality processes (e.g. testing, validation, approvals, audit)
Evidential enthusiasm for continuous learning and proactively embracing new technologies, including AI, to enhance personal and team effectiveness
Change management exposure
Strong business analysis toolkit: requirements elicitation, process mapping, user stories, acceptance criteria
Excellent communication skills—able to translate technical concepts into clear business language and practical guidance
Competence with prompt design and evaluation for business scenarios
Structured problem-solving, stakeholder management, and facilitation skills
Data-literate: comfortable with metrics, dashboards, and value tracking

Job Responsibility

Conduct analysis to assess feasibility, impact and risks of AI and technology opportunities
Facilitate use-case discovery workshops and map processes to AI and technology scenarios (e.g., drafting, summarisation, search, data extraction, analysis)
Drive AI and technology adoption across Legal, OGL, and Business Services by understanding role-specific workflows and identifying AI and technology-enablement opportunities
Provide hands-on support for pilots and early-life adoption, transitioning to sustainable, self-serve use
Build trusted relationships with the business to foster understanding and adoption
Support structured change activities
Participate in user groups such as the Legal AI Working Group to share progress, capture feedback, and act as a conduit to service lines
Participate in ongoing technology reviews to capture outcomes and lessons learned
Design reusable AI prompts, patterns and templates and curate scenario playbooks for fee earners and business teams
Proactively identify opportunities to retire/retrain/update AI prompts, workflows and guidance

Fulltime

Select Country

AI Evaluation Analyst

Job Description

Job Responsibility

Requirements

Nice to have

What we offer

Looking for more opportunities?