CrawlJobs Logo

AI Evaluation Analyst

ema.co Logo

Ema

Location Icon

Location:
India , Bengaluru

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided

Job Description:

Ema is at the forefront of artificial intelligence research and application, dedicated to creating cutting-edge AI solutions that empower users globally. Join our dynamic team to help shape the future of AI technology.

Job Responsibility:

  • Create Enterprise AI Model Evaluation Test Sets
  • Evaluate AI Models
  • Generate AI Evaluation Reports
  • Test UX Flows
  • Automate Testing Flows

Requirements:

  • Bachelor’s degree in Engineering
  • Exceptional attention to detail with the ability to identify rare and complex edge cases
  • Strong verbal and written English communication abilities
  • 1-3 years of prior professional experience

Nice to have:

  • Bachelor’s degree in Computer Science, Data Science, or a related field
  • Background in analytics, prompt engineering, or QA testing
What we offer:
  • Stay at the cutting edge of GenAI research, guided and coached by top researchers in the field
  • Learn best practices for solving user problems with GenAI products
  • Collaborate across cross-functional teams to effectively achieve customer impact

Additional Information:

Job Posted:
February 21, 2026

Employment Type:
Fulltime
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for AI Evaluation Analyst

Senior Marketing Analyst, AI and Insights

Senior Marketing Analyst who’s passionate about turning data into stories and in...
Location
Location
Slovakia , Bratislava
Salary
Salary:
2800.00 EUR / Month
bloomreach.com Logo
Bloomreach
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years in marketing analytics, data science, or marketing operations (B2B experience a plus)
  • Hands-on experience with Marketing Attribution tools, and data visualization tools (like Tableau, Power BI, or Looker)
  • Strong analytical skills with SQL, Python is a plus
  • Experience using AI or machine learning tools for marketing insights, forecasting, or automation
  • A curiosity for new tech — you love testing tools and figuring out how to make them work for your team
  • Excellent communication skills — you can explain complex data in a way that gets people’s attention
Job Responsibility
Job Responsibility
  • Dig into marketing data from Salesforce, Hockeystack, and other tools to uncover insights that shape campaigns, content, and customer strategy
  • Build dashboards and reports that make complex data simple and actionable for marketing and leadership teams
  • Experiment with AI-powered analytics — from predictive models and segmentation to generative AI tools that surface new marketing opportunities
  • Evaluate and recommend new technologies that can help us level up our marketing intelligence stack
  • Collaborate with marketing, sales, and data teams to make sure insights drive real business impact
  • Tell great data stories — not just numbers, but clear, compelling narratives that influence decisions and strategies
What we offer
What we offer
  • Restricted stock units
  • Company performance bonus
  • Great deal of freedom and trust
  • Flexible working hours
  • Work virtual-first with several Bloomreach Hubs
  • Company events
  • 5 paid days off to volunteer
  • People Development Program
  • Communication coach available
  • Leader Development Program
  • Fulltime
Read More
Arrow Right

Analyst – Automation, AI and Data Analysis

This is a role for someone with strong technical and analytical skills that is l...
Location
Location
Australia , Melbourne
Salary
Salary:
Not provided
frontieradvisors.com.au Logo
Frontier Advisors
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Finance, Commerce, Science, or Engineering undergraduate degree
  • Problem solving skills and a keen attention to detail are critical
  • Strong Excel competency, including experience with formulas and pivot tables
  • Experience in roles requiring data administrative functions highly regarded
  • Tech savvy with the ability to streamline processes and systems to optimise efficiency outcomes
  • Skills to evaluate AI outputs for accuracy, bias, and appropriateness while understanding AI limitations and potential ethical concerns
  • Working knowledge of Python and SQL
  • comfortable using tools such as GitHub Copilot to speed up development, with sound judgement to review and validate generated code.
Job Responsibility
Job Responsibility
  • Be at the forefront of our drive for innovation in operational efficiency by building lightweight AI and automation solutions using Python and SQL (e.g. data validation, templated reporting, summarisation)
  • Write clear, testable AI prompts and templates tailored to Frontier’s style and use-cases
  • fact-check outputs within AI tools and against source data
  • Data administration: Assisting with tasks associated with ongoing data governance including clean up, mapping, management and analysis as required
  • Platform administration: Engaging in support of internal and external platforms and processes to provide guidance and assistance to users
  • Assisting with data mapping and maintenance to ensure the data feeding into Frontier Advisors proprietary systems is accurate, timely and available within our agreed schedule
  • Identifying improvements to current data processes, and where appropriate, assisting with the implementation of new processes
  • Provide support to the Research Team including being an active participant in the manager research process, including the manager review and rating process, the preparation of due diligence documents such as annual reviews, MAPS and other documents.
Read More
Arrow Right

Business Analyst

Andersen is hiring a Business Analyst in Germany to support a large AI-driven di...
Location
Location
Germany
Salary
Salary:
Not provided
andersenlab.com Logo
Andersen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience working as a Business Analyst in software or data-driven environments for 5+ years
  • Experience working on AI/ML or GenAI initiatives, including model-driven features, automation, RAG, and LLM-based products
  • Ability to collect, analyse, and document business requirements for AI systems with unclear or evolving problem spaces
  • Understanding of AI fundamentals, including data quality, model lifecycle, inference vs. training, evaluation metrics, risks, and limitations
  • Experience transforming business needs into clear functional and non-functional requirements for AI teams
  • Strong skills in process analysis, workflow definition, and optimisation for AI-enabled use cases
  • Ability to analyse data sources, constraints, privacy implications, and readiness for AI use
  • Experience preparing user stories, acceptance criteria, and success metrics for AI features
  • Ability to work closely with Data Science, ML Engineering, Product, and Software Engineering teams
  • Strong stakeholder communication skills, with the ability to explain AI capabilities and limitations in simple terms
Job Responsibility
Job Responsibility
  • Gathering and documenting business requirements for AI/ML and GenAI initiatives, ensuring clarity for both technical and non-technical teams
  • Analyzing business processes and identifying opportunities where AI can improve efficiency, accuracy, or decision-making
  • Translating requirements into functional specifications, user stories, acceptance criteria, and measurable outcomes
  • Collaborating with Data Scientists and ML Engineers to align requirements with model capabilities and data constraints
  • Analyzing available data sources and assessing data readiness for AI use, including quality, structure, gaps, and risks
  • Facilitating communication between business stakeholders and technical teams to ensure expectations and limitations are well understood
  • Supporting product discovery and experimentation cycles (PoCs, prototypes, pilots), capturing learnings and shaping next steps
  • Reviewing and validating AI feature outputs, model behaviour, and user experience against defined acceptance criteria
  • Preparing process diagrams, workflows, documentation, and decision materials related to AI-enabled solutions
  • Monitoring the performance of delivered AI features and supporting iterative improvements based on user feedback and insights
What we offer
What we offer
  • Experience in teamwork with leaders in FinTech, Healthcare, Retail, Telecom, and others
  • The opportunity to change the project and/or develop expertise in an interesting business domain
  • Job conditions – you can work both fully remotely and from the office or can choose a hybrid variant
  • Guarantee of professional, financial, and career growth
  • The company has introduced systems of mentoring and adaptation for each new employee
  • The opportunity to earn up to an additional 1,000 USD per month, depending on the level of expertise, which will be included in the annual bonus, by participating in the company's activities
  • Access to the corporate training portal, where the entire knowledge base of the company is collected and which is constantly updated
  • Bright corporate life (parties / pizza days / PlayStation / fruits / coffee / snacks / movies)
  • Certification compensation (AWS, PMP, etc)
  • Referral program
Read More
Arrow Right

Applications Development Senior Programmer Analyst

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
United States , Tampa
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5-8 years of relevant experience
  • experience in systems analysis and programming of software applications
  • experience in managing and implementing successful projects
  • working knowledge of consulting/project management techniques/methods
  • ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • bachelor’s degree/university degree or equivalent experience
  • Python 3-5 years
  • very familiar with different LLM models including Google Vertex, OpenAI, and Anthropic
  • strong understanding of how to code middleware services, either in Python FastAPI or Java Springboot
  • very strong organizational and communication skills
Job Responsibility
Job Responsibility
  • conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • ensure essential procedures are followed and help define operating standards and processes
  • serve as advisor or coach to new or lower level analysts
  • operate with a limited level of direct supervision
  • exercise independence of judgement and autonomy
  • act as SME to senior stakeholders and/or other team members
What we offer
What we offer
  • medical, dental & vision coverage
  • 401(k)
  • life, accident, and disability insurance
  • wellness programs
  • paid time off packages, including planned time off (vacation), unplanned time off (sick leave), and paid holidays
  • Fulltime
Read More
Arrow Right

Technology Business Senior Analyst

The Technology Business Senior Analyst is a position responsible for driving cro...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Techno-functional profile with hands-on experience building tech products that solve real and pressing problems in highly complex and regulated environments
  • Strong background in financial domain, 5+ years of experience as a business analyst in financial services, driving business process automation, digitization and transformation initiatives in a global set up preferred
  • Understanding of machine learning and AI and ability to apply them in practical scenarios, prior experience with such projects is preferred
  • Data Oriented, demonstrate strong analytical/problem-solving skills, with an excellent attention to detail
  • A deeply analytical and experimental mindset and experience working with data sets
  • Proven ability to use data to inform and validate decision
  • MS Excel, SQL knowledge required, able to query data and analyze
  • Consistently demonstrate clear and concise written and verbal communication
  • Proven interpersonal skills with ability to partner and influence across organizational lines
  • Hands-on experience building tech products that solve real and pressing problems in highly complex and regulated environments using latest cutting-edge technologies like AI/ML, low code automation tools
Job Responsibility
Job Responsibility
  • Understand requirements through partnership with business users, cross functional teams, and stakeholders
  • Understand and anticipate user needs
  • Analyze gaps in capabilities, opportunities for enhancing user experience, remediating risk, process optimization for efficiencies and control gaps
  • Manage the Product backlog, requirement prioritization with Business and technology stakeholders, to maximize benefits to the end users
  • Appropriately assess risk and impact when business decisions are made, demonstrating consideration for the firm's reputation and safeguarding Citigroup, its clients, and assets, by driving compliance with applicable laws, rules, and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing, and reporting control issues with transparency
  • Provide evaluative judgment based on analysis of information in complicated and unique situations
  • Develop plans and execute functional strategies, requiring coordination and integration across units
  • Help the team prioritize, negotiate, and remove obstacles to achieve business results, and assist in identifying and removing product development blockers
  • Coordinate UAT testing with technology and business users
  • Conduct data analysis to document business requirements at the right level
What we offer
What we offer
  • Equal opportunity employment
  • Affirmative action employer
  • Accommodation for persons with disabilities
  • Inclusive workplace
  • Fulltime
Read More
Arrow Right

Quantitative Senior Analyst

The Model/Anlys/Valid Sr Analyst is a seasoned professional role requiring advan...
Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8-13 years experience in business and data analysis, process improvement and project management
  • Ability to effectively use complex analytical, interpretive and problem solving techniques
  • Demonstrated interpersonal, verbal and written communication skills
  • Methodical attention to detail
  • Knowledge in statistics and statistical modeling
  • Experience using tools for statistical modeling of large data set
  • Advanced Python programming skills
  • Advanced Analytics skills
  • Experience in ML / Generative AI
  • Proficient in MS Office applications (Excel, Word, PowerPoint)
Job Responsibility
Job Responsibility
  • Applies in-depth disciplinary knowledge, contributing to the development of new techniques and the improvement of processes and work-flow for the area or function
  • Requires in-depth understanding of how areas collectively integrate within the sub-function as well as coordinate and contribute to the objectives of the function and overall business
  • Evaluates moderately complex and variable issues with substantial potential impact
  • Requires good analytical skills in order to filter, prioritize and validate potentially complex and dynamic material from multiple sources
  • Regularly assumes informal/formal leadership role within teams
  • Involved in coaching and training of new recruits
What we offer
What we offer
  • Resources to meet your unique needs
  • Empowerment to make healthy decisions
  • Management of financial well-being to help plan for your future
  • Inclusive workplace
  • Volunteerism opportunities
  • Fulltime
Read More
Arrow Right

Senior Financial Analyst

Finance’s mission is to help support Shopify’s growth over the long-term and ens...
Location
Location
Salary
Salary:
Not provided
shopify.com Logo
Shopify
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven track record of driving significant impact in complex business situations
  • Significant experience in financial modeling and strategic business planning
  • Experience creating investment business cases in a fast-growth business
  • Strong understanding of SQL, with the ability to write complex queries, perform data extraction and manipulation, and maintain data integrity
  • A growth mindset, a builder’s mentality and a proven history of effective execution
  • The ability to thrive in a fast-moving, ambiguous work environment and to navigate through change
  • Superior written and verbal communication skills, business partnering expertise, and an excellent ability to mentor and teach
  • Track record of strong performance in data science, management consulting, investment banking, FP&A, or a related field
  • Experience working with a B2B marketing-sales team
  • Experience in tech, fintech or other SaaS-based business models
Job Responsibility
Job Responsibility
  • Owning ongoing model development and business casing to evaluate growth initiatives, product launches, and overall business line performance
  • Partnering with data science, business operations, commercial and product teams, and Product Line leadership to analyze business drivers, identify areas of opportunity and risk, drive improvements to our unit economics, and strengthen LTV
  • Preparing quarterly P&L and operational metric forecasts
  • Developing actionable management reporting, dashboards, and KPIs, and overseeing monthly variance analysis and operating reviews to track business success
  • Staying updated on emerging AI technologies, and identifying opportunities to automate repetitive manual tasks using tools like SQL, Python and AI to improve FP&A workflows
  • Training team members on AI and other tools while promoting their adoption and benefits to enhance overall efficiency
  • Communicating complex financial and technical concepts in a clear and accessible manner, ensuring understanding across various stakeholders
  • Fostering closer collaboration with business partners to advance our team's mission to make commerce better for everyone
  • Owning overall strategic business planning, including preparing annual operating plans and rolling long range plans
Read More
Arrow Right

Senior Support Operations Analyst

As a Senior Support Operations Analyst, you'll be the technical architect behind...
Location
Location
United States
Salary
Salary:
111220.00 - 133422.00 USD / Year
babylist.com Logo
Babylist
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7+ years in Customer Support Operations, Business Intelligence/Analytics, or similar roles in high-growth, customer-facing environments
  • Strong analytical skills and operational instincts, with a track record of designing scalable processes that improve agent performance and user experience
  • Advanced SQL proficiency with experience writing complex queries, joins, and building data models for customer support analytics
  • 3+ years hands-on experience with Zendesk (or Salesforce Service Cloud), including building automation, workflows, triggers, and API integrations
  • Proven track record implementing and optimizing AI chatbot platforms such as Sierra, Decagon, Forethought, Ada, or similar enterprise solutions
  • Expert-level proficiency building dashboards and reports in Sigma, Tableau, or PowerBI—able to work independently with minimal technical support
  • Experience integrating data across multiple systems and working with APIs
  • Experienced in building and maintaining reporting and performance analysis, including CSAT, cost per contact, and ROI-focused metrics
  • Proven ability to lead end-to-end change management for major tool rollouts and workflow changes, including stakeholder communication, training development, adoption tracking, and post-launch support
  • Familiar with AI support technologies and their practical integration into support systems (chatbots, knowledge base platforms)
Job Responsibility
Job Responsibility
  • Ensure high accuracy visibility into support performance through scalable tools (like dashboards) and internal reporting systems
  • Identify and work with our tech team to drive system improvements across the CS Tech stack (Zendesk, chatbots, phones platform)
  • Work cross-functionally with engineering, product, supply chain, and offshore support teams to identify root causes of user frustration and implement solutions
  • Partner with CS leadership to identify trends and key issues within our reporting frameworks and take active role in resolution
  • Support regular reporting cadences (weekly, monthly, quarterly), with summaries that guide cross-functional decisions
  • Surface actionable insights from data to support empathetic, user-centered decision-making and analyze trends across channels and issue types, supporting data-driven improvements to the support model
  • Drive adoption of AI-enabled workflows that reduce handle time and enhance support quality
  • Own and evolve self-service content, tools, and AI integrations such as Copilots and knowledge bases
  • Maintain ownership of CS-tools and vendor relationships, ensuring we operate on a modern, AI-enabled stack that supports scale, agent effectiveness, and automation—while optimizing for quality, usability, and value
  • Identify inefficiencies and recommend solutions that improve agent effectiveness and reduce user friction
What we offer
What we offer
  • Competitive salary with equity and bonus opportunities
  • Company-paid medical, dental, and vision insurance
  • Retirement savings plan with company matching and flexible spending accounts
  • Generous paid parental leave and PTO
  • Remote work stipend to set up your office
  • Perks for physical, mental, and emotional health, parenting, childcare, and financial planning
  • Fulltime
Read More
Arrow Right