CrawlJobs Logo

Quality Assurance Analyst, AI Delivery

cresta.com Logo

Cresta

Location Icon

Location:
United States

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

50000.00 - 70000.00 USD / Year

Job Description:

We’re looking for a Quality Assurance Analyst to ensure Cresta’s AI systems perform with accuracy, fairness, and compliance. You’ll audit AI-generated conversations, evaluate model behavior, and deliver feedback that improves model performance and customer trust.

Job Responsibility:

  • Review AI-generated summaries for accuracy and completeness
  • provide feedback to improve models
  • Conduct behavioral audits to verify agent triggers and responses match expected outcomes
  • Perform redaction audits to protect sensitive customer data
  • Run bias audits to detect and document fairness issues
  • Compare audio and text for transcription accuracy
  • Support Opera platform maintenance and rule configuration to keep AI outputs accurate

Requirements:

  • 1–3 years in QA or data labeling
  • Exceptional attention to detail and analytical rigor
  • Strong documentation and communication skills
  • Comfort with accuracy-driven tasks

Nice to have:

  • Familiarity with conversational AI or annotation tools
  • Experience with bias / redaction audits or speech-to-text QA
What we offer:
  • Competitive pay and equity
  • Medical, dental, vision coverage
  • Flexible PTO, remote-first culture
  • Monthly wellness stipend & home-office support

Additional Information:

Job Posted:
December 07, 2025

Employment Type:
Fulltime
Work Type:
Remote work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Quality Assurance Analyst, AI Delivery

Quality Analyst, AI Business

As a Quality Analyst within Prolific AI Data Services, you will play a critical ...
Location
Location
Mexico , Mexico City
Salary
Salary:
Not provided
prolific.com Logo
Prolific
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 1–3 years of experience in analysis, quality, operations, or data-focused roles, or equivalent academic projects or internships
  • A background in computer science, data science, statistics, engineering, or a related quantitative discipline
  • Strong SQL skills (required), with the ability to independently query, join, and summarise datasets to answer quality and operational questions
  • Working proficiency in Python and/or R, with experience writing scripts or notebooks to clean data, compute metrics, and automate checks
  • Strong analytical rigour, with a high bar for precision, consistency, and clear documentation of decisions
  • Solid statistical fundamentals, including intuition for sampling, distributions, confidence, and agreement concepts
  • Clear and structured communication skills, with the ability to explain issues using concrete examples and propose practical fixes
  • A collaborative mindset and willingness to partner cross-functionally to improve quality outcomes and delivery reliability
Job Responsibility
Job Responsibility
  • Run day-to-day quality measurement for managed service studies, including sampling, rubric-based review, and structured reporting
  • Apply rubrics and guidelines consistently, flag ambiguities, and propose clear, actionable clarifications as task types evolve
  • Maintain and update golden sets, and support calibration efforts by preparing materials, tracking outcomes, and documenting decisions
  • Track defects and failure modes by categorising issues, assessing severity, and keeping defect taxonomies current
  • Run automated quality checks using SQL and Python or R, including completeness checks, schema and format validation, anomaly detection, and label distribution monitoring
  • Support launch gates by preparing readiness artefacts such as rubric versioning, gold status, sampling plans, and early pilot findings
  • Investigate quality drops by pulling examples, quantifying scope, and supporting root-cause analysis and corrective and preventive actions (CAPAs)
  • Build dashboards and weekly updates covering defect rates, rework, agreement, throughput versus quality trade-offs, and SLA risk
  • Partner with Operations, reviewers, and—when needed—Product and Engineering to improve task design, reviewer workflows, and escalation paths
What we offer
What we offer
  • competitive salary, benefits, and hybrid working within our impactful, mission-driven culture
  • equity
  • opportunity to earn a cash variable element, such as a bonus or commission
Read More
Arrow Right

DevOps Quality Engineer

Engineer reliability at scale. We’re looking for a DevOps Quality Engineer who t...
Location
Location
Bulgaria , Sofia
Salary
Salary:
Not provided
ebrd.com Logo
European Bank for Reconstruction and Development
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Holds ISTQB Foundation as a minimum
  • Advanced Test Analyst or equivalent certifications desirable
  • Qualification in IT Service Management (ITIL v3 or v4 Foundation) or demonstrable experience integrating QA practices into ITSM processes
  • Familiar with the NIST Cybersecurity Framework (CSF) and Digital Operational Resilience Act (DORA), with practical awareness of how they influence quality standards and assurance planning
  • Demonstrates solid understanding of automation and non-functional testing concepts, including performance, accessibility, and shift-left/shift-right practices
  • Experience working within Agile, DevOps, and product-aligned teams, contributing to sprint-based delivery and continuous integration testing strategies
  • Proficient in test tooling and CI/CD frameworks including Azure DevOps, Selenium, Cypress, Jenkins, Git, and test management platforms such as TestRail or Zephyr
  • Familiarity with AI/ML use cases in quality engineering, including AI-assisted test case generation, defect clustering, and predictive analytics
  • Strong communication and collaboration skills, with the ability to explain test scenarios, defects, and coverage to technical and non-technical stakeholders
  • Awareness of security, compliance, and resilience considerations such as OWASP Top 10, ISO 27001, GDPR, and DORA, with practical experience embedding these into quality practices
Job Responsibility
Job Responsibility
  • Plans and performs testing of infrastructure changes, configuration updates, and cloud deployments across data centres, branch offices, and Azure environments, ensuring functional and non-functional validations
  • Collaborates with DevOps, Infrastructure, and Cloud Platform teams to verify IaC deployments, perform smoke testing post-deployment, and document any issues in platform reliability
  • Writes and maintains basic automation scripts or checklists for network, AV, and platform components, contributing to regression detection and provisioning assurance
  • Investigates environment or deployment defects, escalates issues with supporting diagnostics, and retests fixes with attention to uptime, latency, and failover coverage
  • Engages in sprint reviews and planning sessions, raising risks tied to infrastructure changes, recommending testing scope for AV, cloud services, or network resilience improvements
What we offer
What we offer
  • Varied, stimulating and engaging work that gives you an opportunity to interact with a wide range of experts in the financial, political, public and private sectors across the regions we invest in
  • A working culture that embraces inclusion and celebrates diversity
  • An environment that places sustainability, equality and digital transformation at the heart of what we do
  • Fulltime
Read More
Arrow Right

Associate Product Manager

A Tyfone Associate Product Manager will be working on cutting edge digital banki...
Location
Location
United States
Salary
Salary:
Not provided
tyfone.com Logo
Tyfone
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Ability to use AI-driven tools, workflows, and frameworks to build rapid prototypes, test concepts, communicate requirements, create documentation, and optimize processes
  • An understanding of the banking ecosystem in the US
  • Bachelor’s degree in computer science, engineering or related fields
  • Business master’s degree preferred
  • Insatiable curiosity
  • High agency to get things done
  • Energy to solve problems exploring multiple angles
  • Disdain for the status quo
  • Communication skills to listen more than speak. And speak with purpose and clarity
Job Responsibility
Job Responsibility
  • Carry out market research to establish competitive landscape and market definition and engage with customers, vendors, analysts, and other players in the fintech ecosystem for the same
  • Research potential product features, in terms of — usage, best practices, relevant regulations, and fraud potential
  • Support decision-making with data: Build dashboards, create reports, and provide actionable recommendations to help product managers make informed decisions
  • Analyze product performance and user behavior: Gather and interpret data from various sources to uncover insights about how users interact with our products and identify areas for Improvement
  • Assist in product experimentation: Design, execute, and analyze A/B tests and other experiments to validate hypotheses and guide product improvements
  • Participate in product planning, backlog grooming, and roadmap definition
  • Help manage the product lifecycle from market/customer requirements to tactical activities leading to delivery
  • Drive product development across development teams (primarily Customer Support / Sales Development/Engineering and Quality Assurance) through requirements clarifications and finalization, scope definition and coordination between teams
  • Devise go-to market assets: Define, plan and automate creation of market-facing product collaterals that will be consumed by end users as well as financial institutions
  • Coordinate and build strong working relations with various internal stakeholders, including – Executive Management team, Engineering, Sales, Marketing, Quality Assurance to drive outcomes
Read More
Arrow Right

Engineering Excellence Testing Developer

The Applications Development Intermediate Programmer Analyst is an intermediate ...
Location
Location
India , Pune; Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proficiency in programming languages such as Java, Python, VB Script or JavaScript
  • Good Programming experience with Pega
  • Extensive experience with test automation tools like Selenium, Playwright or UFT
  • Proficient Experience with tools such as ALM, JIRA, Zephyr, advanced excel features
  • Experience in Oracle and SQL queries for test data mining, data generation & conditioning
  • Strong understanding of banking workflows, financial products, and regulatory requirements
  • Experience testing banking applications such as loan management systems, payment gateways, or risk management platforms
  • Experience in using mainframe for testing and automation
  • Experience in BDD/TDD, Cucumber framework development
  • Experience in building and maintaining CI/CD pipelines with tools like Jenkins
Job Responsibility
Job Responsibility
  • Test Automation Development: Design and implement robust, scalable, and maintainable test automation frameworks using AI-driven tools and technologies
  • Develop automated test scripts for functional, regression testing of banking applications
  • Ability to identify the scope and E2E delivery of Automation
  • Maintenance of Automation framework and repository
  • AI Integration: Utilize AI and machine learning models to optimize test case generation, execution, and defect prediction
  • Implement AI-based solutions for intelligent test data generation and anomaly detection
  • Banking Domain Expertise: Apply knowledge of banking processes, regulations, and workflows to ensure comprehensive test coverage especially with Disputes workflow
  • Collaborate with business analysts and product owners to understand banking requirements and translate them into effective test strategies
  • Quality Assurance: Analyze requirements of projects, develop test plans, design test cases for broad range of systems and applications and ensure they meet specified standards and end-user requirements
  • Ensure end-to-end quality of banking applications by identifying risks, creating test plans, and executing test cases
  • Fulltime
Read More
Arrow Right

Senior Trust Analyst

The Trust Engagement team is a globally distributed group of Security and Risk p...
Location
Location
Poland , Gdańsk
Salary
Salary:
Not provided
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7+ years experience working with customers across Security, Governance, Risk and Compliance domain areas
  • Expertise to lead discussions with decision makers about security and risk trade-offs
  • Direct customer-facing or consulting engagement experience
  • Success with customer, executive, or board member briefings on strategy, including impactful and effective presentations
  • Knowledge of security, compliance, and privacy frameworks & standards (e.g., GDPR, HIPAA, SOC2, ISO27001, NIST 800-53, PCI-DSS, C5, DORA, NIS2)
  • Experience working with Regulatory Compliance, Global Privacy Programs, Cloud Security or related functions
  • Familiarity with large-scale enterprise SaaS platforms and challenges across security, privacy and compliance related to customer trust
  • Experience with cybersecurity principles and controls, including their application in a SaaS context
  • Understanding infrastructure and delivery models of SaaS products
  • Proficient knowledge of cloud platforms (e.g., AWS, Azure, Google Cloud) and their services
Job Responsibility
Job Responsibility
  • Represent Atlassian with our Strategic & Enterprise customers to manage trust-related questions and concerns for customer acquisition, customer migration, and long term customer retention
  • Be accountable for engaging in a variety of Atlassian customer facing initiatives. Your primary focus will involve helping our customers gain an understanding of Atlassian’s commitment to Trust by articulating our Security practices, Compliance posture, Privacy measures, Resilience efforts and Anti-Abuse program
  • Manage complex customer security reviews & audits
  • Support Atlassian’s legal teams in negotiating security and compliance terms & exhibits for Atlassian Customer Agreements
  • Represent the Voice of the Customer by identifying deal blockers and friction points to influence Security, Product & Engineering strategy in meeting customer expectations
  • Drive customer facing Trust collateral by publishing publicly facing materials, presentations, white papers
  • Provide security, compliance, and privacy education and training to internal partner teams in Sales, Product Marketing, Legal, etc.
  • Mentor team through technical leadership and knowledge sharing.
What we offer
What we offer
  • Health and wellbeing resources
  • Paid volunteer days
  • Wide range of perks and benefits designed to support you and your family.
Read More
Arrow Right
New

Casual Sales Assistant

As a Sales Assistant at Evans Cycles, you’ll play a key role in delivering a gre...
Location
Location
United Kingdom , Wigan
Salary
Salary:
Not provided
frasers.group Logo
Frasers Group
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Customer-focused with strong communication skills
  • Enthusiastic about cycling
  • A team player who thrives in a fast-paced retail environment
  • Reliable, flexible, and eager to learn
Job Responsibility
Job Responsibility
  • Welcoming and engaging customers, offering friendly and professional advice
  • Developing strong product knowledge across bikes, accessories, and clothing
  • Supporting sales through upselling and cross-selling to meet customer needs
  • Assisting with stock management and keeping the shop floor organised
  • Handling payments accurately and efficiently at the till
  • Maintaining high standards of merchandising and store presentation
  • Promoting safe cycling practices and helping customers choose the right safety gear
What we offer
What we offer
  • We’ll provide training to set you up for success
  • Great career opportunities across our retail network
Read More
Arrow Right
New

Room Attendant

As a Room Attendant at Cairn Group, you’ll be a vital part of the guest experien...
Location
Location
United Kingdom , Carlisle
Salary
Salary:
Not provided
Redworth Hall Hotel
Expiration Date
February 10, 2026
Flip Icon
Requirements
Requirements
  • A Hospitality Hero – You help create relaxing, spotless rooms that feel like a true escape for every guest
  • A Service Superstar – You take pride in every detail, creating a consistent and polished experience
  • A Team Player – You work independently and support your colleagues with flexibility and respect
  • Cool Under Pressure – You manage your time well and adapt to changes in your daily routine
  • A Polished Professional – You bring a courteous, reliable presence to every interaction
  • Proactive & Organised – You prepare ahead, minimise waste, and report issues promptly
  • Experienced & Reliable – You’ve worked in housekeeping or a similar role and have strong communication and time management skills.
Job Responsibility
Job Responsibility
  • Creating clean, comfortable, and welcoming spaces for work, rest, or play
  • Preparing rooms and public areas to the highest standard
  • Changing bed linen and cleaning bathrooms
  • Replenishing amenities and restocking mini-bars
  • Supporting deep cleans and special projects as needed
  • Planning your day with care and managing supplies effectively
  • Lending a hand when team support is needed elsewhere.
What we offer
What we offer
  • Service Excellence Bonus
  • Get Paid in Real-Time with Wagestream
  • Stay, Relax and Recharge: Enjoy discounted stays across our all our venues
  • Delicious Discounts: 50% off food and drinks across our hotels, bars and restaurants
  • Spa Discounts
  • Retail, Cinema & Fun Discounts
  • Rise with Us: development programs
  • Superstar Awards
  • Long-Service Awards
  • Recommend a Friend bonus
  • Parttime
Read More
Arrow Right
New

Casual Sales Assistant

As a Casual Sales Assistant, you’ll play a key role in delivering a high-energy,...
Location
Location
United Kingdom , Bury St Edmunds
Salary
Salary:
Not provided
frasers.group Logo
Frasers Group
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Customer-focused with a passion for retail
  • Confident, friendly, and a strong communicator
  • Flexible and adaptable to business needs
  • Driven to achieve goals and contribute to team success
  • Proud to represent the Sports Direct brand and values
  • Available to work a range of shifts, including evenings, weekends, and holidays
Job Responsibility
Job Responsibility
  • Engage with every customer to deliver outstanding service
  • Use product knowledge to provide tailored recommendations
  • Actively contribute to achieving and exceeding store targets
  • Maintain store presentation through stock replenishment and organisation
  • Support visual merchandising standards in line with the Sports Direct brand
  • Assist with deliveries and stock processing
  • Ensure pricing is accurate and up to date
  • Support stock counts and inventory accuracy
What we offer
What we offer
  • Competitive hourly rate + sales commission
  • Flexible working hours to fit around your lifestyle
  • Monthly group rewards and recognition
  • Uniform discount and 20% discount across Frasers Group brands
  • Discounted gym membership
  • Career development opportunities, including nationally recognised qualifications and internal training programmes
  • A fast-paced, supportive team environment
  • Parttime
Read More
Arrow Right