CrawlJobs Logo

Delivery Quality Engineer, AI Business

Mexico, Mexico City · Job Posted January 07, 2026
Apply Position
Job Link Share

Job Description

As a Delivery Quality Engineer within Prolific AI Data Services, you will be the quality guardian for our managed service studies. You will design and operationalise the measurement systems, automation, and launch gates that ensure the data we deliver is trustworthy, authentic, and scalable. This role sits at the intersection of data quality, automation, and integrity. You’ll work closely with Product, Engineering, Operations, and Client teams to embed quality and authenticity into study design and execution—enabling faster launches without compromising trust as task types and volumes evolve.

Job Responsibility

  • Own end-to-end quality design for Prolific managed service studies, including rubrics, acceptance criteria, defect taxonomies, severity models, and clear definitions of done
  • Define, implement, and maintain quality measurement systems, including sampling plans, golden sets, calibration protocols, agreement targets, adjudication workflows, and drift detection
  • Build and deploy automated quality checks and launch gates using Python and SQL, such as schema and format validation, completeness checks, anomaly detection, consistency testing, and label distribution monitoring
  • Design and run launch readiness processes, including pre-launch checks, pilot calibration, ramp criteria, full-launch thresholds, and pause/rollback mechanisms
  • Partner with Product and Engineering to embed in-study quality controls and authenticity checks into workflows, tooling, and escalation paths
  • Write and continuously improve guidelines and training materials to keep participants, reviewers, and internal teams aligned on evolving quality standards
  • Investigate quality and integrity issues end to end, running root-cause analysis across guidelines, UX, screening, training, and operations, and driving corrective and preventive actions (CAPAs)
  • Build dashboards and operating cadences to track defect rates, rework, throughput versus quality trade-offs, integrity events, and SLA adherence
  • Lead calibration sessions and coach QA leads and reviewers to improve decision consistency, rubric application, and overall quality judgement
  • Translate one-off quality fixes into repeatable, scalable playbooks across customers, programs, and study types

Requirements

  • 5+ years of experience in quality engineering, data or annotation quality, analytics engineering, trust and integrity, or ML/LLM evaluation operations
  • Strong proficiency in Python and SQL, with comfort applying statistical concepts such as sampling strategies, confidence levels, and agreement metrics
  • A proven track record of turning ambiguous or messy quality problems into clear metrics, automated checks, and durable process improvements
  • Strong quality systems thinking, with the ability to translate complex edge cases into clear rules, tests, rubrics, and governance mechanisms
  • Hands-on experience instrumenting workflows and implementing pragmatic automation that catches quality and integrity issues early
  • Demonstrated ability to influence cross-functional teams (Product, Engineering, Operations, Client teams) and drive change without direct authority
  • Strong customer empathy, with a clear understanding of what “useful, trustworthy data” means for research, AI training, and evaluation use cases

Nice to have

  • Familiarity with data collection mechanics (screeners, quota/routing constraints, study design patterns)
  • LLM evals, red teaming, or policy-based annotation experience
  • Data/versioning discipline (dataset lineage, change control, reproducibility)
  • Experience with integrity/fraud detection systems and anti-abuse tooling

What we offer

  • competitive salary
  • benefits
  • remote working
  • impactful, mission-driven culture
  • equity
  • opportunity to earn a cash variable element, such as a bonus or commission

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Delivery Quality Engineer, AI Business

8 matching positions

Head of Quality, AI Business

As Head of Quality, AI Data, you will own the systems that turn messy human and ...
Location
Location
Mexico , Mexico City
Salary
Salary:
Not provided
prolific.com Logo
Prolific
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8–12+ years of experience in quality, data operations, trust and safety, or ML evaluation, with 3+ years leading managers or senior leads
  • Proven experience running quality programs across globally distributed teams and multiple regions
  • Strong statistical intuition (sampling, agreement, bias/variance) and comfort working with quality data
  • Fluency with SQL and dashboards
  • working knowledge of Python or R for analysis or automation
  • Deep understanding of human-in-the-loop systems and how data quality impacts model behaviour
  • Strong judgment balancing speed, cost, and quality, and the ability to make trade-offs explicit
  • Clear, confident communication with both engineers and executives
Job Responsibility
Job Responsibility
  • Set the global quality strategy across annotation, human feedback, model evaluation, red-teaming, and multilingual data
  • Define and enforce quality standards, acceptance criteria, and playbooks across project and pipeline types
  • Own the quality measurement system, including sampling strategies, agreement metrics, defect taxonomies, and drift detection
  • Lead a globally distributed quality organisation and design operating rhythms that work asynchronously across regions
  • Ensure quality is embedded from project kickoff through delivery, including audits, remediation loops, and root-cause analysis
  • Act as the senior quality partner for customers, clearly explaining methodology, results, risks, and improvement plans
  • Partner with Product, Engineering, and Operations to embed quality into tooling, workflows, and automation
  • Hire, develop, and lead quality managers and senior individual contributors, building a culture of rigor and accountability
What we offer
What we offer
  • competitive salary
  • equity
  • benefits
  • remote working
  • impactful, mission-driven culture
  • opportunity to earn a cash variable element, such as a bonus or commission
Read More
Arrow Right

Digital Engineer Service Delivery AI

We are seeking a dynamic and experienced Digital Engineering Service Delivery Le...
Location
Location
Ireland , Dublin
Salary
Salary:
Not provided
vodafone.com Logo
Vodafone
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years’ experience in digital service delivery including delivery of complex platforms or services
  • Strong working knowledge of Agile delivery frameworks (Scrum, Kanban, SAFe, Lean)
  • Proven experience delivering digital products, platforms, or services in an Agile environment
  • Experience working on or alongside AI-enabled solutions, such as machine learning models, intelligent automation, analytics platforms, or generative AI use cases
  • Excellent stakeholder management and communication skills, with the ability to bridge business, technical, and AI teams
  • Strong problem-solving skills and adaptability in fast-paced, evolving environments
Job Responsibility
Job Responsibility
  • Partner closely with the Digital Engineering Lead and senior stakeholders to define and execute digital and GenAI / Agentic AI transformation roadmaps
  • Work with stakeholders to ensure delivery timelines, quality standards, and service outcomes are met in alignment with business and strategic objectives
  • Act as a key liaison between technical, AI, and business teams, ensuring clear communication, alignment, and transparency
  • Collaborate with Product Owners and Digital Leads to prioritise backlogs, manage dependencies, and remove delivery impediments
  • Track and report on delivery metrics (e.g. velocity, burn-down charts, flow metrics) to support continuous delivery and improvement
  • Oversee service transition into production, including AI model deployment, and coordinate operational handover to ensure full support and monitoring readiness
  • Support incident and problem resolution in collaboration with DevOps, Integration, QA, Ops, and support teams
  • Work with our Responsible AI Team to promote responsible AI practices, including awareness of model performance, bias, explainability, and compliance considerations
  • Foster a culture of collaboration, accountability, innovation, and continuous learning across delivery teams
What we offer
What we offer
  • Competitive Salary
  • Flexible working (times and location) is part of our culture where your hours can facilitate your personal work-life balance
  • Generous Employer Pension contribution
  • Free Phone Bill – When you join the team, your phone bill is on us! You will also receive mobile phone purchasing discounts
  • Learning and Development – We provide extensive training to all of our employees so they can progress in their careers! There is also the Vodafone University where our employees can gain incredible skills!
  • Exclusive Family and Friends Discounts – Receive additional discounts for you, your Family and Friends on Vodafone Products!
  • 23 days annual leave and 3 charity days off – PLUS a dedicated day off for your birthday
  • Access to our Wellness centre including free access to our Gym, health and wellness centre and external practitioners
Read More
Arrow Right

Quality Analyst, AI Business

As a Quality Analyst within Prolific AI Data Services, you will play a critical ...
Location
Location
Mexico , Mexico City
Salary
Salary:
Not provided
prolific.com Logo
Prolific
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 1–3 years of experience in analysis, quality, operations, or data-focused roles, or equivalent academic projects or internships
  • A background in computer science, data science, statistics, engineering, or a related quantitative discipline
  • Strong SQL skills (required), with the ability to independently query, join, and summarise datasets to answer quality and operational questions
  • Working proficiency in Python and/or R, with experience writing scripts or notebooks to clean data, compute metrics, and automate checks
  • Strong analytical rigour, with a high bar for precision, consistency, and clear documentation of decisions
  • Solid statistical fundamentals, including intuition for sampling, distributions, confidence, and agreement concepts
  • Clear and structured communication skills, with the ability to explain issues using concrete examples and propose practical fixes
  • A collaborative mindset and willingness to partner cross-functionally to improve quality outcomes and delivery reliability
Job Responsibility
Job Responsibility
  • Run day-to-day quality measurement for managed service studies, including sampling, rubric-based review, and structured reporting
  • Apply rubrics and guidelines consistently, flag ambiguities, and propose clear, actionable clarifications as task types evolve
  • Maintain and update golden sets, and support calibration efforts by preparing materials, tracking outcomes, and documenting decisions
  • Track defects and failure modes by categorising issues, assessing severity, and keeping defect taxonomies current
  • Run automated quality checks using SQL and Python or R, including completeness checks, schema and format validation, anomaly detection, and label distribution monitoring
  • Support launch gates by preparing readiness artefacts such as rubric versioning, gold status, sampling plans, and early pilot findings
  • Investigate quality drops by pulling examples, quantifying scope, and supporting root-cause analysis and corrective and preventive actions (CAPAs)
  • Build dashboards and weekly updates covering defect rates, rework, agreement, throughput versus quality trade-offs, and SLA risk
  • Partner with Operations, reviewers, and—when needed—Product and Engineering to improve task design, reviewer workflows, and escalation paths
What we offer
What we offer
  • competitive salary, benefits, and hybrid working within our impactful, mission-driven culture
  • equity
  • opportunity to earn a cash variable element, such as a bonus or commission
Read More
Arrow Right

AI-Augmented Functional Engineer (Senior Business Analyst)

AI-augmented techno-functional role within Market Risk focused on delivering P&L...
Location
Location
Poland , Warsaw
Salary
Salary:
Not provided
talan.com Logo
Talan
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Degree in Computer Science, Engineering, or similar
  • 5+ years in business analysis or techno-functional roles in financial services
  • Strong understanding of data modeling and process design for financial applications, especially financial derivatives/P&L
  • High level in SQL, and experience with data analysis / ETL
  • Experience with AI-assisted development tools (or strong willingness to learn)
  • Experience translating requirements into functional designs and data models
  • Ability to read and review code (Python or similar)
  • Strong analytical skills and attention to data quality
  • Strong communication skills in English and ability to work with cross-functional agile teams (SCRUM)
  • Experience in production support and issue resolution
Job Responsibility
Job Responsibility
  • Translate business and technical needs into user stories and functional designs using AI tools (e.g., Rovo)
  • Design data flows and processes for P&L systems, ensuring business value
  • Collaborate with engineers, leveraging AI tools (Copilot, Devin, Windsurf) to support design, build, and validation
  • Review and guide AI-generated solutions with senior developers
  • Own end-to-end delivery in Agile (backlog, sprints, documentation in Jira/Confluence)
  • Ensure strong end-to-end solution quality, defining test approach
  • Handle production issues, including incident triage and root cause analysis
What we offer
What we offer
  • Smart Office Pack so that you can work comfortably from home
  • Training and career development
  • Private medical insurance
  • Life insurance
  • Possibility to be part of a multicultural team and work on international projects
  • Fulltime
Read More
Arrow Right

AI-Augmented Functional Engineer (Senior Business Analyst)

AI-augmented techno-functional role within Market Risk focused on delivering P&L...
Location
Location
Spain , Málaga
Salary
Salary:
Not provided
talan.com Logo
Talan
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Degree in Computer Science, Engineering, or similar
  • 5+ years in business analysis or techno-functional roles in financial services
  • Strong understanding of data modeling and process design for financial applications, especially financial derivatives/P&L
  • High level in SQL, and experience with data analysis / ETL
  • Experience with AI-assisted development tools (or strong willingness to learn)
  • Experience translating requirements into functional designs and data models
  • Ability to read and review code (Python or similar)
  • Strong analytical skills and attention to data quality
  • Strong communication skills in English and ability to work with cross-functional agile teams (SCRUM)
  • Experience in production support and issue resolution
Job Responsibility
Job Responsibility
  • Translate business and technical needs into user stories and functional designs using AI tools (e.g., Rovo)
  • Design data flows and processes for P&L systems, ensuring business value
  • Collaborate with engineers, leveraging AI tools (Copilot, Devin, Windsurf) to support design, build, and validation
  • Review and guide AI-generated solutions with senior developers
  • Own end-to-end delivery in Agile (backlog, sprints, documentation in Jira/Confluence)
  • Ensure strong end-to-end solution quality, defining test approach
  • Handle production issues, including incident triage and root cause analysis
What we offer
What we offer
  • Smart Office Pack so that you can work comfortably from home
  • Training and career development
  • private medical insurance
  • life insurance
  • Language lessons
  • Possibility to be part of a multicultural team and work on international projects
  • Fulltime
Read More
Arrow Right
New

Ai Engineer (Ai-native Development & Claude-based Systems)

As an AI Engineer, you play a key role in building and running the technical bac...
Location
Location
Romania , Cluj
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of professional experience in software engineering, backend, or platform engineering roles
  • Strong software engineering background with hands-on experience in modern architectures
  • Experience in the insurance domain, with exposure to insurance platforms, business processes, or core systems (e.g. policies, claims, billing, underwriting, integrations)
  • Familiarity with AI‑native engineering practices, such as agent‑based or orchestration‑driven approaches (e.g. Gastown‑style concepts)
  • Hands-on experience with Claude or similar large language models, including: Using AI tools directly within an IDE or development workflow
  • Writing effective prompts for code analysis, code generation, and iterative refinement
  • Solid understanding of modern engineering patterns, including: API design and integration
  • Microservices architectures
  • Event‑driven and integration‑based systems
  • Experience working with existing, non‑trivial codebases and enabling AI‑driven analysis on top of them
Job Responsibility
Job Responsibility
  • Implement and maintain the AI backbone that supports end‑to‑end delivery
  • Configure, train, and orchestrate AI models and agents working directly on existing codebases
  • Enable automated gap analysis, requirement refinement, and specification generation using AI
  • Integrate AI capabilities into real-world software systems, not isolated PoCs
  • Ensure AI‑driven outputs are robust, scalable, and accurate enough for enterprise use
  • Continuously improve prompts, workflows, and agent coordination to increase quality and consistency
  • Work closely with the Intent Engineer to iterate on outputs and ensure alignment with business intent
  • Support a shift from traditional requirement creation to AI‑assisted, exploration‑driven delivery
What we offer
What we offer
  • Smooth integration and a supportive mentor
  • Pick your working style: Remote, Hybrid or Office work opportunities
  • Flexible working hours
  • Sponsored certifications, trainings and top e-learning platforms
  • Private Health Insurance
  • Individual coaching sessions
  • Accredited Coaching School
  • Epic parties or themed events
  • Fulltime
Read More
Arrow Right

Lead AI Engineer (Generative AI)

Location
Location
Australia , Melbourne
Salary
Salary:
160000.00 - 170000.00 AUD / Year
welovesalt.com Logo
Salt
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2 – 4 years experience applying modern Generative AI in real delivery environments
  • A strong track record of providing technical leadership
  • A strong background in software engineering before stepping into a more AI-focused role in recent times
  • Deep expertise in Generative AI, large language models, retrieval‑augmented generation (RAG), and modern machine learning techniques
  • Strengths in prompt engineering, offline/online evaluation, safety guardrails, and telemetry‑driven improvement
  • Practical experience with RAG, embeddings/vector search, and tool‑use/function‑calling orchestration
  • The ability to define trustworthy AI metrics
  • A Software engineering background using the likes of React, Typescript, React Native
  • AWS experience / certifications
  • Strong knowledge of CI/CD
Job Responsibility
Job Responsibility
  • Accelerating the adoption of Generative AI for this company, leading the adoption of AI‑first engineering practices and an 'AI-first' mindset
  • Providing technical leadership and shaping AI-enabled solutions
  • Reporting into the Head of Technology and working closely with the Head of Enterprise AI
  • Collaborating with a variety of cross-functional teams including
  • Engineering, QA, BA, Product Management, Senior Business Stakeholders, Data Science, Cyber Security, Enterprise Architecture
  • Coaching and training engineers and leaders in the organisation in safe, effective AI‑assisted development
  • Encouraging teams to embed AI‑driven thinking in solution design and delivery
  • Defining and helping to operationalise trustworthy metrics to measure the impact of AI features, and developer productivity uplifts
  • Producing AI-augmented solution designs when required
  • Driving engineering excellence by uplifting standards, modernising technical practices, and championing AI‑aligned SDLC patterns
What we offer
What we offer
  • Super
  • Bonus
  • Fulltime
Read More
Arrow Right

Full Stack Engineer (AI & Agentic AI Systems)

The Full Stack Engineer (AI & Agentic AI Systems) is a strategic professional wh...
Location
Location
India , Pune; Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2-5 years in an Apps Development role
  • Demonstrated execution capabilities
  • Strong analytical and quantitative skills
  • Data driven and results-oriented
  • Experience in running high traffic, distributed, cloud based services
  • Experience in affecting large culture change
  • Experience leading infrastructure programs
  • Skilled at working with third party service providers
  • Excellent written and oral communication skills
  • Bachelor’s/University degree or equivalent experience
Job Responsibility
Job Responsibility
  • Design and deliver end‑to‑end solutions spanning architecture, system design, low‑level design, and high‑quality coding across modern full‑stack environments
  • Build responsive, modular UI applications using React, integrating complex AI-driven workflows and real‑time interactions
  • Develop scalable, high‑performance backend services in Java / Python, implementing resilient APIs, event‑driven patterns, and microservices architectures
  • Engineer AI‑powered features leveraging Google Gemini LLM, Vertex AI, ADK, vector databases (A2A), RAG pipelines, MCP, context engineering, and advanced prompt engineering techniques
  • Implement secure, well‑structured REST and GraphQL APIs, ensuring reliability, versioning discipline, and clean integration patterns across platforms
  • Optimize system performance and scalability, applying profiling, load‑testing insights, caching strategies, and distributed system tuning
  • Drive robust CI/CD practices, integrating automated testing, code quality gates, containerization, and cloud‑native deployment pipelines
  • Partner with QE to build and maintain automated test suites (UI, API, integration, and performance), improving release quality and reducing regression risk
  • Identify, diagnose, and remediate performance bottlenecks, penetration testing vulnerabilities, and production issues with precision and root‑cause clarity
  • Collaborate cross‑functionally with AI scientists, architects, and product teams to translate business challenges into production‑ready, intelligent agentic systems
  • Fulltime
Read More
Arrow Right