CrawlJobs Logo
Barclays Logo Barclays · IT - Software Development

Ai Evaluation & Assurance Architect

United Kingdom, Knutsford · Job Posted May 17, 2026
Apply Position
Job Link Share

Job Description

Join us at Barclays as an AI Evaluation & Assurance Architect -We’re looking for a forward-thinking architect to shape how AI is evaluated, governed, and trusted across the enterprise. This is a foundational position within our AI Accelerator, focused on building and scaling frameworks that ensure AI systems are reliable, trustworthy, and defensible. You’ll define evaluation standards, assurance processes, and reference architectures across Generative AI, Agentic AI, and AI/ML—supporting their full lifecycle from design through to live operation. Working across architecture, engineering, risk, and governance, you’ll enable rapid AI adoption while maintaining control, transparency, and confidence.

Job Responsibility

  • Shape how AI is evaluated, governed, and trusted across the enterprise
  • Build and scale frameworks that ensure AI systems are reliable, trustworthy, and defensible
  • Define evaluation standards, assurance processes, and reference architectures across Generative AI, Agentic AI, and AI/ML
  • Support their full lifecycle from design through to live operation
  • Enable rapid AI adoption while maintaining control, transparency, and confidence
  • Provision of guidance and expertise to engineering teams to ensure alignment with best practices and foster a culture of technical excellence
  • Contribution to strategic planning by aligning technical decisions with business goals, anticipating future technology trends, and providing insights to optimize product roadmaps
  • Design and implementation of complex, scalable, and maintainable software solutions, considering long-term viability and business objectives
  • Mentoring and coaching to junior and mid-level engineers to foster professional growth and knowledge sharing
  • Collaboration with business partners, product managers, designers, and other stakeholders to translate business requirements into technical solutions and ensure a cohesive approach to product development
  • Innovation within the organization by identifying and incorporating new technologies, methodologies, and industry practices into the engineering process

Requirements

  • Evaluation architecture for probabilistic AI: Design robust evaluation strategies for non-deterministic systems (e.g. LLMs), including test datasets, scoring methods, and release gates
  • Risk-based assurance design: Translate AI risks into effective controls, covering areas like bias, hallucination, adversarial testing, and data leakage
  • Operationalisation (LLMOps & monitoring): Embed evaluation into production through continuous monitoring, observability, CI/CD gating, and auditability
  • Simulation & synthetic data: Create realistic scenarios and stress tests to improve coverage and resilience
  • Statistics & measurement: Apply sampling, confidence metrics, and evaluation consistency techniques
  • Governance literacy: Produce assurance evidence, align with model risk frameworks, and support independent validation practices

Nice to have

  • Simulation & synthetic data: Create realistic scenarios and stress tests to improve coverage and resilience
  • Statistics & measurement: Apply sampling, confidence metrics, and evaluation consistency techniques
  • Governance literacy: Produce assurance evidence, align with model risk frameworks, and support independent validation practices

What we offer

  • Competitive holiday allowance
  • Life assurance
  • Private medical care
  • Pension contribution
Barclays - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Ai Evaluation & Assurance Architect

8 matching positions

Architect - AI Business Solutions

Microsoft Industry Solutions Delivery (ISD) is a global organization hosting ove...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Information Technology, Engineering, or related field AND 14+ years in technology solutions, architecture, consulting, or specialized technical domains (e.g., Security)
  • Master's degree in related field AND 12+ years in technology solutions, architecture, consulting, or specialized technical domains
  • Equivalent experience
  • Experience with a Global Systems Integrator, IT Vendor, or Microsoft Partner delivering enterprise-scale Microsoft Business Applications
  • Strong hands-on background in Business Applications and CRM solution delivery across medium to large enterprises
Job Responsibility
Job Responsibility
  • Understanding Customer Technical Environment
  • Engage business sponsors, product owners, Enterprise Architects, Security, and Operations leaders to frame outcomes and define the target agentic scenarios
  • Assess customer AI readiness across identity, data, security/compliance, and operating model
  • Identify industry and technology trends including AI adoption patterns, Low-Code modernization, Copilot usage, GitHub Copilot assisted development, and data readiness
  • Translate business requirements into end-to-end solution options across Power Platform, Azure, and Microsoft 365 Copilot/Copilot Studio extensibility
  • Socialize solution approaches with customers, validating feasibility, operating impact, scalability, and change management
  • Maintain an iterative feedback loop with Product Engineering and internal communities
  • Partner with account and delivery teams to shape the solution play, refine estimates, adjust pre-sales solution components, govern scope changes, and position technical risks
  • Architecture Design and Deployment
  • Apply structured delivery methodologies (Agile/Iterative/Hybrid) and reference architectures to design scalable Copilot and agentic solutions
  • Fulltime
Read More
Arrow Right

Chief AI Architect

We are currently seeking a Chief Architect to join our team in Noida, Haryāna (I...
Location
Location
India , Noida
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's/Master's degree in Computer Science, AI/ML, or related field
  • PhD preferred
  • Cloud architecture certifications (AWS Solutions Architect Professional, Azure Solutions Architect Expert, GCP Professional Cloud Architect)
  • 15+ years in software/platform architecture with 8+ years specializing in AI/ML systems
  • Proven experience architecting 3+ large-scale AI/ML platforms serving 1000+ users
Job Responsibility
Job Responsibility
  • Platform Architecture Design: Own end-to-end AI/ML platform architecture across data ingestion, processing, ML training, inference, MLOps, and GenAI layers. Design for 1000+ concurrent users and PB-scale data
  • Reference Architecture Development: Define comprehensive reference architectures for datasets, models, feature stores, model registries, metadata management, orchestration, and AI services
  • Technology Selection: Evaluate and recommend technologies, frameworks, and tools. Ensure cloud-neutral, open-standards-based design supporting on-prem, cloud, and hybrid deployments
  • GenAI & Advanced AI: Guide architectural decisions for GenAI, LLM fine-tuning, RAG (Retrieval Augmented Generation), vector databases, intelligent search, and conversational AI
  • MLOps & Model Lifecycle: Design CI/CD pipelines for ML models, A/B testing frameworks, model monitoring, drift detection, and automated retraining mechanisms
  • Security & Compliance: Ensure alignment with security, compliance, data sovereignty, and regulatory requirements (GDPR, data localization, etc.). Design for zero-trust architecture
  • Data Architecture: Design data lake, lakehouse, and data mesh architectures. Define data governance, lineage, quality, and metadata management frameworks
  • Integration & APIs: Design API strategies, microservices architecture, event-driven patterns, and integration with enterprise systems (ERP, CRM, etc.)
  • Performance & Scalability: Architect for high availability (99.9%+), low latency
  • Technical Leadership: Provide architectural oversight and technical assurance throughout delivery. Conduct architecture reviews, design sessions, and POCs
Read More
Arrow Right

Cloud Solution Architect - AI Workforce

The AI Workforce team is a global group of technical architects and strategists ...
Location
Location
United States , Multiple Locations
Salary
Salary:
85100.00 - 169800.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science, Information Technology, Engineering, Business, Liberal Arts, or related field AND 2+ years experience in cloud/infrastructure technologies, information technology (IT) consulting/support, systems administration, network operations, software development/support, technology solutions, practice development, architecture, and/or consulting
  • OR equivalent experience
  • Microsoft is unable to sponsor a work visa for this role due to the nature of the role’s job duties
  • 2+ years experience working in a customer-facing role (e.g., internal and/or external)
  • 2+ years experience working on technical projects
  • Technical Certification in Cloud (e.g., Azure, Amazon Web Services, Google, security certifications)
  • Advanced experience in Azure/M365 administration, development, and DevOps processes
  • Proven expertise in Azure AI workloads, Copilot extensibility, and agentic frameworks
  • Strong understanding of evaluation methodologies for AI systems (offline and real-time)
  • Proficiency in TypeSpec, Microsoft 365 Agents Toolkit, and orchestration patterns
Job Responsibility
Job Responsibility
  • Architect and deploy AI agent solutions leveraging Microsoft Copilot Studio, Agent Builder, and declarative frameworks
  • Define evaluation strategies for agent performance, safety, and compliance using metrics such as task success rate, latency, and failure rate
  • Mentor peers and guide readiness efforts aligned to corporate priorities like security, resilience, and AI innovation
  • Generate and evolve IP for emerging AI scenarios and represent Microsoft in technical communities through events, presentations, and content creation
  • Act as the voice of the customer, synthesizing insights to influence product roadmaps and program improvements
  • Translate customer challenges into solutions aligned with Microsoft’s platform strategy, ensuring clear articulation of ROI and business outcomes
  • Anticipate dissatisfaction, define conditions of success, and ensure a seamless, connected customer experience that accelerates adoption and transformation
  • Deliver operational excellence across engagements by applying governance principles and managing risk proactively
  • Execute value-based deliverables and prepare customers for operational readiness, long-term adoption, and well-architected, AI-enabled solutions
  • Identify and capture new opportunities for consumption and usage, collaborating across teams to accelerate deployment and ensure measurable customer impact
  • Fulltime
Read More
Arrow Right

Security Architect - AI

The AI Security Architect role focuses on leveraging expertise in information an...
Location
Location
United Kingdom , Birmingham
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum of 7 years of experience
  • Strong consulting background
  • Experience in defining enterprise security architecture
  • Experience in managing security risks
  • Experience in ensuring compliance with industry standards
  • Relevant certifications such as Open CA and TOGAF
  • Multi-year client-facing project and presentation experience in a professional services environment
  • Ability to operate in demanding situations
  • Strong influencer with ability to build good working relationships
  • Must be able to interface with and build relationships with Enterprise Architects, business people, and technologists at all levels
Job Responsibility
Job Responsibility
  • Work in the capabilities phases of the systems development life cycle
  • unpacking and translating technology security requirements, business and environmental conditions into enterprise security architecture patterns and processes
  • Identify Business objectives, goals and strategy
  • Identify Business attributes that are required to achieve those goals
  • Identify Enterprise
  • assets, systems, and information processes to support enterprise mission needs
  • Identify Stakeholder security requirements for protecting the organization’s mission and business processes
  • Identify The risk associated with the attributes that can prevent a business from achieving its goals
  • Identify The required controls to manage the risk
  • Identify assumptions, issues and dependencies
What we offer
What we offer
  • Tailored benefits that support your physical, emotional, and financial wellbeing
  • Continuous growth and development opportunities
  • Flexible work options
Read More
Arrow Right

Ai Security Architect

We are currently seeking a AI Security Architect to join our team in Bangalore o...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years in cybersecurity architecture with proven experience securing large-scale LLM deployments and multi-agent workflows
  • Hands-on capability with agent frameworks (e.g., LangChain, LangGraph, AutoGen) and MLOps platforms
  • Deep familiarity with model risk management principles and AI security standards
Job Responsibility
Job Responsibility
  • Defining security architecture and implementing robust security controls for AI/ML systems and their underlying platforms
  • Serving as the team’s technical mentor and architecture authority, driving secure-by-design patterns across the AI/ML lifecycle (data, training, evaluation, deployment, and production monitoring)
  • Proactively mitigating AI-specific threats such as model integrity risks, data poisoning, adversarial attacks, prompt injection, model extraction, and inference-time abuse
  • Leading technically, setting standards, and guiding engineers day-to-day through architecture, reviews, and delivery
  • Ensuring AI systems are secure, compliant, and resilient by implementing data protection, threat detection, guardrails, and ongoing risk monitoring across the AI lifecycle
  • Agent Security: Define strict Role-Based Access Control (RBAC) and least-privilege models for AI agents
  • Design runtime environments with restricted permissions
  • Implement defenses against adversarial attacks, prompt injections, jailbreaking, and sensitive data leakage (DLP) across agent workflows
  • Observability & Monitoring: Architect logging and monitoring standards for decision traceability
  • Monitor models and prompt templates for behavioral drift, anomalies, and attacks
  • Fulltime
Read More
Arrow Right
New

Agentic Platform Architect

We are looking for an experienced Agentic Platform Architect to lead the archite...
Location
Location
Poland; United Kingdom
Salary
Salary:
Not provided
itransition.com Logo
Itransition
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 12+ years of overall experience in software engineering, platform engineering, or enterprise technology
  • Proven experience in platform architecture and designing enterprise-scale cloud-native platforms
  • Strong knowledge of microservices architecture, API-led integration, and distributed systems
  • Hands-on experience with public cloud platforms (AWS, Azure, or Google Cloud)
  • Solid understanding of security architecture, identity and access management, and enterprise security best practices
  • Experience with DevSecOps, CI/CD pipelines, and infrastructure automation
  • Experience designing AI/ML or Generative AI platforms in enterprise environments
  • Strong understanding of Large Language Models (LLMs), AI agents, orchestration frameworks, and Retrieval-Augmented Generation (RAG)
  • Experience establishing architecture standards, governance models, and reusable platform capabilities
  • Excellent stakeholder management, communication, and technical leadership skills
Job Responsibility
Job Responsibility
  • Define and govern the end-to-end architecture of the enterprise Agentic AI platform
  • Design the AI Gateway and model access architecture
  • Define identity, authentication, authorization, and token management patterns for AI agents
  • Establish platform governance, security guardrails, and responsible AI controls
  • Define the Agentic SDK, orchestration framework, and reusable engineering standards
  • Design knowledge management, memory, and Retrieval-Augmented Generation (RAG) architecture
  • Define observability, evaluation, monitoring, and quality assurance frameworks for AI agents
  • Design agent deployment, release, and lifecycle management processes
  • Drive platform adoption by establishing architecture standards, best practices, and governance
  • Provide technical leadership and collaborate with engineering teams to ensure consistent implementation of architectural decisions
What we offer
What we offer
  • Projects for such clients as PayPal, Wargaming, Xerox, Philips, Adidas and Toyota
  • Competitive compensation that depends on your qualification and skills
  • Career development system with clear skill qualifications
  • Flexible working hours aligned to your schedule
  • Options to work remotely
  • Corporate medical insurance covering services of private and public medical centers
  • English courses online
  • Corporate parties and events for employees and their children
  • Internal conferences, workshops and meetups for learning and experience sharing
  • Gym membership compensation
Read More
Arrow Right
New

Senior AI Developer

The AI Developer will be part of a team of experts charged with providing suppor...
Location
Location
United States , Dahlgren
Salary
Salary:
106480.81 - 185149.47 USD / Year
arcfield.com Logo
Arcfield
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • BS 8-10, MS 6-8, PhD 3-5
  • Must have an active Secret clearance and the ability to obtain and maintain a TS/SCI Clearance
  • DoDD 8140.01 Information Assurance Management (IAM) Level I certification or higher
  • Experience with Python programming language
  • Experience developing capabilities in software applications that interface with AI systems
  • Experience using frameworks for defining various AI workflows including but not limited to any of the following: Haystack, LangChain, LlamaIndex
  • Experience implementing retrieval augmented generation (RAG) workflows
  • Experience in context engineering: determining the inputs, shape, and size provided to AI endpoints, architecting enrichment steps to feed into the submission context, handling outputs received from AI calls, defining processes to ensure system prompts and guards are included in context before submission to the AI endpoints
  • Experience developing model context protocol (MCP) servers to allow AI agents to interface with software application APIs
  • Experience evaluating the performance of different models used by capabilities to identify the best model for a given capability
Job Responsibility
Job Responsibility
  • Working from requirements specifications to develop or modify software applications
  • Designing, coding, benchmark testing, debugging, and documenting application features and changes
  • Reviewing and assessing existing enterprise applications, to support new features, performance improvements, upgrades, and ongoing sustainment
  • Participating in performance, load, and stress testing, and remediation of issues identified
  • Participating in security architecture reviews of large enterprise systems
  • Participating in the development of new systems throughout the entire application lifecycle: requirements analysis & definition, system design, implementation, testing, deployment, and sustainment
  • Participating in activities that ensure the successful use of these systems including developing documentation, test scripts, and training materials
  • Working in an agile team environment participating in sprint planning, retrospectives, reviews, and daily standups
What we offer
What we offer
  • Health Insurance
  • Life Insurance
  • Paid Time Off
  • Holiday Pay
  • Short Term and Long-Term Disability
  • Retirement and Savings
  • Learning and Development opportunities
  • wellness programs
  • Fulltime
Read More
Arrow Right

Tech Lead / Solutions Architect (Python & GenAI)

Provectus is a global AI and cloud consulting company helping enterprises turn a...
Location
Location
Salary
Salary:
Not provided
provectus.com Logo
Provectus
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Full-stack mindset, comfortable across AI, backend development, and cloud infrastructure
  • Already using AI tools in your daily workflow (Claude Code, Copilot, or similar)
  • Proactive and self-directed
  • you own outcomes end-to-end and spot problems before they're handed to you
  • B2+ English, comfortable collaborating across distributed, multicultural teams
  • Owns the client technical relationship
  • leading discovery, decomposing ambiguous requirements into technical components, presenting architecture, and pushing back on scope when it doesn't match timeline or budget
  • Produces scoped, phased delivery plans with clear deliverables, dependencies, and risks
  • Experience with cost estimation and cloud architecture cost optimization
  • 7+ years building and running production systems not only demos and POCs
Job Responsibility
Job Responsibility
  • Write clean, production-grade Python across AI integrations, backend services, and RESTful APIs
  • Implement and optimize RAG systems for production use cases
  • Design and build LLM-based and agentic AI solutions that address real client business challenges
  • Own the technical direction of client engagements from discovery through delivery
  • Support presales: discovery calls, technical proposals, scoping, and client-facing demos
  • Lead architecture reviews, produce technical design documents, and contribute to standards across the Python practice
  • Mentor engineers, lead code reviews, and share knowledge across the team
  • Build and maintain strong relationships with key client stakeholders as a trusted technical advisor
What we offer
What we offer
  • Opportunity to work with cutting-edge AI and cloud solutions
  • Internal training programs (Leadership, Public Speaking, and more) with full support for AWS and other professional certifications
  • Career growth: a clear path toward SA or beyond
  • we actively develop our engineers
  • Access to the latest AI tools and premium subscriptions
  • Long-term B2B collaboration
  • Remote with flexible hours
  • Private medical insurance or a budget for your medical needs
  • Paid sick leave, vacation, and public holidays
  • Equipment and all the tech you need for comfortable, productive work
  • Fulltime
Read More
Arrow Right