CrawlJobs Logo

Ai Evaluation & Assurance Architect

barclays.co.uk Logo

Barclays

Location Icon

Location:
United Kingdom , Knutsford

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided
Save Job
Save Icon
Apply Position

Job Description:

Join us at Barclays as an AI Evaluation & Assurance Architect -We’re looking for a forward-thinking architect to shape how AI is evaluated, governed, and trusted across the enterprise. This is a foundational position within our AI Accelerator, focused on building and scaling frameworks that ensure AI systems are reliable, trustworthy, and defensible. You’ll define evaluation standards, assurance processes, and reference architectures across Generative AI, Agentic AI, and AI/ML—supporting their full lifecycle from design through to live operation. Working across architecture, engineering, risk, and governance, you’ll enable rapid AI adoption while maintaining control, transparency, and confidence.

Job Responsibility:

  • Shape how AI is evaluated, governed, and trusted across the enterprise
  • Build and scale frameworks that ensure AI systems are reliable, trustworthy, and defensible
  • Define evaluation standards, assurance processes, and reference architectures across Generative AI, Agentic AI, and AI/ML
  • Support their full lifecycle from design through to live operation
  • Enable rapid AI adoption while maintaining control, transparency, and confidence
  • Provision of guidance and expertise to engineering teams to ensure alignment with best practices and foster a culture of technical excellence
  • Contribution to strategic planning by aligning technical decisions with business goals, anticipating future technology trends, and providing insights to optimize product roadmaps
  • Design and implementation of complex, scalable, and maintainable software solutions, considering long-term viability and business objectives
  • Mentoring and coaching to junior and mid-level engineers to foster professional growth and knowledge sharing
  • Collaboration with business partners, product managers, designers, and other stakeholders to translate business requirements into technical solutions and ensure a cohesive approach to product development
  • Innovation within the organization by identifying and incorporating new technologies, methodologies, and industry practices into the engineering process

Requirements:

  • Evaluation architecture for probabilistic AI: Design robust evaluation strategies for non-deterministic systems (e.g. LLMs), including test datasets, scoring methods, and release gates
  • Risk-based assurance design: Translate AI risks into effective controls, covering areas like bias, hallucination, adversarial testing, and data leakage
  • Operationalisation (LLMOps & monitoring): Embed evaluation into production through continuous monitoring, observability, CI/CD gating, and auditability
  • Simulation & synthetic data: Create realistic scenarios and stress tests to improve coverage and resilience
  • Statistics & measurement: Apply sampling, confidence metrics, and evaluation consistency techniques
  • Governance literacy: Produce assurance evidence, align with model risk frameworks, and support independent validation practices

Nice to have:

  • Simulation & synthetic data: Create realistic scenarios and stress tests to improve coverage and resilience
  • Statistics & measurement: Apply sampling, confidence metrics, and evaluation consistency techniques
  • Governance literacy: Produce assurance evidence, align with model risk frameworks, and support independent validation practices
What we offer:
  • Competitive holiday allowance
  • Life assurance
  • Private medical care
  • Pension contribution

Additional Information:

Job Posted:
May 17, 2026

Employment Type:
Fulltime
Work Type:
On-site work
Icon
Barclays - All Job Offers
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon
PREMIUM
More languages and countries
Unlock 29494 hidden job offers
Languages
English Čeština Deutsch Ελληνικά Español Français +15
Countries
United States United Kingdom India Canada Australia +
See plans
Plans from $2.99 / month

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Ai Evaluation & Assurance Architect

Cloud Solution Architect - AI Workforce

The AI Workforce team is a global group of technical architects and strategists ...
Location
Location
United States , Multiple Locations
Salary
Salary:
85100.00 - 169800.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science, Information Technology, Engineering, Business, Liberal Arts, or related field AND 2+ years experience in cloud/infrastructure technologies, information technology (IT) consulting/support, systems administration, network operations, software development/support, technology solutions, practice development, architecture, and/or consulting
  • OR equivalent experience
  • Microsoft is unable to sponsor a work visa for this role due to the nature of the role’s job duties
  • 2+ years experience working in a customer-facing role (e.g., internal and/or external)
  • 2+ years experience working on technical projects
  • Technical Certification in Cloud (e.g., Azure, Amazon Web Services, Google, security certifications)
  • Advanced experience in Azure/M365 administration, development, and DevOps processes
  • Proven expertise in Azure AI workloads, Copilot extensibility, and agentic frameworks
  • Strong understanding of evaluation methodologies for AI systems (offline and real-time)
  • Proficiency in TypeSpec, Microsoft 365 Agents Toolkit, and orchestration patterns
Job Responsibility
Job Responsibility
  • Architect and deploy AI agent solutions leveraging Microsoft Copilot Studio, Agent Builder, and declarative frameworks
  • Define evaluation strategies for agent performance, safety, and compliance using metrics such as task success rate, latency, and failure rate
  • Mentor peers and guide readiness efforts aligned to corporate priorities like security, resilience, and AI innovation
  • Generate and evolve IP for emerging AI scenarios and represent Microsoft in technical communities through events, presentations, and content creation
  • Act as the voice of the customer, synthesizing insights to influence product roadmaps and program improvements
  • Translate customer challenges into solutions aligned with Microsoft’s platform strategy, ensuring clear articulation of ROI and business outcomes
  • Anticipate dissatisfaction, define conditions of success, and ensure a seamless, connected customer experience that accelerates adoption and transformation
  • Deliver operational excellence across engagements by applying governance principles and managing risk proactively
  • Execute value-based deliverables and prepare customers for operational readiness, long-term adoption, and well-architected, AI-enabled solutions
  • Identify and capture new opportunities for consumption and usage, collaborating across teams to accelerate deployment and ensure measurable customer impact
  • Fulltime
Read More
Arrow Right

Security Architect - AI

The AI Security Architect role focuses on leveraging expertise in information an...
Location
Location
United Kingdom , Birmingham
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum of 7 years of experience
  • Strong consulting background
  • Experience in defining enterprise security architecture
  • Experience in managing security risks
  • Experience in ensuring compliance with industry standards
  • Relevant certifications such as Open CA and TOGAF
  • Multi-year client-facing project and presentation experience in a professional services environment
  • Ability to operate in demanding situations
  • Strong influencer with ability to build good working relationships
  • Must be able to interface with and build relationships with Enterprise Architects, business people, and technologists at all levels
Job Responsibility
Job Responsibility
  • Work in the capabilities phases of the systems development life cycle
  • unpacking and translating technology security requirements, business and environmental conditions into enterprise security architecture patterns and processes
  • Identify Business objectives, goals and strategy
  • Identify Business attributes that are required to achieve those goals
  • Identify Enterprise
  • assets, systems, and information processes to support enterprise mission needs
  • Identify Stakeholder security requirements for protecting the organization’s mission and business processes
  • Identify The risk associated with the attributes that can prevent a business from achieving its goals
  • Identify The required controls to manage the risk
  • Identify assumptions, issues and dependencies
What we offer
What we offer
  • Tailored benefits that support your physical, emotional, and financial wellbeing
  • Continuous growth and development opportunities
  • Flexible work options
Read More
Arrow Right

Technical Project Manager

As a Technical Project Manager at 321, you will serve as the strategic leader re...
Location
Location
United States , Orlando
Salary
Salary:
75000.00 - 90000.00 USD / Year
321 Brand + Social House
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience in technical project management, preferably managing web development, software projects, or system integrations in an agency or professional services environment
  • Strong technical background with understanding of software development, system architecture, APIs, databases, and modern web technologies
  • Deep expertise in AI-assisted development including hands-on experience with Replit, GitHub Copilot, Cursor, or similar AI-powered development platforms
  • Proven ability to write comprehensive technical and functional requirements documentation for AI-enhanced applications and intelligent systems
  • Understanding of AI/ML concepts including model integration, training data requirements, API-based AI services, and practical AI implementation constraints
  • Experience managing custom website development projects across various CMS platforms and modern web frameworks
  • Expertise with CRM platforms (HubSpot, Salesforce, High Level) including custom integrations, API development, and complex implementations
  • Strong understanding of API design, RESTful services, authentication protocols, data modeling, and integration patterns
  • Proficiency with project management methodologies including Agile, Scrum, Waterfall, and hybrid approaches
  • Advanced skills in technical documentation including technical specifications, system architecture diagrams, data flow diagrams, and API documentation
Job Responsibility
Job Responsibility
  • Own end-to-end project management for complex technical initiatives including custom website development, CRM integrations, API development, and proprietary platform builds like 321 AIM
  • Develop comprehensive project plans including scope definition, technical architecture, resource allocation, timeline milestones, risk management, and budget tracking
  • Lead technical discovery sessions with clients and internal stakeholders to understand business requirements, technical constraints, and success criteria
  • Translate business objectives into detailed technical specifications, user stories, and acceptance criteria that guide development teams
  • Manage project budgets and timelines, tracking actuals against estimates and proactively addressing variances to protect profitability
  • Coordinate resources across development, design, Client Success, and external vendors to ensure seamless collaboration and delivery
  • Facilitate agile ceremonies including sprint planning, daily standups, backlog grooming, sprint reviews, and retrospectives
  • Create comprehensive technical requirements documents (TRDs) that define system architecture, data models, API specifications, integration patterns, and technical constraints
  • Write detailed functional requirements documents (FRDs) that describe user workflows, business logic, interface requirements, and acceptance criteria
  • Document system integrations including API endpoints, authentication methods, data mapping, error handling, and testing procedures
  • Fulltime
Read More
Arrow Right

Security Architect - AI

The AI Security Architect role focuses on leveraging expertise in information an...
Location
Location
United Kingdom , Birmingham
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum of 7 years of experience
  • Strong consulting background
  • Experience in defining enterprise security architecture
  • Experience in managing security risks
  • Experience in ensuring compliance with industry standards
  • Relevant certifications such as Open CA and TOGAF
  • Multi-year client-facing project and presentation experience in a professional services environment
  • Ability to operate in demanding situations
  • Strong influencer with ability to build good working relationships
  • Must be able to interface with and build relationships with Enterprise Architects, business people, and technologists at all levels
Job Responsibility
Job Responsibility
  • Work in the capabilities phases of the systems development life cycle
  • unpacking and translating technology security requirements, business and environmental conditions into enterprise security architecture patterns and processes
  • Identify Business objectives, goals and strategy
  • Identify Business attributes that are required to achieve those goals
  • Identify Enterprise
  • assets, systems, and information processes to support enterprise mission needs
  • Identify Stakeholder security requirements for protecting the organization’s mission and business processes
  • Identify The risk associated with the attributes that can prevent a business from achieving its goals
  • Identify The required controls to manage the risk
  • Identify assumptions, issues and dependencies
What we offer
What we offer
  • Range of tailored benefits that support your physical, emotional, and financial wellbeing
  • Continuous growth and development opportunities
  • Flexible work options
Read More
Arrow Right
New

Senior Consultant - Full Stack AI Application Developer

Microsoft Industry Solutions – Global Center for Innovation and Delivery (GCID) ...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of overall professional experience in software engineering, application development, and solution delivery, with a strong focus on enterprise‑scale, customer‑facing engagements.
  • 5+ years of experience in customer‑facing delivery leadership
  • Bachelor's degree in computer science engineering, or a related field.
  • Strong experience designing, building, and deploying business‑ and mission‑critical applications using .NET (C#), .NET Web APIs, Node.js.
  • Full‑stack development expertise across UI, API, service, domain, data, and integration layers, with proficiency in C#, JavaScript, TypeScript, Angular or React, HTML5, and CSS/SCSS.
  • Deep understanding of software architecture patterns and anti‑patterns for large‑scale systems, including MVC, CQRS, and Saga, with a focus on performance, scalability, and resiliency.
  • Excellent foundation in software engineering principles, data structures, algorithms, testing, debugging, and analytical problem‑solving.
  • Proficiency with cloud data platforms including Azure SQL, Cosmos DB, Azure Database for PostgreSQL, MySQL, Azure SQL Managed Instance, and MariaDB.
  • Experience creating reusable application frameworks and shared components that address cross‑cutting concerns such as logging, security, resiliency, and configuration management.
  • Hands‑on experience with multi‑threaded and parallel programming and high‑throughput service design.
Job Responsibility
Job Responsibility
  • Lead the technical delivery of complex, cloud‑native and AI‑enabled engagements, contributing as a hands‑on individual contributor and, where required, leading engineering teams to ensure high‑quality, predictable outcomes.
  • Define and drive the technology strategy for large and complex customer engagements, ensuring solutions are scalable, secure, and aligned to business objectives, architectural best practices, and Microsoft reference architectures.
  • Design, develop, and deploy solutions on Microsoft Azure, accelerating customer value through the effective use of Microsoft technologies, platforms, and delivery methodologies.
  • Applies user‑centered design principles to translate customer needs into clear technical requirements and scalable solution designs through iterative prototyping and feedback.
  • Provide delivery leadership by setting technical priorities, estimating effort, managing dependencies, making disciplined trade‑offs, and driving successful releases.
  • Ensure engineering excellence and solution quality, with accountability for the overall efficacy, performance, security, and compliance of technical delivery within assigned engagements.
  • Identify, assess, and manage technical risks and dependencies across projects, including defining mitigation strategies, handling escalations, and planning for both short‑ and long‑term delivery risks.
  • Apply information security, compliance, and assurance standards to maintain stakeholder confidence and meet customer, regulatory, and Microsoft governance requirements.
  • Drive innovation and digital transformation, leveraging existing intellectual property (IP), reusable assets, and best practices to improve delivery predictability and speed.
  • Collaborate with cross‑functional teams, including Sales, account‑aligned teams, and peers, to articulate technical value propositions and support the expansion and adoption of Microsoft cloud and AI solutions with strategic customers.
  • Fulltime
Read More
Arrow Right

Cloud / Application Security Architect

The Application Security Assurance Specialist - Principal is a pivotal role focu...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ Years experience in providing technical expertise in managing security frameworks and tools (SAST, DAST, SCA, container security, etc.)
  • Advanced knowledge of application lifecycle management methodologies (Waterfall, Agile, DevSecOps, CI/CD)
  • Strong understanding of compliance with standards such as OWASP Top 10, NIST CSF, and CIS Controls
  • Demonstrated ability to lead security assurance initiatives across complex development environments
  • Proficiency in designing and executing technical assessments and risk evaluations
Job Responsibility
Job Responsibility
  • Define and enforce embedded security practices across SDLC and CI/CD pipelines, ensuring compliance with organisational security policies and standards
  • Oversee the integration of advanced security tools (e.g., SAST, DAST, SCA, automated secret scanning) with development environments
  • Provide technical guidance on security configuration management, deployment hardening, and secure integration of tooling across all phases of software delivery
  • Conduct in-depth security risk assessments for high- and low-level technical designs, evaluating compliance against OWASP, CIS Benchmarks, and secure coding standards
  • Perform comprehensive security testing across application environments, including API security, container scanning, and dynamic runtime assessments, while evaluating residual risk post-assessment
  • Collaborate with stakeholders to assess the security maturity of existing practices and recommend improvements
  • Provide expert-level recommendations on the refinement of automation processes, risk mitigation strategies, and the deployment of compensating controls
  • Evaluate emerging technologies and leverage AI-driven application security tools to optimise assurance activities
  • Partner with development and DevSecOps teams to embed robust security measures within workflows
  • Actively engage in the training of development teams, fostering a culture of security awareness
What we offer
What we offer
  • Tailored benefits that support your physical, emotional, and financial wellbeing
  • Continuous growth and development opportunities
  • Flexible work options
Read More
Arrow Right

Senior Software Engineer or Solutions Architect

Provectus is an AWS Premier Consulting Partner and AI consultancy featured in Fo...
Location
Location
Salary
Salary:
Not provided
provectus.com Logo
Provectus
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Full-stack mindset, comfortable across AI, backend development, and cloud infrastructure
  • Already using AI tools in your daily workflow (Claude Code, Copilot, or similar)
  • Proactive and self-directed
  • you own outcomes end-to-end and spot problems before they're handed to you
  • B2+ English, comfortable collaborating across distributed, multicultural teams
  • Owns the client technical relationship
  • leading discovery, decomposing ambiguous requirements into technical components, presenting architecture, and pushing back on scope when it doesn't match timeline or budget
  • Produces scoped, phased delivery plans with clear deliverables, dependencies, and risks
  • Experience with cost estimation and cloud architecture cost optimization
  • 7+ years building and running production systems not only demos and POCs
Job Responsibility
Job Responsibility
  • Write clean, production-grade Python across AI integrations, backend services, and RESTful APIs
  • Implement and optimize RAG systems for production use cases
  • Design and build LLM-based and agentic AI solutions that address real client business challenges
  • Own the technical direction of client engagements from discovery through delivery
  • Support presales: discovery calls, technical proposals, scoping, and client-facing demos
  • Lead architecture reviews, produce technical design documents, and contribute to standards across the Python practice
  • Mentor engineers, lead code reviews, and share knowledge across the team
  • Build and maintain strong relationships with key client stakeholders as a trusted technical advisor
What we offer
What we offer
  • Opportunity to work with cutting-edge AI and cloud solutions
  • Internal training programs (Leadership, Public Speaking, and more) with full support for AWS and other professional certifications
  • Career growth: a clear path toward SA or beyond
  • we actively develop our engineers
  • Access to the latest AI tools and premium subscriptions
  • Long-term B2B collaboration
  • Remote with flexible hours
  • Private medical insurance or a budget for your medical needs
  • Paid sick leave, vacation, and public holidays
  • Equipment and all the tech you need for comfortable, productive work
Read More
Arrow Right

Senior AI Developer

The AI Developer will be part of a team of experts charged with providing suppor...
Location
Location
United States , Dahlgren
Salary
Salary:
Not provided
arcfield.com Logo
Arcfield
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • BS 8-10, MS 6-8, PhD 3-5
  • Must have an active Secret clearance and the ability to obtain and maintain a TS/SCI Clearance
  • DoDD 8140.01 Information Assurance Management (IAM) Level I certification or higher
  • Experience with Python programming language
  • Experience developing capabilities in software applications that interface with AI systems
  • Experience using frameworks for defining various AI workflows including but not limited to any of the following: Haystack, LangChain, LlamaIndex
  • Experience implementing retrieval augmented generation (RAG) workflows
  • Experience in context engineering
  • Determining the inputs, shape, and size provided to AI endpoints
  • Architecting enrichment steps to feed into the submission context
Job Responsibility
Job Responsibility
  • Working from requirements specifications to develop or modify software applications
  • Designing, coding, benchmark testing, debugging, and documenting application features and changes
  • Reviewing and assessing existing enterprise applications, to support new features, performance improvements, upgrades, and ongoing sustainment
  • Participating in performance, load, and stress testing, and remediation of issues identified
  • Participating in security architecture reviews of large enterprise systems
  • Participating in the development of new systems throughout the entire application lifecycle: requirements analysis & definition, system design, implementation, testing, deployment, and sustainment
  • Participating in activities that ensure the successful use of these systems including developing documentation, test scripts, and training materials
  • Working in an agile team environment participating in sprint planning, retrospectives, reviews, and daily standups
  • Fulltime
Read More
Arrow Right