CrawlJobs Logo

Senior Data Engineer – SC Cleared

United Kingdom, London · Job Posted June 09, 2026
Apply Position
Job Link Share

Job Description

Whitehall Resources are currently looking for a Data Engineer on a remote basis for an initial 5-month contract.

Job Responsibility

  • Data engineering at scale in government or similarly complex domains, including performance tuning and data‑quality management
  • Hands‑on expertise in AWS & Spark: Amazon EMR, S3, Lambda
  • strong PySpark/Python and SQL for large‑scale batch processing
  • CI/CD & DevOps: pipelines and IaC (e.g., Terraform), automated testing, and release governance
  • Version control & collaboration: Git/GitLab, code review, branching strategies, and trunk/PR workflows
  • APIs & integration: building/consuming data services to move and expose data safely and reliably
  • Agile ways of working with Jira/Confluence
  • clear stakeholder communication and concise technical documentation
  • Data warehousing & modelling (e.g., Redshift
  • dimensional modelling
  • dbt)
  • Basic Power BI familiarity to partner with BI developers and validate end‑to‑end data flows

Requirements

  • MUST BE SC CLEARED
  • INSIDE IR35
  • Data engineering at scale in government or similarly complex domains, including performance tuning and data‑quality management
  • Hands‑on expertise in AWS & Spark: Amazon EMR, S3, Lambda
  • strong PySpark/Python and SQL for large‑scale batch processing
  • CI/CD & DevOps: pipelines and IaC (e.g., Terraform), automated testing, and release governance
  • Version control & collaboration: Git/GitLab, code review, branching strategies, and trunk/PR workflows
  • APIs & integration: building/consuming data services to move and expose data safely and reliably
  • Agile ways of working with Jira/Confluence
  • clear stakeholder communication and concise technical documentation
  • Data warehousing & modelling (e.g., Redshift
  • dimensional modelling
  • dbt)
  • Basic Power BI familiarity to partner with BI developers and validate end‑to‑end data flows
  • Informatica experience
  • Data project experience
  • AWS
  • Azure

Nice to have

  • AWS ecosystem depth (Athena, Redshift, EC2, CloudWatch, IAM) and event‑driven patterns
  • AWS Certified Cloud Practitioner (or higher)
  • Azure AI Fundamentals (awareness of ML/AI services)

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Senior Data Engineer – SC Cleared

8 matching positions

Senior Data Engineer – Informatica

Whitehall Resources currently require an experienced Data Engineer to work with ...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
whitehallresources.com Logo
Whitehall Resources Ltd
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Hands on expertise in AWS & Spark: Amazon EMR, S3, Lambda
  • strong PySpark/Python and SQL for large scale batch processing
  • Data engineering at scale in government or similarly complex domains, including performance tuning and data quality management
  • CI/CD & DevOps: pipelines and IaC (e.g., Terraform), automated testing, and release governance
  • Version control & collaboration: Git/GitLab, code review, branching strategies, and trunk/PR workflows
  • APIs & integration: building/consuming data services to move and expose data safely and reliably
  • Agile ways of working with Jira/Confluence
  • clear stakeholder communication and concise technical documentation
  • Security clearance: BPSS (minimum) and SC cleared or SC clearable for UK government work.
Job Responsibility
Job Responsibility
  • Engineer production grade data pipelines on AWS (EMR, S3, Lambda), using PySpark/Python and SQL, with a focus on performance, resilience, testing, and observability
  • Migrate and modernise legacy workloads (e.g., ETL jobs and reporting feeds) onto cloud native services, creating reusable components and shared frameworks
  • Support reporting & MI use cases, including transformations and data models that feed downstream tools (e.g., Power BI)
  • Own CI/CD and version control practices (e.g., Git/GitLab), review code, and enforce engineering standards
  • Coach and mentor engineers, provide technical guidance/code reviews, and contribute to architectural decisions across squads
  • Work in Agile delivery, collaborating across product, data, and platform teams using Jira/Confluence
  • translate requirements into robust engineering tasks
  • Embed security and compliance by design, aligning with BPSS/SC constraints and department data handling policies
  • Fulltime
Read More
Arrow Right

Senior Technical Support Engineer

This is **not** a ticket-queue role. This is the job you take if you *like* bein...
Location
Location
Mexico , Mexico DF
Salary
Salary:
Not provided
bentley.com Logo
Bentley Systems
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years in senior technical support / escalation engineering / production support for enterprise software applications (SaaS, cloud-hosted, or hybrid strongly preferred).
  • Proven record owning P1/P0 escalations and guiding issues through resolution with Engineering/Operations partners.
  • Strong, practical expertise in: Azure cloud architecture concepts (identity, networking, compute, storage, monitoring)
  • Microsoft Windows (server and client), troubleshooting, performance, and diagnostics
  • Networking fundamentals and packet-level troubleshooting methods
  • API/integration troubleshooting and familiarity with modern auth patterns (OAuth/SSO concepts)
  • Demonstrated ability to interpret complex diagnostic data (e.g., packet captures, HAR files, deep logs) and drive clear hypotheses to closure.
  • Exceptional communication: able to explain deeply technical findings to both engineers and customer stakeholders
  • skilled at de-escalation under pressure.
  • Required: at least one current, role-relevant cloud certification (or evidence you can achieve within 90 days).
Job Responsibility
Job Responsibility
  • Technical Leadership in Escalations (Customer-Facing): Act as a senior technical lead during high severity (P1 / Sev1) customer escalations
  • Join and actively participate in live escalation bridges, providing: Technical diagnosis and direction
  • Clear explanation of system behavior and failure modes
  • Credible technical input to support customer and executive conversations
  • Partner closely with Escalation Managers, who own: Customer and executive messaging
  • Communication cadence
  • Stakeholder alignment
  • while you ensure the technical narrative is accurate, coherent, and defensible
  • Help stabilize emotionally charged situations by bringing clarity, structure, and technical confidence to live discussions
  • Deep Technical Diagnostics & Resolution (Prove root cause, no guessing, no vibes): Lead complex technical investigations across: Cloud hosted and hybrid application architectures
  • Fulltime
Read More
Arrow Right
New

Data Architect

Whitehall Resources are currently looking for a Data Architect in the West Midla...
Location
Location
United Kingdom , West Midlands
Salary
Salary:
Not provided
whitehallresources.com Logo
Whitehall Resources Ltd
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong experience as a SAS Architect or Senior SAS Technical Lead
  • Hands-on architectural experience with Legacy SAS (Base SAS, SAS Grid, SAS 9.x)
  • Proven experience integrating SAS with enterprise data warehouses
  • Experience working in regulated or large-scale enterprise environments
Job Responsibility
Job Responsibility
  • Own the architecture and technical design of SAS-based solutions across legacy SAS and SAS Viya environments
  • Provide design assurance and governance for SAS services, ensuring alignment with data platform and enterprise architecture standards
  • Support and guide teams consuming data warehouse sources (on prem and/or cloud) from SAS
  • Define and document integration patterns between data warehouses and SAS platforms
  • Advise on migration, coexistence and modernisation strategies between legacy SAS and Viya
  • Act as the technical authority for SAS-related decisions, risks and trade-offs
  • Collaborate with data engineers, platform teams and vendors to resolve complex issues
  • Ensure solutions consider security, performance, scalability and data quality
  • Fulltime
Read More
Arrow Right

Staff Engineer

We're looking for a Staff Engineer (Tech Lead) to provide technical leadership f...
Location
Location
United Kingdom , Warminster
Salary
Salary:
Not provided
4cstrategies.com Logo
4C Strategies
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • An experienced engineer who still enjoys writing code, with several years operating at Senior Engineer level or above and a track record of guiding non-trivial technical work across a team or cluster of teams
  • Sound architectural judgement around data modelling, integrations and API design (REST, event-driven and similar). You weigh long-term consequences without losing sight of short-term delivery
  • Comfortable being a team's technical point of reference — you lead through influence, partnership and consistently good judgement, rather than formal authority
  • Experienced partnering with Engineering Managers, Product Managers, designers and Delivery Leads to shape roadmaps and untangle complex, ambiguous work
  • A mentor by instinct who can point to engineers whose careers you've helped shape
  • Clear, considered communicator — both in writing and in person — including with senior stakeholders in regulated environments
  • UK security clearance: if you already hold UK SC, that's a plus. We're equally happy to hear from candidates who are eligible for, and willing to undergo, SC clearance
  • A genuine interest in our mission — software that helps armed forces, public safety and resilience organisations train, prepare, and respond
Job Responsibility
Job Responsibility
  • Hold the technical vision for our work on the programme, in close partnership with Engineering Management, Product and Delivery
  • Shape the architecture across data, integrations and APIs — including how Exonaut connects with MoD and partner systems, and how training data flows through the platform
  • Help scope the team's most complex initiatives end-to-end, and share implementation work across the team so others can grow while you remain accountable for the outcome
  • Help raise our engineering standards around design, code quality, security, performance and operability
  • Mentor Senior Engineers toward Staff-level scope, and represent engineering thoughtfully with customers, partners and 4C peers
What we offer
What we offer
  • 25 days holidays (plus bank holidays)
  • Private healthcare including dental
  • Pension Scheme with 5% employer contribution
  • Enhanced maternity and paternity benefits
  • Life Assurance
  • Cycle to work scheme
  • Work Extras Discounts at over 80 retailers
  • Company social events
  • Fulltime
Read More
Arrow Right

Junior SOC Engineer

The Junior SOC Engineer role at NTT DATA involves supporting the operation and m...
Location
Location
United Kingdom , Birmingham
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Hands on experience with SIEM platforms
  • Understanding of log formats, parsing, and data normalisation
  • Working knowledge of SIEM query languages such as SPL, KQL, or AQL
  • Basic scripting ability (Python and/or PowerShell) for automation and enrichment
  • Understanding of network traffic flows and the ability to recognise normal vs suspicious activity
  • Familiarity with vulnerability scanning and vulnerability management concepts
  • Foundational experience with SIEM technologies (e.g. Splunk/QRadar)
  • Understanding of log data, alert workflows, and basic threat detection concepts
  • Awareness of common attack techniques and defensive frameworks (e.g. MITRE ATT&CK)
  • Basic scripting or automation experience (Python, PowerShell – desirable)
Job Responsibility
Job Responsibility
  • Assist with the deployment, configuration, and ongoing maintenance of SIEM platforms and data feeds
  • Support onboarding, parsing, and normalisation of log sources across on premises and cloud environments
  • Contribute to the creation, tuning, and maintenance of SIEM detection rules and correlation searches
  • Perform basic health checks and troubleshooting of SIEM data ingestion issues
  • Assist senior engineers and analysts during investigations and incident response activities
  • Support continuous improvement of detections based on incident findings and threat intelligence
  • Support integration of playbooks with SOAR platforms
  • Help update and refine playbooks based on lessons learned and feedback from incidents
  • Participate in threat modelling activities using frameworks such as MITRE ATT&CK and STRIDE
  • Assist in translating threat models and attack techniques into SIEM detection use cases
What we offer
What we offer
  • A range of tailored benefits that support your physical, emotional, and financial wellbeing
  • Continuous growth and development opportunities
  • Flexible work options
  • Fulltime
Read More
Arrow Right

Quality Engineering Lead

We are seeking an experienced Test Director / Quality Engineering Director with ...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Engineering, or related field
  • Extensive experience (typically 15+ years) in testing, quality engineering or data delivery roles, with a strong technical background
  • BPSS minimum, SC not mandated but strong likelihood to be required
  • Proven Director level experience leading testing and quality engineering on large, complex data-centric programmes
  • Excellent stakeholder management skills, with the ability to influence at Programme Board level and to convey complex technical concepts to both technical and non-technical stakeholders
  • Experience managing large SDET and QA teams, including test data and environment management
  • Demonstrated experience producing Programme-level Test Strategies and Master Test Plans for functional and non-functional testing in a data environment with a strong focus on data quality and integrity
  • Strong understanding of data engineering concepts, including data pipelines data migration, data architecture and related testing methodologies
  • Experience leading teams responsible for designing, building and maintaining automated data testing frameworks, leveraging scripting and orchestration where appropriate
  • Strong problem-solving skills and ability to think critically
Job Responsibility
Job Responsibility
  • Programme-Level Accountability & Leadership: Sit as a full member of the Programme Leadership Team, accountable for quality outcomes across the delivery
  • Act as the single point of accountability for all testing and quality engineering activity
  • Provide clear, evidence-based recommendations on go-live readiness, data integrity and operational risk
  • Lead, coach and motivate a team of Test Leads and SDETs
  • Multi-Supplier & Stakeholder Management: Serve as the primary point of contact for the client and third-parties on all testing matters
  • Coordinate testing across multiple suppliers, ensuring consistency in standards, tooling and reporting
  • Challenge delivery partners constructively on test coverage, defect quality, dependencies and risk exposure
  • Communicate complex technical test outcomes in clear business terms to senior stakeholders
  • Test Strategy & Operating Model: Define, own and maintain the Test Strategy, aligned to a Data Factory testing model
  • Author and assure Programme-level Master Test Plans and Test Phase / Test Level Plans
What we offer
What we offer
  • Tailored benefits that support your physical, emotional, and financial wellbeing
  • Continuous growth and development opportunities
  • Opportunity to have flexible work options
  • Fulltime
Read More
Arrow Right

Security Architect

Security Architect Permanent Location: Home / Andover (Hybrid – onsite 2 days a ...
Location
Location
United Kingdom , Andover
Salary
Salary:
100000.00 - 110000.00 GBP / Year
datacareers.co.uk Logo
DataCareers
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Active SC clearance
  • Strong UK Defence, MOD or secure Central Government experience
  • Proven background as a Security Architect, Enterprise Security Architect, Cyber Security Architect or Security Consultant
  • Experience providing security architecture assurance across complex systems or major programmes
  • Ability to produce clear architecture documentation and security artefacts
  • Strong understanding of secure design principles, risk management, threat modelling and security controls
  • Knowledge of UK Government / Defence security standards, guidance and assurance processes
  • Experience working with senior stakeholders across security, architecture, engineering and delivery
  • Strong communication skills, with the ability to explain security risk clearly to technical and non-technical audiences
Job Responsibility
Job Responsibility
  • Lead security architecture reviews across complex Defence or Central Government systems and programmes
  • Identify risks, control gaps and design changes required to achieve the required security posture
  • Produce and maintain security architecture artefacts, including HLDs, LLDs, data flows, security principles, patterns and architecture building blocks
  • Ensure security is embedded across solution design, integration, deployment and operational readiness
  • Support cyber assurance activity across design, build, deploy and run phases
  • Work closely with enterprise architects, solution architects, delivery teams, security stakeholders and customer representatives
  • Translate complex security risks and design issues into clear options, recommendations and decision papers
  • Support alignment to relevant Defence, Government and industry security standards
  • Contribute to technical management plans, covering areas such as configuration standards, patching, cryptography, IAM and assurance checkpoints
  • Provide confident stakeholder engagement across mixed technical, operational and senior leadership audiences
Read More
Arrow Right

Vulnerability Management Senior Cyber Security Analyst

Location (Primary) - Noida / Chennai (Secondary) GCF Level - 2A(01 nos) & 2B(01 ...
Location
Location
India , Noida
Salary
Salary:
Not provided
https://www.soprasteria.com Logo
Sopra Steria
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Must hands on experience with detection and monitoring tools (Microsoft Defender for Endpoint EDR/XDR, WIZ, NESSUS PRO, Dynatrace)
  • Experience information system management and mapping tools (CMDB, VISIT)
  • Good knowledge of software distribution tools (SCCM, Intune, Satellite, Ansible, etc.), and reporting and analysis tools (PowerBI)
  • Experience with ServiceNow (SNOW), specifically the SECOPS module
  • Strong understanding of - On-prem infrastructure, SaaS / IaaS / Cloud workloads, Application vulnerability context
  • Ability to enrich findings using CMDB / asset mapping tools
  • Working knowledge of - SCCM, Intune, Ansible, Satellite
  • Stakeholder & Coordination Skills
  • Documentation & Effective Communication - Clear communication (EN/FR preferred) with structured documentation mindset
  • Process & Continuous Improvement Mindset
Job Responsibility
Job Responsibility
  • Oversee the receipt, analysis, and tracking of vulnerabilities from various sources (CERT, scanning tools, division reports)
  • manage backlog processing, and create or update vulnerability tickets using multiple detection and asset management tools
  • Assess and identify impacted assets across various environments
  • Qualify vulnerabilities by evaluating exposure, versions, severity, attack vectors, and client context
  • Enhance asset information using CMDB, and promptly issue alerts for critical vulnerabilities
  • Develop, implement, and coordinate remediation plans by analyzing security advisories and scan reports
  • Defining tailored action strategies (including patches, workarounds, and updates)
  • Prioritizing tasks, tracking requests in ServiceNow
  • Sending criticality-based reminders, and supporting remediation teams
  • Monitor and validate patch application by ensuring timely verification
  • Fulltime
Read More
Arrow Right