CrawlJobs Logo
Capital One Logo Capital One · IT - Software Development

Sr. Lead Data Engineer

United States, McLean, Virginia | Richmond, Virginia Employment contract 209000.00 - 262400.00 USD / Year · Job Posted May 15, 2026
Apply Position
Job Link Share

Job Description

Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative,inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking Data Engineers who are passionate about marrying data with emerging technologies. As a Capital One Senior Lead Data Engineer, you’ll have the opportunity to be on the forefront of driving a major transformation within Capital One.

Job Responsibility

  • Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies
  • Lead a team of developers with deep experience in machine learning, distributed microservices, and full stack systems
  • Utilize programming languages like Java, Scala, Python and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services such as Redshift and Snowflake
  • Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community
  • Collaborate with digital product managers, and deliver robust cloud-based solutions that drive powerful experiences to help millions of Americans achieve financial empowerment
  • Perform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance

Requirements

  • Bachelor's Degree
  • At least 6 years of experience in application development
  • At least 2 years of experience in big data technologies
  • At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud)

Nice to have

  • Master's Degree
  • 9+ years of experience in application development including Python, SQL, Scala, or Java
  • 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud)
  • 5+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL)
  • 4+ year experience working on real-time data and streaming applications
  • 4+ years of experience with NoSQL implementation (Mongo, Cassandra)
  • 4+ years of data warehousing experience (Redshift or Snowflake)
  • 4+ years of experience with UNIX/Linux including basic commands and shell scripting
  • 2+ years of experience with Agile engineering practices
  • Experience leveraging interactive AI tooling to accelerate productivity

What we offer

  • health benefits
  • financial benefits
  • performance based incentive compensation
  • cash bonus
  • long term incentives (LTI)
Capital One - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Sr. Lead Data Engineer

8 matching positions

Sr. Engineer / Lead Engineer- BIW Processing & Tool Design

Mid-level Processing & Tool Design engineer; able to work autonomously with occa...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
gm.com Logo
General Motors
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in mechanical/production/Automobile/Industrial Engineering
  • Minimum 6-10 years of experience working in automotive industry / equivalent field
  • Strong understanding of automotive manufacturing processes, tool designs and materials handling systems
  • Proficiency in Microsoft Office Suite applications such as Excel, PowerPoint, Word, Access, SharePoint, etc.
  • Sound Knowledge in Siemens NX -Modelling/Drafting Knowledge along with GM Toolkit
  • Knowledge of TcAE
  • Excellent verbal and written communication skills
  • Timings: General / Late General shift to support global operations.
Job Responsibility
Job Responsibility
  • Execute Tool design done as per MPMs or suggested datum strategy, Input checking for the data received from GMNA
  • Responsible to execute & coordinate for tool design & weld template for Pre-production operation-PTB builds, Tool Design for Innovation / Optimization projects of manufacturing operations
  • Perform Tool design Internal/ Stake holder review with approval & Detailing for the approved tool with all deliverables
  • Revise the Tool design as per change log
  • Responsible for Supplier/Stake holders’ coordination & issue resolution in Tooling activities
  • Delivery of the design kits to stake holders in Teamcenter
  • Communicate with customer for reviews and Query resolution
  • Weld template creation for new tools from MPMs & revision as per change log
  • Sound knowledge of GD&T, understanding of Clamping and Locating concept (CDLS)
  • Project kickoff and wrap up documents. Drive project productivity improvement activity
  • Fulltime
Read More
Arrow Right
New

Sr. Technical Data Product Owner (Sr. Business Data Analyst)

We are looking for a Sr. Technical Data Product Owner (Sr. Business Data Analyst...
Location
Location
United States , Richmond
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5 to 7 years of experience in business analysis, product ownership, or a related role within data, business intelligence, or data engineering environments
  • Demonstrated involvement in data migration initiatives, including work related to mapping, integration, or delivery coordination
  • Strong ability to gather requirements, identify gaps, and translate business requests into clear documentation and actionable plans
  • Working knowledge of data modeling, ETL processes, and data warehousing concepts
  • Experience validating datasets and analyzing outputs, with familiarity using BI tools such as Power BI for review purposes
  • Proven success managing stakeholders and collaborating with business partners, data engineers, and BI teams in fast-paced settings
  • Strong organizational skills, attention to detail, and comfort working within Agile delivery frameworks
Job Responsibility
Job Responsibility
  • Partner with stakeholders to capture business objectives, clarify data needs, and turn those inputs into thorough functional documentation
  • Lead data-focused project activities by coordinating requirements, source-to-target mapping, and planning for migration and integration efforts
  • Serve as the key connection point between business teams and Data/BI engineering resources to keep scope, priorities, and expectations aligned
  • Oversee delivery progress across multiple data workstreams, ensuring milestones are met and outcomes reflect agreed business goals
  • Review data results and perform analysis to confirm accuracy, completeness, and consistency with stakeholder expectations
  • Create process mockups or early solution concepts that help teams evaluate requirements and refine the proposed approach
  • Support testing, validation, and quality checks for data products to improve confidence in delivered solutions
  • Guide business users through handoffs, knowledge transfer, and adoption activities so new data capabilities are understood and used effectively
  • Build strong working relationships across teams and establish effective communication paths that support timely decisions and issue resolution
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • enrollment in company 401(k) plan
Read More
Arrow Right

Sr Data Engineer

As a Senior Data Engineer at Amgen, you will be responsible for managing and opt...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
amgen.com Logo
Amgen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master's degree with 8 - 13 years of experience in Computer Science, IT or related field
  • Hands-on experience in data engineering technologies such as Databricks, PySpark, SparkSQL Apache Spark, AWS, Python, SQL, and Scaled Agile methodologies
  • Proficiency in workflow orchestration, performance tuning on big data processing
  • Strong understanding of AWS services
  • Experience with Data Fabric, Data Mesh, or similar enterprise-wide data architectures
  • Ability to quickly learn, adapt and apply new technologies
  • Strong problem-solving and analytical skills
  • Excellent communication and teamwork skills
  • Experience with Scaled Agile Framework (SAFe), Agile delivery practices, and DevOps practices
  • Excellent analytical and troubleshooting skills
Job Responsibility
Job Responsibility
  • Design, develop, and maintain scalable ETL/ELT pipelines to support structured, semi-structured, and unstructured data processing across the Enterprise Data Engineering for Biotech or Pharma functional knowledge of R&D
  • Implement real-time and batch data processing solutions, integrating data from multiple sources into a unified, governed data fabric architecture
  • Optimize big data processing frameworks using Apache Spark, Hadoop, or similar distributed computing technologies to ensure high availability and cost efficiency
  • Work with metadata management and data lineage tracking tools to enable enterprise-wide data discovery and governance
  • Ensure data security, compliance, and role-based access control (RBAC) across data environments
  • Optimize query performance, indexing strategies, partitioning, and caching for large-scale data sets
  • Develop CI/CD pipelines for automated data pipeline deployments, version control, and monitoring
  • Implement data virtualization techniques to provide seamless access to data across multiple storage systems
  • Collaborate with cross-functional teams, including data architects, business analysts, and DevOps teams, to align data engineering strategies with enterprise goals
  • Stay up to date with emerging data technologies and best practices, ensuring continuous improvement of Enterprise Data Fabric architectures
  • Fulltime
Read More
Arrow Right

Data Engineer Sr (DBT)

Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years in modern ELT / transformation engineering, with strong expertise in DBT-based development for curated enterprise data layers.
  • Experience defining DBT project standards, modular model design, testing frameworks, documentation, macros, environment strategy, and deployment controls.
  • Proven ability to lead development of conformed Silver layer models across Policy, Claims, Billing, and shared dimensions / reference domains.
  • Strong experience integrating DBT with Databricks / Delta Lake, orchestration tools, version control, and automated quality checks.
  • Able to guide developer productivity, code review standards, reusability, performance tuning, and traceability from mapping to model implementation.
Job Responsibility
Job Responsibility
  • Design and implement staging, core, and mart layers using standard dbt patterns and folder structures.
  • Develop and maintain dbt tests (schema tests, data tests, custom tests) to ensure data quality and integrity.
  • Implement snapshots, seeds, macros, and reusable components where appropriate.
  • Collaborate with Snowflake developers to ensure physical data structures support dbt models efficiently.
  • Work with functional teams to ensure functional equivalence between legacy Informatica outputs and new dbt outputs.
  • Participate in performance tuning of dbt models and Snowflake queries.
  • Integrate dbt with CI/CD pipelines (e.g., Azure DevOps, GitHub Actions) for automated runs and validations.
  • Contribute to documentation of dbt models, data lineage, and business rules.
  • Participate in defect analysis, bug fixes, and enhancements during migration and stabilization phases.
Read More
Arrow Right

Sr. Data Engineer

We are seeking an experienced candidate to drive our data engineering initiative...
Location
Location
United States , Las Vegas
Salary
Salary:
80000.00 - 90000.00 USD / Year
techmahindra.com Logo
Tech Mahindra
Expiration Date
June 30, 2026
Flip Icon
Requirements
Requirements
  • 7-8+ years of hands-on experience in data engineering or related fields
  • Strong proficiency in: Apache Airflow (DAG development, workflow orchestration)
  • Snowflake
  • Advanced SQL (complex queries, performance tuning)
  • Apache Kafka (streaming, event-driven architectures)
  • API development and integration (REST/SOAP)
  • Proven experience leading technical teams (3+ years preferred)
  • Strong problem-solving and communication skills
  • Experience with Agile/Scrum methodologies
  • A Bachelor's or Higher Degree is the minimum entry required for the position
Job Responsibility
Job Responsibility
  • Lead and mentor a team of 5-6 data engineers, fostering technical growth and collaboration
  • Design and implement scalable data pipelines using Apache Airflow
  • Understanding of Snowflake architecture
  • Integrate and maintain real-time streaming applications using Apache Kafka / Spark Streaming
  • Build and integrate APIs across multiple systems and platforms
  • Drive technical decision-making and establish best practices
  • Ensure code quality through reviews and implementation
What we offer
What we offer
  • Medical
  • Vision
  • Dental
  • Life
  • Disability insurance
  • Paid time off (including holidays, parental leave, and sick leave, as required by law)
  • Opportunity to lead impactful data initiatives
  • Collaborative and innovative work environment
  • Fulltime
Read More
Arrow Right

Sr. Data Engineer III

Piper Companies is seeking a Sr. Data Engineer III to join an enterprise technol...
Location
Location
United States
Salary
Salary:
130000.00 - 150000.00 USD / Year
pipercompanies.com Logo
Piper Companies
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of data engineering or related experience
  • Strong expertise in SQL, Python, dbt, and or C#
  • Experience with Azure Data Factory or similar orchestration tools
  • Hands-on experience with Microsoft Azure and Fabric
  • Experience with Databricks or Snowflake is acceptable
  • Strong understanding of data modeling and ETL concepts
  • Experience with Git and version control best practices
  • Ability to work independently and solve complex problems
  • Background working in small or mid-sized environments with leadership experience
  • Must be authorized to work in the United States
Job Responsibility
Job Responsibility
  • Design and maintain scalable data pipelines and ETL workflows
  • Develop and optimize advanced SQL queries for performance and reliability
  • Leverage Azure, Fabric, Spark, and Python to automate complex data processes
  • Build and support REST and SOAP APIs for system integrations
  • Monitor and troubleshoot data pipelines to ensure high availability
  • Identify and resolve bottlenecks in data workflows
  • Lead engineering efforts supporting AI and machine learning initiatives
  • Collaborate with stakeholders and cross-functional teams
  • Conduct code reviews and contribute to technical documentation
  • Provide Tier 3 support for reporting and integrated systems
What we offer
What we offer
  • medical
  • dental
  • vision
  • 401k
  • PTO
  • sick leave as required by law
  • Fulltime
Read More
Arrow Right

Sr Data Engineer

The purpose of the Data Engineer position is to leverage their technical experti...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.randstad.com Logo
Randstad
Expiration Date
July 03, 2026
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Data Science, Information Technology, or a related field with a focus on data engineering or data analytics
  • Strong proficiency in programming languages such as Python, SQL, Java, or Scala for data processing and analysis
  • Experience with data modeling, ETL processes, data warehousing, data integration, and data pipeline development
  • Proficiency in relational databases (e.g., SQL, PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra)
  • Working knowledge of cloud platforms such as Snowflake, AWS, Azure, or Google Cloud Platform for data storage and processing
  • Experience with data visualization tools (e.g., Power BI) to create meaningful insights from data
  • Understanding of data quality principles, data governance, and data validation processes
  • Ability to analyze complex data sets, identify trends, patterns, and insights to drive data-driven decision-making
  • Proficiency in troubleshooting data-related issues, identifying root causes, and implementing solutions
  • Familiarity with project management methodologies to contribute effectively to project planning and execution
Job Responsibility
Job Responsibility
  • Build pipelines using standard patterns for data pipelines and workflows utilizing Streamsets, Kestra, dbt, Git
  • Design and Implement data storage and processing solutions employing Snowflake
  • Utilize AWS services for cloud-based platform tooling infrastructure including but not limited to Lambda,ECS,MSK,RDS,EC2, Secrets Manager, ALB, Cloud Watch, Event Bridge
  • Utilize Terraform for AWS and Azure deployments
  • Leverage and integrate APIs for data access and manipulation
  • Write Python scripts for data common processing and automation tasks
  • Leverage Platform API’s and Web Applications to enforce Platform Security
  • Create and execute comprehensive test plans to ensure the pipelines functionality and performance
  • Develop unit tests, integration tests, and end-to-end tests for data pipelines and workflows
  • Ensure data accuracy and consistency through rigorous testing processes
What we offer
What we offer
  • Commitment to your ongoing development, including on the job opportunities and formal programs
  • Inclusive parental leave entitlements for both parents
  • Values led culture
  • Flexible work options
  • Generous annual leave, sick leave and casual leave
  • Cultural and religious leave with flexible public holiday opportunities
  • A competitive remuneration package featuring performance based incentives with uncapped Employer Provident Fund
  • Fulltime
!
Read More
Arrow Right

Sr Data Engineer

The Senior Data Engineer role involves leading data architecture and modernizati...
Location
Location
India , Chennai
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong expertise in Azure and Databricks
  • Focus on data modeling and integration
  • Azure
  • Databricks
  • Data Modeling
  • Team Leadership
  • Client Interviews
  • SQL
Job Responsibility
Job Responsibility
  • Engage heavily with business users across North America and Europe, facilitating workshops and data discovery sessions
  • Drive consensus on business rules, data definitions, and data sources, especially where regional processes differ
  • Serve as the architectural thought leader enabling teams to transition from manual, inconsistent processes to standardized, modernized workflows
  • Partner closely with business analysts, data analysts, product owners, and engineering teams across multiple geographies
  • Architect a unified master stitched data model to replace downstream reliance on Varicent for data assembly
  • Lead the re‑architecture of compensation data processing—including internal and external compensation flows—into a scalable, cloud‑native Azure environment
  • Define patterns, frameworks, and integration strategies across Azure services (Data Factory, Databricks, Data Lake, SQL, etc.)
  • Evaluate and evolve the use of rules engines/ODM/Drools to externalize and modernize embedded business logic currently locked in application code
  • Guide decisions to shift logic and data ownership into enterprise‑owned systems rather than third‑party tools
  • Analyze current‑state processes (38 in NA, 9 in Europe) and identify opportunities for re‑engineering, automation, and consolidation
Read More
Arrow Right