CrawlJobs Logo
Plaid Logo Plaid · IT - Software Development

Senior Software Engineer - Data Infrastructure

United States, San Francisco 180000.00 - 270000.00 USD / Year · Job Posted December 11, 2025
Apply Position
Job Link Share

Job Description

We build the data and machine learning infrastructure to enable Plaid engineers to prototype and iterate on products and features built on top of consumer-permissioned financial data. Engineers on Data Infrastructure are domain experts in Data Warehouse, Data Lakehouse, Spark, Workflow Orchestration, and Streaming technologies. We scale our existing data pipelines in a performant and cost efficient way while creating the necessary abstractions to make developing on top of this platform extremely simple for other engineers at Plaid.

Job Responsibility

  • Contribute towards the long-term technical roadmap for data-driven and machine learning iteration at Plaid
  • Leading key data infrastructure projects such as improving ML development golden paths, implementing offline streaming solutions for data freshness, building net new ETL pipeline infrastructure, and evolving data warehouse or data lakehouse capabilities
  • Working with stakeholders in other teams and functions to define technical roadmaps for key backend systems and abstractions across Plaid
  • Debugging, troubleshooting, and reducing operational burden for our Data Platform
  • Growing the team via mentorship and leadership, reviewing technical documents and code changes

Requirements

  • 5+ years of software engineering experience
  • Extensive hands-on software engineering experience, with a strong track record of delivering successful projects within the Data Infrastructure or Platform domain at similar or larger companies
  • Deep understanding of one of: ML Infrastructure systems, including Feature Stores, Training Infrastructure, Serving Infrastructure, and Model Monitoring OR Data Infrastructure systems, including Data Warehouses, Data Lakehouses, Apache Spark, Streaming Infrastructure, Workflow Orchestration
  • Strong cross-functional collaboration, communication, and project management skills, with proven ability to coordinate effectively
  • Proficiency in coding, testing, and system design, ensuring reliable and scalable solutions
  • Demonstrated leadership abilities, including experience mentoring and guiding junior engineers

Nice to have

Experience with Databricks, Airflow, AWS EMR

What we offer

  • medical, dental, vision, and 401(k)
  • equity and/or commission
Plaid - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Senior Software Engineer - Data Infrastructure

8 matching positions

Senior Software Engineer, Data Infrastructure & AI

Fullstory Anywhere is one of Fullstory's three primary product verticals, and it...
Location
Location
United States , Atlanta
Salary
Salary:
160000.00 - 170000.00 USD / Year
fullstory.com Logo
Fullstory
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Significant experience building and operating high-throughput data pipelines (batch and/or streaming) in a major cloud platform, including work with cloud data warehouses like BigQuery, Snowflake, or Databricks.
  • Proficiency in Go, Python, Java or a similar language.
  • Hands-on experience with data transformation tooling such as dbt, with a strong understanding of data modeling and pipeline observability.
  • Familiarity with LLM integration patterns and evaluation approaches (e.g., LangSmith, Vertex AI, or comparable frameworks), or demonstrated ability to ramp quickly in applied AI.
  • A track record of owning major system areas end-to-end: driving architectural decisions, maintaining production health, and improving reliability over time.
Job Responsibility
Job Responsibility
  • Maintain, extend, and scale Go microservices that transform and deliver Fullstory session data into customer warehouses and power the team's MCP server that enables AI agent integrations.
  • Develop and maintain dbt models and pipeline orchestration to ensure timely, fault-tolerant data migrations across hundreds of customer destinations.
  • Define evaluation frameworks for LLM outputs using tools like Langsmith and Vertex AI, ensuring AI-powered customer agents produce accurate, useful results.
  • Investigate and resolve production incidents across the data pipeline, implementing systemic fixes that prevent entire classes of failure from recurring.
  • Write technical design documents that drive consensus on architectural changes, proactively surfacing scaling bottlenecks, edge cases, and cross-team dependencies.
  • Demonstrate sound technical judgment by de-risking work through spikes, taking on tech debt deliberately, and knowing when to escalate versus dig in.
What we offer
What we offer
  • Flexibility and Connection
  • flexible PTO policy
  • annual company-wide closure
  • Benefits
  • paid parental leave
  • Bereavement leave, including miscarriage/pregnancy loss
  • Learning opportunities
  • annual learning subsidy
  • Productivity support
  • monthly productivity stipend
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Data Infrastructure

LMArena is seeking a Software Engineer to join our team and build the data pipel...
Location
Location
United States , Bay Area
Salary
Salary:
Not provided
arena.ai Logo
Arena Intelligence, Inc.
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience in software engineering, with a dedicated focus on data engineering and big data technologies
  • Proficiency in SQL and at least one programming language commonly used for data analysis (Python (preferred), Scala, R)
  • Hands-on experience with data processing and pipeline frameworks (Apache Spark, Ray Data, etc.) and at least one popular big data analytics platform (Databricks, Snowflake)
  • Demonstrated experience in designing, implementing, optimizing, and debugging production data pipelines
Job Responsibility
Job Responsibility
  • Design and build robust data pipelines to ingest, process, and transform user vote data to features essential for model performance evaluation
  • Collaborate with researchers and product leadership to understand product goals and necessary data
  • Design and implement solutions to generate result dashboards and reports, providing useful information for the public, model providers, and researchers
  • Ensure the integrity, data quality, and reliability of the pipelines
  • Scale our data infrastructure to accommodate increasing data volumes and evolving analytical needs
What we offer
What we offer
  • Comprehensive health and wellness benefits, including medical, dental, vision, and additional support programs
  • The opportunity to work on cutting-edge AI with a small, mission-driven team
  • A culture that values transparency, trust, and community impact
  • Fulltime
Read More
Arrow Right

Senior .NET Software Engineer (Data Infrastructure)

At the core of Bentley's global infrastructure solutions lies a critical data se...
Location
Location
Lithuania , Vilnius; Kaunas
Salary
Salary:
4000.00 EUR / Month
bentley.com Logo
Bentley Systems
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • A Bachelor’s degree in Computer Science, Software Engineering, or a related field
  • At least 5 years of proven experience in software development with C#, .NET Core, and a strong understanding of OOP, data structures, and test frameworks
  • Expert-level, hands-on experience with major object storage platforms (Azure Blob Storage, Google Cloud Storage, AWS S3). This must include deep knowledge of storage usage optimization, lifecycle policies, and designing cost-efficient data access patterns
  • Solid experience with Docker and Kubernetes for deploying and managing containerized applications
  • Proven ability to write clean, maintainable, testable, and secure code, with an intuitive understanding of the long-term impact of architectural decisions
  • A self-motivated and proactive mindset, with the ability to work effectively as an individual contributor and as part of a high-performing team in an Agile/Scrum environment
  • Strong verbal and written communication skills in English
Job Responsibility
Job Responsibility
  • Architecting for Scale & Stability: Design and develop robust, event-driven cloud services and core components, with a primary focus on stability, performance, and long-term maintainability
  • Modernizing Our Storage Solutions: Implement and optimize solutions using the latest cloud object storage technologies (Google Cloud storage, Azure Blob, AWS S3, etc.) to enhance performance and cost-efficiency
  • Hands-On Implementation: Use the latest .NET development tools to turn complex architectural designs into high-quality, production-ready software
  • Championing DevOps & Automation: Develop and utilize fully automated CI/CD pipelines to deliver both application and infrastructure changes seamlessly and safely into production
  • Driving Technical Excellence: Mentor and share your deep expertise with colleagues, elevating the team's technical capabilities
  • Ensuring System Health: Support the existing code base, troubleshoot complex production issues, and collaborate across teams to ensure end-to-end service reliability
What we offer
What we offer
  • A great Team and culture
  • An exciting career as an integral part of a world-leading software company
  • An attractive salary and benefits package
  • A commitment to inclusion, belonging and colleague wellbeing
  • Training and professional development opportunities (certifications programs, conferences etc.)
  • Additional annual leave days and extra paid days for different occasions (marriage, moving day, bereavement leave etc.)
  • Health insurance package and accidents insurance 24/7
  • Referral program with bonuses
  • Extra paid day for volunteering in the organization of your choice
  • Ability to work from office or hybrid from home
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - Data Platform, AI Infrastructure

We are building a large-scale, productized data platform that powers critical in...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.
  • Strong programming experience in Python
  • Experience building and operating large-scale distributed systems
  • Hands-on experience with: Backend services or APIs (e.g., FastAPI, Flask, or similar)
  • Cloud-based infrastructure (Azure, AWS, or GCP)
  • Monitoring and observability systems (metrics, logging, alerting)
  • Experience designing systems with reliability, scalability, and operational clarity in mind
  • Proven ability to own and deliver production systems end-to-end
  • Ability to break down ambiguous problems, ask the right questions, and execute effectively
Job Responsibility
Job Responsibility
  • Design, build, and operate core components of a distributed data platform, including: Orchestration systems (e.g., Airflow or equivalent)
  • Backend services and APIs (Python/FastAPI or similar)
  • Monitoring, alerting, and reliability systems
  • Own the end-to-end lifecycle of platform components - from design through deployment, scaling, and maintenance
  • Ensure systems meet requirements for availability, performance, and data reliability at large scale
  • Define and enforce standardized patterns for infrastructure, deployment, and observability across the platform
  • Partner with data engineering teams to enable efficient, reliable data processing workflows
  • Diagnose and resolve complex issues in distributed systems, including performance bottlenecks and failure modes
  • Contribute to infrastructure-as-code and deployment systems to support reproducibility and operational excellence
  • Drive continuous improvements in system robustness, cost efficiency, and operational clarity
What we offer
What we offer
  • Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay
  • Fulltime
Read More
Arrow Right

Senior Solutions Engineer – Big Data & Data Infrastructure

This is a great opportunity to be part of one of the fastest-growing infrastruct...
Location
Location
Israel , Tel Aviv
Salary
Salary:
Not provided
vastdata.com Logo
VAST Data
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2–4 years in software / solution or infrastructure engineering
  • 2–4 years focused on building / maintaining large-scale data pipelines / storage & database solutions
  • Proficiency in Trino, Spark (Structured Streaming & batch) and solid working knowledge of Apache Kafka
  • Coding background in Python (must-have)
  • Deep understanding of data storage architectures including SQL, NoSQL, and HDFS
  • Solid grasp of DevOps practices, including containerization (Docker), orchestration (Kubernetes), and infrastructure provisioning (Terraform)
  • Experience with distributed systems, stream processing, and event-driven architecture
  • Hands-on familiarity with benchmarking and performance profiling for storage systems, databases, and analytics engines
  • Excellent communication skills
Job Responsibility
Job Responsibility
  • Build distributed data pipelines using technologies like Kafka, Spark (batch & streaming), Python, Trino, Airflow, and S3-compatible data lakes
  • Design, deploy, and troubleshoot hybrid cloud/on-prem environments using Terraform, Docker, Kubernetes, and CI/CD automation tools
  • Implement event-driven and serverless workflows
  • Create technical guides, architecture docs, and demo pipelines
  • Integrate data validation, observability tools, and governance directly into the pipeline lifecycle
  • Own end-to-end platform lifecycle: ingestion → transformation → storage (Parquet/ORC on S3) → compute layer (Trino/Spark)
  • Benchmark and tune storage backends (S3/NFS/SMB) and compute layers for throughput, latency, and scalability using production datasets
  • Work cross-functionally with R&D to push performance limits across interactive, streaming, and ML-ready analytics workloads
  • Operate and debug object store–backed data lake infrastructure
Read More
Arrow Right

Senior Software Engineer and Principal Software Engineer - Power Point AI Team

The PowerPoint team is embarking on an exciting new chapter - evolving a product...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • 8+ years of experience in backend service engineering, including work on high-scale infrastructures
  • Proficiency in one or more systems programming languages such as C#, C++
  • 1+ years of experience in software engineering, designing and developing systems (and APIs) that deploy and integrate with AI models
  • 2+ years of experience working with rich telemetry, making data driven decisions, and carrying out rapid experimentation
  • 2+ years of experience building software for scale, performance, and reliability
  • Academic or industry experience with building, finetuning, deploying or building eval-driven systems utilizing the models (any category)
Job Responsibility
Job Responsibility
  • Lead design and delivery of complex, scalable AI features ensuring resilience and exceptional user experience
  • Drive technical strategy and architecture decisions across multiple services, influencing partner teams and aligning with compliance and security requirements
  • Champion modern engineering practices, including AI-driven approaches, automation, and cloud-native patterns, across the full development lifecycle
  • Mentor and guide engineers, fostering technical excellence and continuous improvement in security, reliability, and performance
  • Collaborate cross-org to solve challenging technical problems, streamline processes, and reduce operational costs while improving live-site health
  • Design and implement scalable backend services optimized for machine learning workflows and large language model integration
  • Develop and maintain evaluation-driven systems that leverage text and multimodal inputs (e.g., images) to power visual-creation experiences
  • Build and optimize APIs and infrastructure to support high-performance model inference and experimentation at scale
  • Collaborate with product, ML, and design teams to integrate models into user-facing features, ensuring seamless functionality and performance
  • Conduct model evaluations and experiments, analyze results, and iterate on improvements to enhance accuracy and user experience
  • Fulltime
Read More
Arrow Right

Software Engineer - Data Infrastructure

As a Data Infrastructure Engineer at Luma, you will play a critical role in buil...
Location
Location
United States; United Kingdom , Palo Alto; London
Salary
Salary:
Not provided
lumalabs.ai Logo
Luma AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proficiency in Python (or similar languages with willingness to learn Python) and experience with large-scale, high-throughput data infrastructure
  • Familiarity with distributed computing frameworks (e.g., Ray, Spark, Beam)
  • Ability to design and optimize data pipelines for ML research and internal teams
  • Strong problem-solving skills and understanding of data engineering at scale
  • Collaborative, product-focused mindset
  • comfortable in fast-paced environments
  • Experience sourcing, integrating, and optimizing data from diverse and large datasets
  • Comfortable working in a fast-paced, product-focused environment with a strong execution mindset
  • Open to candidates across seniority levels, from mid-level individual contributors to senior engineers and managers.
Job Responsibility
Job Responsibility
  • Build and maintain scalable data infrastructure for high-throughput machine learning workflows
  • Collaborate with ML researchers and product teams to ensure data systems meet evolving needs
  • Develop and optimize large-scale data pipelines and batch processing jobs
  • Contribute to the architecture and implementation of reliable, high-performance data platforms
  • Integrate open-source tools and continuously improve data infrastructure through monitoring and tuning
  • Participate in cross-functional projects to improve data reliability, scalability, and operational excellence
  • Support the evaluation and adoption of new programming languages and frameworks relevant to data infrastructure
  • Engage in continuous improvement of data infrastructure through monitoring, troubleshooting, and performance tuning
  • Collaborate with research & engineering teams to help define and refine best practices for data infrastructure development
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Data

We're the world's leading sports technology company, at the intersection between...
Location
Location
Austria , Vienna
Salary
Salary:
Not provided
sportradar.com Logo
Sportradar
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of data engineering experience with proven track record of leading complex data projects from conception to delivery
  • Exceptional communication skills and experience working in cross-functional teams with analysts, product managers, and business stakeholders
  • AWS & Data Engineering: Very strong hands-on experience with AWS services (S3, Lambda, Glue, Athena, Redshift, EMR, etc.) and proficiency with Apache Spark for large-scale data processing
  • Backend Development: Strong experience with Python for building data processing services and APIs, plus expert-level SQL for data processing and analytics
  • Infrastructure & DevOps: Hands-on experience with Docker, Terraform, and CI/CD pipelines with automation best practices for data systems
  • Clean Code Advocate: Strong commitment to writing clean, maintainable, well-documented code with comprehensive testing and deep knowledge of analytics/reporting requirements
  • Data Architecture: Experience designing scalable data architectures, data modeling, and optimizing data processing workflows
  • Dashboard Development: Experience creating and managing analytics dashboards in bi tools (Tableau, Qlik Sense, Quicksuite, Power BI) and data visualization solutions to present complex insights to stakeholders
Job Responsibility
Job Responsibility
  • Scale & Performance Engineering: Processing and analyzing terabytes of advertising data with sub-second query performance while building and maintaining robust ETL pipelines using Spark and AWS services to handle massive data volumes daily
  • Data Pipeline Architecture & Development: Designing and building scalable data processing systems, developing backend APIs and microservices (Python or Go), architecting data flows that support both batch and real-time analytics requirements, and managing user-facing dashboards that visualize complex data insights
  • Infrastructure & Data Quality Operations: Implementing robust monitoring and alerting systems to detect data quality issues, managing AWS infrastructure using Terraform, implementing CI/CD best practices, and maintaining high coding standards across data processing systems
  • Cross-Functional Leadership & Collaboration: Leading large-scale data projects from requirements gathering through delivery, bridging technical implementation with business requirements, mentoring team members, and presenting technical concepts to stakeholders while challenging requirements constructively
  • End-to-End Data System Ownership: Taking complete ownership of complex data engineering projects while ensuring high availability and accuracy for both internal stakeholders and external clients, championing clean code principles, and serving as a knowledge leader who supports delivering the right data solutions
What we offer
What we offer
  • A collaborative environment with colleagues from all over the world (Engineering offices in Europe, Asia and US)
  • Ability to shape your own workday and career via a clearly defined professional and personal development plan
  • Opportunity to work with senior leadership, develop yourself and build your career within an inspiring and fast-growing company and digital sports environment
  • A vibrant and inclusive community, including Women in Tech and Pride groups which welcome all participants
  • A company culture that promotes social aspects, sports, physical exercise and fun
  • Innovative and cross-team challenges like ShipIt, office sports tournaments in Darts and Table Tennis and unique beer brewing competitions
  • Competitive salary and benefits (e.g. retirement pension and insurance plan)
  • Sportradar takes over the full costs of € 365.- for the Öffi-Ticket (Jahreskarte) for you
  • Fulltime
Read More
Arrow Right