CrawlJobs Logo

Senior Software Engineer, Core Data

United States 190000.00 - 220000.00 USD / Year · Job Posted December 11, 2025
Apply Position
Job Link Share

Job Description

As a Senior Software Engineer on our Core Data team, you will take a leading role in building and scaling the data infrastructure that powers Pomelo's data-driven insights and product features. You will be a technical leader and a mentor to others, responsible for reducing the company's time to insight.

Job Responsibility

  • Build and maintain elegant data pipelines that orchestrate ingestion from diverse sources and normalize data for company-wide consumption
  • Lead the design and development of robust, scalable data infrastructure that enables our clinical and product teams to make data-driven decisions, using dbt, Dagster, Beam and Dataflow
  • Write clean, performant SQL and Python to transform raw data into actionable insights that power our platform
  • Architect data models and transformations that support both operational analytics and new data-driven product features
  • Mentor other engineers, providing technical guidance on data engineering best practices and thoughtful code reviews, fostering a culture of data excellence
  • Collaborate with product, clinical and analytics teams to understand data needs and ensure we are building infrastructure that unlocks the most impactful insights
  • Optimize data processing workflows for performance, reliability and cost-effectiveness

Requirements

  • 5+ years of experience building high-quality, scalable data systems and pipelines
  • Expert-level proficiency in SQL and Python, with a deep understanding of data modeling and transformation best practices
  • Hands-on experience with dbt for data transformation and Dagster, Beam, Dataflow or similar tools for pipeline orchestration
  • Experience with modern data stack tools and cloud platforms, with a strong understanding of data warehouse design principles
  • A track record of delivering elegant and maintainable solutions to complex data problems that drive real business impact

Nice to have

  • Experience building data infrastructure that supports AI/ML workloads or real-time analytics
  • Previously worked in a fast-paced, product-oriented environment where data drives key decisions
  • Experience with healthcare data, including understanding of data privacy and compliance requirements

What we offer

  • Competitive healthcare benefits
  • Generous equity compensation
  • Unlimited vacation
  • Membership in the First Round Network (a curated and confidential community with events, guides, thousands of Q&A questions, and opportunities for 1-1 mentorship)

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Senior Software Engineer, Core Data

8 matching positions

Senior Software Engineer - Core Java & Apache Spark

We are hiring an elite Senior Software Engineer to build and scale our core data...
Location
Location
India , Chennai, Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Core Java & JVM: Expert-level proficiency in Java, including the Collections Framework, Lambdas, and the Java Concurrency API. Demonstrable experience tuning the JVM and troubleshooting memory/GC issues
  • Apache Spark: Proven, hands-on experience developing, deploying, and tuning complex Spark applications for large-scale data transformation and analysis
  • Spring Ecosystem: Extensive, practical experience with the Spring Framework, particularly Spring Boot, Spring Data, and Spring Batch in a production environment
  • Data Structures & Algorithms: Deep understanding of fundamental data structures and algorithms, with a focus on their application in distributed computing and performance-critical systems
  • Containerization & Cloud-Native: Hands-on experience with Docker for building images and Kubernetes/OpenShift for deploying and managing distributed applications
  • Database Engineering: Strong command of SQL and relational database design, including transaction management and indexing. Experience with at least one production NoSQL database (MongoDB, Graph DB, etc.)
  • Architectural Design: Practical application of OOP, SOLID, and DDD principles to build maintainable and scalable systems. You write tests first (TDD) and believe in robust, automated testing
Job Responsibility
Job Responsibility
  • Architect & Build: Design and construct high-throughput, low-latency data processing pipelines using Apache Spark and the Spring ecosystem
  • Performance Engineering: Dive deep into JVM internals, garbage collection tuning, and Spark job optimization to maximize performance and resource efficiency
  • Distributed Systems Design: Implement scalable, resilient, and transactional architectures leveraging container orchestration (Kubernetes/OpenShift) and distributed data stores
  • Code & Design Excellence: Champion and enforce best practices in software engineering, including SOLID principles, advanced design patterns, Domain-Driven Design (DDD), and Test-Driven Development (TDD)
  • Database Mastery: Engineer and optimize data models for both relational and NoSQL databases, ensuring data integrity, performance, and scalability
  • CI/CD Automation: Own and enhance CI/CD pipelines for automated build, test, and deployment of Java applications and Spark jobs in a containerized environment
  • Technical Leadership: Lead design and code reviews, mentor junior engineers, and drive the adoption of new technologies and architectural patterns across the team
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - Data Platform, AI Infrastructure

We are building a large-scale, productized data platform that powers critical in...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.
  • Strong programming experience in Python
  • Experience building and operating large-scale distributed systems
  • Hands-on experience with: Backend services or APIs (e.g., FastAPI, Flask, or similar)
  • Cloud-based infrastructure (Azure, AWS, or GCP)
  • Monitoring and observability systems (metrics, logging, alerting)
  • Experience designing systems with reliability, scalability, and operational clarity in mind
  • Proven ability to own and deliver production systems end-to-end
  • Ability to break down ambiguous problems, ask the right questions, and execute effectively
Job Responsibility
Job Responsibility
  • Design, build, and operate core components of a distributed data platform, including: Orchestration systems (e.g., Airflow or equivalent)
  • Backend services and APIs (Python/FastAPI or similar)
  • Monitoring, alerting, and reliability systems
  • Own the end-to-end lifecycle of platform components - from design through deployment, scaling, and maintenance
  • Ensure systems meet requirements for availability, performance, and data reliability at large scale
  • Define and enforce standardized patterns for infrastructure, deployment, and observability across the platform
  • Partner with data engineering teams to enable efficient, reliable data processing workflows
  • Diagnose and resolve complex issues in distributed systems, including performance bottlenecks and failure modes
  • Contribute to infrastructure-as-code and deployment systems to support reproducibility and operational excellence
  • Drive continuous improvements in system robustness, cost efficiency, and operational clarity
What we offer
What we offer
  • Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - Data Acquisition

Join TxODDS as a Senior Software Engineer and help build scalable, high-performa...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
txodds.net Logo
TXODDS
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong experience with at least one core programming language (e.g. Python, Java, Scala)
  • Hands-on experience with Kubernetes, container orchestration, and Docker
  • Experience working with distributed systems and event‑driven technologies (e.g. Kafka)
  • Solid understanding of networking fundamentals (HTTP, APIs)
  • Experience with relational and NoSQL databases
  • Strong Git skills and familiarity with modern development practices (code reviews, testing, CI/CD)
  • Comfort working in a Linux/Unix command-line environment
  • Experience designing and debugging software from inception to deployment
  • Excellent problem‑solving skills and a proactive approach to improving systems and processes
  • Strong communication and collaboration skills, and the ability to work effectively across teams
Job Responsibility
Job Responsibility
  • Developing, testing, and deploying high‑quality software that processes data from diverse sources
  • Building, improving, and maintaining distributed systems and data pipelines (including Kafka-based services)
  • Deploying and supporting containerised workloads running in Kubernetes environments
  • Creating and maintaining clear, accurate documentation for the systems you build
  • Validating and monitoring data quality using internal tools and processes
  • Supporting data‑gathering workflows, including those involving web‑scraping or automated data acquisition
  • Investigating and resolving data‑related issues escalated from the Client Services team
  • Participating in an out‑of‑hours on‑call rotation to support critical data acquisition systems
  • Sharing knowledge widely and contributing to a positive, collaborative team culture
  • Mentoring junior engineers and helping raise the overall technical bar
What we offer
What we offer
  • Competitive benefits package tailored to your location
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - Core AI

The Experimentation Platform (ExP) team powers Microsoft’s end-to-end experiment...
Location
Location
China , Beijing
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience in software engineering OR equivalent experience
  • Strong coding skills in one or more languages: C#, Java, Scala, Python, or C++
  • Proven ability to design and deliver components of large distributed systems or data platforms
  • Solid understanding of data structures, algorithms, distributed system fundamentals, and debugging complex systems
  • Fast learner with strong communication skills and the ability to work effectively across teams
Job Responsibility
Job Responsibility
  • Design and build components of the experimentation platform, including large-scale data pipelines, metrics/analysis authoring, distributed workflows, and critical experiment lifecycle services
  • Develop high-performance and reliable systems using C#, .NET, Spark pipelines, Azure services, and modern data/compute frameworks
  • Integrate AI-driven agents and automation to improve debugging, monitoring, validation, and self-service capabilities for experiment owners
  • Optimize efficiency, cost, resource utilization, and pipeline throughput across a global fleet
  • Improve platform fundamentals including observability, reliability, test automation and recovery workflows
  • Collaborate with PMs, data scientists, platform engineers, and partner teams (across global Microsoft engineering centers) to design scalable solutions
  • Participate in code reviews, incident response, performance tuning, and architectural discussions
  • Engineer features with a strong focus on quality, transparency, resiliency, and operational excellence
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Data

Patreon is looking for a Senior Software Engineer, Data to support our mission. ...
Location
Location
United States , San Francisco; New York
Salary
Salary:
200000.00 - 300000.00 USD / Year
patreon.com Logo
Patreon
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years of experience in software development
  • At least 2+ years of experience in building scalable, production-grade data pipelines
  • Familiarity with SQL and distributed data processing tools like Spark, Flink, Kafka Streams, or similar
  • Strong programming foundations in Python or similar language, with good software engineering design patterns and principles (testing, CI/CD, monitoring)
  • Familiar with modern data lakes (eg: Delta Lake, Iceberg)
  • Familiar with data warehouses (eg: Snowflake, Redshift, BigQuery) and production data stores such as relational (eg: MySQL, PostgreSQL), object (eg: S3), key-value (eg: DynamoDB) and message queues (eg: Kinesis, Kafka)
  • Excellent collaboration and communication skills
  • comfortable partnering with non-technical stakeholders, writing crisp design docs, giving actionable feedback, and can influence without authority across teams
  • Understanding of data modeling and metric design principles
  • Passionate about data quality, system reliability, and empowering others through well-crafted data assets
Job Responsibility
Job Responsibility
  • Design, build, and maintain the pipelines that power all data use cases. This includes ingestion of raw data from production databases, object storage, and message queues, and vendors into our Data Lake, and building core datasets and metrics
  • Develop intuitive, performant, and scalable data models (facts, dimensions, aggregations) that support product features, internal analytics, experimentation, and machine learning workloads
  • Implement robust batch and streaming pipelines using Spark, Python, and Airflow
  • Build pipelines adhering to standards for accuracy, completeness, lineage, and dependency management. Build monitoring and observability so teams can trust what they’re using
  • Work with Product, Data Science, Infrastructure, Finance, Marketing, and Sales to turn ambiguous questions into well-scoped, high-impact data solutions
  • Pay down technical debt, improve automation, and follow best practices in data modeling, testing, and reliability. Mentor peers earlier in their career within the team
What we offer
What we offer
  • Equity plans
  • Healthcare
  • Flexible time off
  • Company holidays and recharge days
  • Commuter benefits
  • Lifestyle stipends
  • Learning and development stipends
  • Patronage
  • Parental leave
  • 401k plan with matching
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Data Platform

We’re looking for a seasoned software engineer to join Parafin’s Infrastructure ...
Location
Location
United States , San Francisco
Salary
Salary:
230000.00 - 265000.00 USD / Year
parafin.com Logo
Parafin
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years of experience in software engineering with a strong background in data infrastructure, pipelines, and distributed systems
  • Advanced proficiency in Python and SQL
  • Hands-on Spark development experience
  • Expertise with modern cloud data stacks—AWS (S3, RDS), Databricks, and Airflow—and lakehouse architectures
  • Hands‑on experience with foundational data‑infrastructure technologies such as Hadoop, Hive, Kafka (or similar streaming platforms), Delta Lake/Iceberg, and distributed query engines like Trino/Presto
  • Familiarity with ingestion frameworks, developer‑experience tooling, and best practices for data versioning, lineage, partitioning, and clustering
  • Strong problem-solving skills and a proactive attitude toward ownership and platform health
  • Excellent communication and collaboration skills, especially in cross-functional settings
Job Responsibility
Job Responsibility
  • Design and build robust, highly scalable data pipelines and lakehouse infrastructure with PySpark, Databricks, and Airflow on AWS
  • Improve the data platform development experience for Engineering, Data Science, and Product by creating intuitive abstractions, self‑service tooling, and clear documentation
  • Own and maintain core data pipelines and models that power internal dashboards, ML models, and customer-facing products
  • Own the Data & ML platform infrastructure using Terraform, including end‑to‑end administration of Databricks workspaces: manage user access, monitor performance, optimize configurations (e.g., clusters, lakehouse settings), and ensure high availability of data pipelines
  • Lead projects to improve data quality, testing, observability, and cost efficiency across existing pipelines and backend systems (e.g., migrating Databricks SQL pipelines to dbt, scaling data ingestion, improving data-lineage tracking, and enhancing monitoring)
  • Act as the primary engineering partner for the Data Science team—embedded closely to gather requirements, design scalable solutions, and provide end-to-end support on all engineering aspects of their work
  • Work closely with backend engineers and data scientists to design performant data models and support new product development initiatives
  • Share best practices and mentor other engineers working on data-centric systems
What we offer
What we offer
  • Equity grant
  • Medical, dental & vision insurance
  • Work from home flexibility
  • Unlimited PTO
  • Commuter benefits
  • Free lunches
  • Paid parental leave
  • 401(k)
  • Employee assistance program
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Core Services

Lime is looking for an experienced Backend Software Engineer to join the team an...
Location
Location
Canada
Salary
Salary:
136000.00 - 170000.00 CAD / Year
li.me Logo
Lime
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of industry professional software development experience and a track record of building impactful products or systems
  • Skilled at writing logical and maintainable code, and providing clear feedback in code reviews
  • Customer obsessed
  • Data driven
  • Owner’s mindset
  • Outcomes oriented
  • Strong influencer
  • Strong collaborator
  • Technical proficiency to understand and contribute to the design and implementation of projects and ideas
  • Excellent communication and effective prioritization skills in working with competing priorities
Job Responsibility
Job Responsibility
  • Identify, define, and solve strategically essential technical problems, including architecting and building highly scalable foundational platforms and revamping existing core application experiences
  • Scrutinize and reason clearly about the technology and architecture we make
  • Building scalable, efficient, and stable software to power our electric vehicles and related features
  • Participate in cross-functional collaboration with PMs, Engineers, Design, Data Scientists, Research, and Operations teams
  • Advise Lime’s leadership team on key technology adoption and process improvements
  • Contribute to shaping the overall engineering culture at the company
What we offer
What we offer
  • Offers Equity
  • Offers Commission
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Core Exchange

Join us in building the future of finance. Our mission is to democratize finance...
Location
Location
Slovenia , Ljubljana
Salary
Salary:
Not provided
robinhood.com Logo
Robinhood
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of backend engineering experience, with deep expertise in Golang
  • Proven experience designing high-performance systems with low-latency and high-throughput requirements
  • Familiarity with cloud infrastructure, especially AWS services
  • Solid database experience: SQL (MySQL) and/or NoSQL (DynamoDB)
  • Experience working on or with production-grade trading APIs or high-volume systems
  • Structured thinker with strong communication skills and a customer-first mindset
Job Responsibility
Job Responsibility
  • Design and build performant, reliable, and scalable services for our API trading ecosystem
  • Optimize low-latency data flows and trading paths, ensuring correctness under pressure
  • Own and evolve core components of our REST, FIX, and WebSocket APIs
  • Collaborate with product, infrastructure, and frontend teams to deliver seamless API solutions
  • Participate in the team’s on-call rotation, supporting mission-critical systems used by our most active clients
  • Continuously measure, monitor, and improve the trading API experience
Read More
Arrow Right