CrawlJobs Logo
Astronomer Logo Astronomer · IT - Software Development

Staff Software Engineer, Data

United States, New York City 215000.00 - 250000.00 USD / Year · Job Posted February 14, 2026
Apply Position
Job Link Share

Job Description

At Astronomer, we’re redefining how companies run Apache Airflow at scale. Our R&D organization is home to some of the most innovative minds in cloud infrastructure and open-source software. You can’t spell “Data Orchestration” without “Data”, and we’re looking to add a world-class Staff+ level engineer to our team, to set out our Data and Database story as we level up our platform’s Reliability, cost profile and growth trajectory. You get to go in at the ground level of how our production infrastructure is designed, built, tested and deployed. Your work will directly influence how we build Astro, Observe and our IDE product, as well as how global organizations orchestrate data pipelines at scale—making them faster, more reliable, and easier to manage.

Job Responsibility

  • Be a subject-matter expert in how we treat Data at scale
  • Recommend, design and later lead building the data systems that will help us continue to scale
  • Own and develop our Database and overall data strategy and practice
  • Be directly involved in decision-making on what we work on, as well as how we work on it
  • Make build vs. buy assessments, and advocate for the right tools for the right job when it comes to data
  • Create and maintain comprehensive internal documentation and decision records for systems and processes, ensuring clarity and accessibility
  • Participate in Architectural forums and discussion and make principled, open decisions

Requirements

  • Hands-on experience designing, developing, and scaling production infrastructure
  • Extensive knowledge of Postgres and Postgres-like cloud offerings (AWS, GCP, Azure)
  • Extensive and recent experience with building low-level data systems and/or managed data platforms
  • In-depth knowledge and practical experience of the product and technology space in Database and related fields (Blob, NoSQL, Timeseries, Graph, Vector)
  • Experience defining requirements and making and justifying technology choices around the data space
  • Strong experience in Non-Abstract Systems design and implementation
  • Strong proficiency in Golang and in-depth experience with Kubernetes
  • Strong communication skills, both written and verbal, with experience in working with a globally distributed team in delivery

Nice to have

  • Experience working with Spanner, AlloyDB and/or other cloud-native databases, including hands-on experience with provisioning, development practices and migration of data
  • Experience working on a SaaS/PaaS product across multiple cloud providers
  • Experience building internal data platforms from cloud-native component parts
  • Experience with Apache Airflow

What we offer

  • equity component
  • comprehensive benefits package
Astronomer - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Staff Software Engineer, Data

8 matching positions

Staff Software Engineer, Data

Patreon is looking for a Staff Software Engineer to support our mission. The Dat...
Location
Location
United States , San Francisco; New York
Salary
Salary:
246400.00 - 369600.00 USD / Year
patreon.com Logo
Patreon
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of experience in software development
  • At least 2+ years of experience in building scalable, production-grade data pipelines
  • Expert-level proficiency in SQL and distributed data processing tools like Spark, Flink, Kafka Streams, or similar
  • Strong programming foundations in Python or similar language, with good software engineering design patterns and principles (testing, CI/CD, monitoring)
  • Expert in modern data lakes (eg: Delta Lake, Iceberg)
  • Familiar with data warehouses (eg: Snowflake, Redshift, BigQuery) and production data stores such as relational (eg: MySQL, PostgreSQL), object (eg: S3), key-value (eg: DynamoDB) and message queues (eg: Kinesis, Kafka)
  • Excellent collaboration and communication skills
  • Understanding of data modeling and metric design principles
  • Passionate about data quality, system reliability, and empowering others through well-crafted data assets
  • Highly motivated self-starter who thrives in a collaborative, fast-paced environment and takes pride in high-craft, high-impact work
Job Responsibility
Job Responsibility
  • Design, build, and maintain the pipelines that power all data use cases
  • Develop intuitive, performant, and scalable data models (facts, dimensions, aggregations) that support product features, internal analytics, experimentation, and machine learning workloads
  • Implement robust batch and streaming pipelines using Spark, Python, and Airflow
  • Define and enforce standards for accuracy, completeness, lineage, and dependency management
  • Work with Product, Data Science, Infrastructure, Finance, Marketing, and Sales to turn ambiguous questions into well-scoped, high-impact data solutions
  • Pay down technical debt, improve automation, and drive best practices in data modeling, testing, and reliability
  • Mentor peers and help shape the future of Patreon’s data
What we offer
What we offer
  • Equity plans
  • Healthcare
  • Flexible time off
  • Company holidays and recharge days
  • Commuter benefits
  • Lifestyle stipends
  • Learning and development stipends
  • Patronage
  • Parental leave
  • 401k plan with matching
  • Fulltime
Read More
Arrow Right

Staff Software Engineer - Data

Uber's mission is to reimagine the way the world moves for the better. Here, bol...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
uber.com Logo
Uber
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Education: Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field
  • Experience Level: 10+ years of hands-on experience in Data Engineering, with a proven track record of delivering results at a Staff Engineer level (or equivalent scope) at a premier technology company
  • Expert SQL Competency: 10+ years of hands-on, expert-level SQL experience
  • Data Modeling & Warehousing: Extensive experience designing dimensional data models (Star/Snowflake schemas) and data warehouses
  • Software Engineering Fundamentals: Proficiency in at least one high-level programming language (Java, Scala, Python, or Go)
  • Big Data Ecosystem: 10+ years of experience working with distributed data systems (Hadoop, Hive, Spark) and MPP databases (Vertica, Redshift, etc.)
  • End-to-End Architecture: Experience designing full-lifecycle data systems, including logging, ingestion (Batch/Stream), quality frameworks, and monitoring
  • Technical Leadership: Excellent written and verbal communication skills
  • Mentorship & Growth: A strong passion for driving engineering excellence and mentoring engineers
Job Responsibility
Job Responsibility
  • Own the Technical Vision: You will own and drive the technical roadmap for the Payments data ecosystem, balancing long-term architectural scalability with short-term business critical deliveries
  • Navigate Ambiguity: Actively identify strategically important problems and inefficiencies without waiting for instruction
  • Drive Alignment: See the big picture and drive consensus on complex technical decisions across the organization
  • Architect at Scale: Design and implement resilient, cost-effective, and high-scale batch and streaming pipelines that power critical support operations and financial analytics
  • Elevate Data Standards: Define and enforce robust data modeling standards, data contracts, and governance frameworks
  • Optimize & Automate: Identify opportunities to automate manual workflows (like SLA tracking and issue detection) and optimize infrastructure efficiency to lower TCO
  • Raise the Bar: Champion sustainable engineering practices
  • Be a Trusted Mentor: Serve as a humble mentor and technical advisor to both junior engineers and peer leaders
  • Force Multiplier: Act as a role model for judgment and responsibility
  • Fulltime
Read More
Arrow Right

Staff Software Engineer (Data)

As a Staff Software Engineer, you will be a technical leader in our Data Science...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
arrive.com Logo
Arrive
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Extensive history of building and scaling data-intensive applications in production, with a track record of leading technical initiatives from conception to deployment
  • Expert-level Python and its data ecosystem (Numpy, Pandas), including designing frameworks for data tasks
  • Deep understanding of distributed data processing engines like Apache Spark
  • A strong command of Linux, containers (Docker), and infrastructure as code for cloud deployments (AWS preferred)
  • A passion for elevating engineering standards through pair programming and detailed code reviews to help other engineers grow their technical depth
Job Responsibility
Job Responsibility
  • Architect and Implement: Own the technical roadmap for our Spark-based data processes to ensure our Airflow pipelines are performant, cost-effective, and scalable. You will enhance our existing services through hands-on development and solve any complex performance bottlenecks, concurrency issues, and systemic bugs
  • Drive Engineering Excellence: Define standards for efficient, testable and reusable Python code across the organisation that ensure our services remain reliable, robust, and easy for other engineers to extend
  • Bridge Strategy and Execution: Partner with Data Scientists to translate modeling requirements into high-performance production services. You will design the architectures necessary to meet sophisticated data-serving needs, ensuring our parking and EV products remain accurate and responsive at scale
  • Modernize Infrastructure: Evolve our infrastructure-as-code (AWS) and CI/CD pipelines to keep up with cutting-edge approaches. You will personally contribute to the automation and observability patterns that allow us to deploy fresh data and production services with high confidence and zero downtime
  • Advance Data Capabilities: Lead the hands-on development of platform enhancements, such as establishing feature stores for machine learning and building automated data monitoring systems to ensure data integrity and model reproducibility
  • Scale AI Practices: Lead the adoption of AI throughout the software development lifecycle, evolving our internal coding practices while ensuring systems remain reliable and maintainable
What we offer
What we offer
  • Flexible working - hybrid home and office-based opportunities
  • Paid Leave if you participate in an event for Charity
  • 25 Days holiday entitlement
  • An enhanced Workplace Pension Scheme - 5% by Arrive, 3% by you
  • Private Medical Health Insurance
  • Fantastic wellbeing programmes, including On-site Sports massages, Reiki and Head massages every week
  • Discounted gym membership
  • Access to Blue Call, a mental health support platform
  • Enhanced Maternity and Paternity offering
  • Fulltime
Read More
Arrow Right

Senior Staff Software Engineer: Data & Storage Platform

Uber’s Data Platform is the heart of the company’s critical decision-making and ...
Location
Location
United States , Seattle; San Francisco; Sunnyvale
Salary
Salary:
267000.00 - 297000.00 USD / Year
uber.com Logo
Uber
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 14+ Years of Engineering Excellence: Proven experience designing and operating world-class distributed data and storage systems
  • Mastery of Storage Internals: Extensive storage experience is a must
  • Deep expertise in: Batch & Object Storage: HDFS, Cloud Object Storage (S3/GCS/OCI), and Blobstore metadata management
  • Storage Optimization: Practical experience with Apache Hudi or Apache Iceberg for lakehouse architectures
  • Transactional Systems: Experience with distributed transactional storage (e.g., Docstore, Google Spanner, TiDB)
  • NoSQL & Cache: Cassandra, Redis, and high-throughput Key-Value stores
  • Data + AI Convergence: Deep understanding of how compute fabrics (Spark, Flink, Ray) integrate with vector databases and model-serving platforms
  • Query Engine Proficiency: Architect-level knowledge of Presto, Trino, or Hive for large-scale analytical processing
  • Systems Programming: Expert-level command of Java, Go, Scala, or C++ with a focus on performance tuning and distributed consensus
Job Responsibility
Job Responsibility
  • Architect the Multi-Modal Fabric: Unify batch, streaming, and AI compute into one intelligent fabric, enabling real-time insights and trustworthy AI agents at a global scale
  • Revolutionize Storage & Catalog: Drive the architecture for a unified catalog and metadata management service for unstructured data, leveraging native cloud object store capabilities
  • Operationalize AI Intelligence: Partner with teams like QueryCopilot and DataIQ to bridge human validation with autonomous reasoning through agentic workflows
  • Lead Storage Modernization: Evolve our massive-scale persistence layers—including Docstore (Transactional Distributed Storage) and Distributed MySQL—to increase resiliency and reduce operational overhead
  • Open Source & Act as a force multiplier by contributing to the community (Hudi, Iceberg, Presto)
What we offer
What we offer
  • Eligible to participate in Uber's bonus program
  • May be offered an equity award & other types of comp
  • All full-time employees are eligible to participate in a 401(k) plan
  • Eligible for various benefits
  • Fulltime
Read More
Arrow Right

Staff Software Engineer, Data Infrastructure

At Docker, we make app development easier so developers can focus on what matter...
Location
Location
United States , Seattle
Salary
Salary:
195400.00 - 275550.00 USD / Year
docker.com Logo
Docker
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of software engineering experience with 3+ years focused on data engineering and analytics systems
  • Expert-level experience with Snowflake including advanced SQL, performance optimization, and cost management
  • Deep proficiency in DBT for data modeling, transformation, and testing with experience in large-scale implementations
  • Strong expertise with Apache Airflow for complex workflow orchestration and pipeline management
  • Hands-on experience with Sigma or similar modern BI platforms for self-service analytics
  • Extensive AWS experience including data services (S3, Redshift, EMR, Glue, Lambda, Kinesis) and infrastructure management
  • Proficiency in Python, SQL, and other programming languages commonly used in data engineering
  • Experience with infrastructure-as-code, CI/CD practices, and modern DevOps tools
  • Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience
  • Proven track record designing and implementing large-scale distributed data systems
Job Responsibility
Job Responsibility
  • Define and drive the technical strategy for Docker's data platform architecture, establishing long-term vision for scalable data systems
  • Lead design and implementation of highly scalable data infrastructure leveraging Snowflake, AWS, Airflow, DBT, and Sigma
  • Architect end-to-end data pipelines supporting real-time and batch analytics across Docker's product ecosystem
  • Drive technical decision-making around data platform technologies, architectural patterns, and engineering best practices
  • Establish technical standards for data quality, testing, monitoring, and operational excellence
  • Design and build robust, scalable data systems that process petabytes of data and support millions of user interactions
  • Implement complex data transformations and modeling using DBT for analytics and business intelligence use cases
  • Develop and maintain sophisticated data orchestration workflows using Apache Airflow
  • Optimize Snowflake performance and cost efficiency while ensuring reliability and scalability
  • Build data APIs and services that enable self-service analytics and integration with downstream systems
What we offer
What we offer
  • Freedom & flexibility
  • fit your work around your life
  • Designated quarterly Whaleness Days plus end of year Whaleness break
  • Home office setup
  • we want you comfortable while you work
  • 16 weeks of paid Parental leave
  • Technology stipend equivalent to $100 net/month
  • PTO plan that encourages you to take time to do the things you enjoy
  • Training stipend for conferences, courses and classes
  • Equity
  • Fulltime
Read More
Arrow Right

Staff Software Engineer, Data Platform

Lead the design and development of core data storage, streaming, caching, and in...
Location
Location
United States , San Francisco; Seattle; New York
Salary
Salary:
248400.00 - 310500.00 USD / Year
scale.com Logo
Scale
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of full-time engineering experience, post-graduation with specialties in back-end systems, specifically related to building large-scale data storage, streaming, and warehousing systems
  • Extensive experience in various database technologies (MongoDB, Postgres), streaming/processing solutions (Kinesis, Flink, Spark), indexing/caching (ElasticSearch, Redis), and various data query engines (Trino, Presto, Snowflake, etc.)
  • Show a track record of mentoring and leading teams in successful projects
  • Possess excellent communication and collaboration skills, and the ability to translate complex technical concepts to non-technical stakeholders
  • Experience working fluently with standard containerization & deployment technologies like Kubernetes and various public cloud offerings
  • Extensive experience in software development and a deep understanding of distributed systems, cloud platforms and data systems
  • Experience driving cross functional collaboration and communication at an organizational or broader level
Job Responsibility
Job Responsibility
  • Drive the architecture, design, implementation, and reliability of our foundational data platforms and systems, working closely with stakeholders and internal customers to understand and refine requirements
  • Collaborate with cross-functional teams to define, design, and deliver new features
  • Proactively identify opportunities for, and driving improvements to, current programming practices, including process enhancements and tool upgrades
  • Present technical information to teams and stakeholders, providing guidance and insight on development processes and technologies
  • Provide technical leadership, including: upholding and upleveling engineering standards across the organization, mentoring junior engineers
What we offer
What we offer
  • Comprehensive health, dental and vision coverage
  • retirement benefits
  • a learning and development stipend
  • generous PTO
  • equity grant
  • commuter stipend
  • Fulltime
Read More
Arrow Right

Staff Software Engineer, Data Engineering

You'll own Gamma's data infrastructure and architecture as we scale to hundreds ...
Location
Location
United States , San Francisco
Salary
Salary:
230000.00 - 310000.00 USD / Year
gamma.app Logo
Gamma
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of experience as a data engineer or software engineer working on data infrastructure with deep expertise in distributed systems
  • Expert-level knowledge of event streaming platforms, especially Apache Kafka (producers, consumers, Kafka Connect, stream processing)
  • Extensive hands-on experience with Snowflake, including performance optimization, cost management, and data modeling at massive scale
  • Strong understanding of relational databases (particularly Postgres) and experience with CDC patterns and replication strategies in distributed environments
  • Proven track record architecting and leading major data infrastructure initiatives that handled orders of magnitude growth
  • Experience establishing data engineering best practices and driving technical strategy across organizations
  • Strong communication skills and experience influencing technical direction across engineering, analytics, and leadership
Job Responsibility
Job Responsibility
  • Own and evolve our end-to-end event pipeline architecture, from Kafka ingestion through Snowflake analytics, setting technical direction for data infrastructure
  • Design and architect distributed data systems that scale to orders of magnitude more data volume while maintaining world-class query performance
  • Lead initiatives to build and optimize CDC (change data capture) pipelines and streaming data transformations at massive scale
  • Establish best practices for data quality, pipeline reliability, and system observability across the organization
  • Drive strategic technical decisions about data modeling, infrastructure architecture, and technology choices
  • Mentor engineers and elevate data engineering practices across analytics, product, and engineering teams
What we offer
What we offer
  • competitive equity
  • Fulltime
Read More
Arrow Right

Software Engineer Staff - Data Scientist

Designs, develops, troubleshoots and debugs software programs for software enhan...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
https://www.hpe.com/ Logo
Hewlett Packard Enterprise
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Masters or PhD in Computer Science, Electrical Engineering, Statistics, Applied Math or equivalent fields with strong mathematical background
  • Proficiency in Python, R, SQL, or other programming languages for data analysis
  • Experience with data wrangling, manipulation, and visualization tools and libraries such as pandas, numpy, scikit-learn, matplotlib, seaborn, etc.
  • Knowledge of machine learning concepts and techniques such as supervised and unsupervised learning, regression, classification, clustering, dimensionality reduction, etc.
  • Familiarity with cloud computing platforms and services such as AWS, Azure, or Google Cloud
  • Strong analytical and problem-solving skills
Job Responsibility
Job Responsibility
  • Collect, clean, and transform data from various sources and formats for model training
  • Perform exploratory data analysis and visualization to understand patterns and trends
  • Build, test, and deploy predictive models and algorithms using appropriate tools and frameworks to a production cloud environment
  • Communicate findings and recommendations to stakeholders and clients using clear and compelling reports and presentations
  • Collaborate with other data scientists, engineers, and domain experts on cross-functional projects
What we offer
What we offer
  • Comprehensive suite of benefits that supports physical, financial and emotional wellbeing
  • Programs for personal and professional development
  • Inclusive environment that celebrates individual uniqueness
  • Fulltime
Read More
Arrow Right