CrawlJobs Logo
Fullstory Logo Fullstory · IT - Software Development

Senior Software Engineer, Data Ingestion

United States, Atlanta Employment contract 160000.00 - 180000.00 USD / Year · Job Posted May 30, 2026
Apply Position
Job Link Share

Job Description

Fullstory’s mission is to help teams create amazing online experiences for their users. We do this by applying deep expertise in distributed systems and computer science to build an ambitiously high-scale and high-value platform. The scale at which Fullstory ingests, processes, and persists data makes our systems uniquely challenging to design and operate. Our Data Ingestion services are responsible for processing hundreds of thousands of browser and mobile application event bundles every second and quickly transforming this data into easily consumable activity for machines and humans alike. As a Senior Software Engineer focused on our ingestion pipeline, you will be working on evolving our highly reliable, secure, performant, and cost efficient foundation for the capture and processing of session data. This data feeds both the product analytics features in our own application as well as data analytics and machine learning systems, where streaming accurate data as it is being captured can power further insights and interactivity for our customers.

Job Responsibility

  • Engineer distributed systems that operate at tens to hundreds of thousands of requests per second using Go, Kubernetes, and GCP
  • Explore ideas about how to unlock new features through thoughtful architecture and framework designs
  • Ensure the quality and reliability of Fullstory's capture and extraction systems across many services and downstream applications both internally and externally
  • Collaborate with technical leaders and product experts to evolve the technical roadmap for Ingestion services, and to participate in collaborative development efforts across the Engineering organization

Requirements

  • You possess a strong understanding the nuances of distributed architectures and tackling capacity and performance challenges when dealing with data at scale
  • You have experience writing Golang code in production
  • You have experience with Kubernetes and supporting highly available and reliable cloud based microservices in production
  • You have worked on asynchronous or streaming ingestion and processing systems and frameworks
  • You leverage AI tools to enhance work quality by implementing AI solutions that optimize efficiency

What we offer

  • Flexibility and Connection
  • flexible PTO policy
  • annual company-wide closure
  • federal holidays
  • sponsored benefit packages for US-based Fullstorians
  • supplemental coverage options for international Fullstorians
  • professional development opportunities through training programs
  • annual learning subsidy for US and EMEA-based employees
  • monthly productivity stipend
  • team off-sites
  • annual full-company meet-up
  • paid parental leave
  • bereavement leave, including miscarriage/pregnancy loss
Fullstory - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Senior Software Engineer, Data Ingestion

8 matching positions

Senior Software Engineer, Data Pipeline

Fullstory’s mission is to help teams create amazing online experiences for their...
Location
Location
United States , Atlanta
Salary
Salary:
160000.00 - 180000.00 USD / Year
fullstory.com Logo
Fullstory
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong understanding of the nuances of distributed architectures and tackling capacity and performance challenges when dealing with data at scale
  • Experience writing Golang code in production
  • Experience with Kubernetes and supporting highly available and reliable cloud based microservices in production
  • Worked on asynchronous or streaming ingestion and processing systems and frameworks
  • Leverage AI tools to enhance work quality by implementing AI solutions that optimize efficiency
Job Responsibility
Job Responsibility
  • Engineer distributed systems that operate at tens to hundreds of thousands of requests per second using Go, Kubernetes, and GCP
  • Explore ideas about how to unlock new features through thoughtful architecture and framework designs
  • Ensure the quality and reliability of Fullstory's capture and extraction systems across many services and downstream applications both internally and externally
  • Collaborate with technical leaders and product experts to evolve the technical roadmap for Ingestion services, and to participate in collaborative development efforts across the Engineering organization
What we offer
What we offer
  • Flexibility and Connection with vibrant HQ in Atlanta and a tight-knit group in London
  • Flexible PTO policy and an annual company-wide closure, along with federal holidays
  • Sponsored benefit packages for US-based Fullstorians, and supplemental coverage options for international Fullstorians
  • Professional development opportunities through training programs and an annual learning subsidy for US and EMEA-based employees
  • Monthly productivity stipend for US and EMEA-based Fullstorians
  • Team Collaboration through team off-sites and an annual full-company meet-up
  • Paid parental leave
  • Bereavement leave, including miscarriage/pregnancy loss
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Data

Patreon is looking for a Senior Software Engineer, Data to support our mission. ...
Location
Location
United States , San Francisco; New York
Salary
Salary:
200000.00 - 300000.00 USD / Year
patreon.com Logo
Patreon
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years of experience in software development
  • At least 2+ years of experience in building scalable, production-grade data pipelines
  • Familiarity with SQL and distributed data processing tools like Spark, Flink, Kafka Streams, or similar
  • Strong programming foundations in Python or similar language, with good software engineering design patterns and principles (testing, CI/CD, monitoring)
  • Familiar with modern data lakes (eg: Delta Lake, Iceberg)
  • Familiar with data warehouses (eg: Snowflake, Redshift, BigQuery) and production data stores such as relational (eg: MySQL, PostgreSQL), object (eg: S3), key-value (eg: DynamoDB) and message queues (eg: Kinesis, Kafka)
  • Excellent collaboration and communication skills
  • comfortable partnering with non-technical stakeholders, writing crisp design docs, giving actionable feedback, and can influence without authority across teams
  • Understanding of data modeling and metric design principles
  • Passionate about data quality, system reliability, and empowering others through well-crafted data assets
Job Responsibility
Job Responsibility
  • Design, build, and maintain the pipelines that power all data use cases. This includes ingestion of raw data from production databases, object storage, and message queues, and vendors into our Data Lake, and building core datasets and metrics
  • Develop intuitive, performant, and scalable data models (facts, dimensions, aggregations) that support product features, internal analytics, experimentation, and machine learning workloads
  • Implement robust batch and streaming pipelines using Spark, Python, and Airflow
  • Build pipelines adhering to standards for accuracy, completeness, lineage, and dependency management. Build monitoring and observability so teams can trust what they’re using
  • Work with Product, Data Science, Infrastructure, Finance, Marketing, and Sales to turn ambiguous questions into well-scoped, high-impact data solutions
  • Pay down technical debt, improve automation, and follow best practices in data modeling, testing, and reliability. Mentor peers earlier in their career within the team
What we offer
What we offer
  • Equity plans
  • Healthcare
  • Flexible time off
  • Company holidays and recharge days
  • Commuter benefits
  • Lifestyle stipends
  • Learning and development stipends
  • Patronage
  • Parental leave
  • 401k plan with matching
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Data Acquisition

The Data Acquisition team within the Foundations organization at OpenAI is respo...
Location
Location
United States , San Francisco
Salary
Salary:
293000.00 - 385000.00 USD / Year
openai.com Logo
OpenAI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • BS/MS/PhD in Computer Science or a related field
  • 6+ years of industry experience in software development
  • Experience with large web crawlers a plus
  • Strong expertise in large stateful distributed systems and data processing
  • Proficiency in Kubernetes, and Infrastructure-as-Code concepts
  • Willingness and enthusiasm for trying new approaches and technologies
  • Ability to handle multiple tasks and adapt to changing priorities
  • Strong communication skills, both written and verbal
Job Responsibility
Job Responsibility
  • Own and lead engineering projects in the area of data acquisition including web crawling, data ingestion, and search
  • Collaborate with other sub-teams, such as Data Processing, Architecture, and Scaling, to ensure smooth data flow and system operability
  • Work closely with the legal team to handle any compliance or data privacy-related matters
  • Develop and deploy highly scalable distributed systems capable of handling petabytes of data
  • Architect and implement algorithms for data indexing and search capabilities
  • Build and maintain backend services for data storage, including work with key-value databases and synchronization
  • Deploy solutions in a Kubernetes Infrastructure-as-Code environment and perform routine system checks
  • Conduct and analyze experiments on data to provide insights into system performance
What we offer
What we offer
  • Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts
  • Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)
  • 401(k) retirement plan with employer match
  • Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)
  • Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees
  • 13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)
  • Mental health and wellness support
  • Employer-paid basic life and disability coverage
  • Annual learning and development stipend to fuel your professional growth
  • Daily meals in our offices, and meal delivery credits as eligible
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Data Platform

We’re looking for a seasoned software engineer to join Parafin’s Infrastructure ...
Location
Location
United States , San Francisco
Salary
Salary:
230000.00 - 265000.00 USD / Year
parafin.com Logo
Parafin
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years of experience in software engineering with a strong background in data infrastructure, pipelines, and distributed systems
  • Advanced proficiency in Python and SQL
  • Hands-on Spark development experience
  • Expertise with modern cloud data stacks—AWS (S3, RDS), Databricks, and Airflow—and lakehouse architectures
  • Hands‑on experience with foundational data‑infrastructure technologies such as Hadoop, Hive, Kafka (or similar streaming platforms), Delta Lake/Iceberg, and distributed query engines like Trino/Presto
  • Familiarity with ingestion frameworks, developer‑experience tooling, and best practices for data versioning, lineage, partitioning, and clustering
  • Strong problem-solving skills and a proactive attitude toward ownership and platform health
  • Excellent communication and collaboration skills, especially in cross-functional settings
Job Responsibility
Job Responsibility
  • Design and build robust, highly scalable data pipelines and lakehouse infrastructure with PySpark, Databricks, and Airflow on AWS
  • Improve the data platform development experience for Engineering, Data Science, and Product by creating intuitive abstractions, self‑service tooling, and clear documentation
  • Own and maintain core data pipelines and models that power internal dashboards, ML models, and customer-facing products
  • Own the Data & ML platform infrastructure using Terraform, including end‑to‑end administration of Databricks workspaces: manage user access, monitor performance, optimize configurations (e.g., clusters, lakehouse settings), and ensure high availability of data pipelines
  • Lead projects to improve data quality, testing, observability, and cost efficiency across existing pipelines and backend systems (e.g., migrating Databricks SQL pipelines to dbt, scaling data ingestion, improving data-lineage tracking, and enhancing monitoring)
  • Act as the primary engineering partner for the Data Science team—embedded closely to gather requirements, design scalable solutions, and provide end-to-end support on all engineering aspects of their work
  • Work closely with backend engineers and data scientists to design performant data models and support new product development initiatives
  • Share best practices and mentor other engineers working on data-centric systems
What we offer
What we offer
  • Equity grant
  • Medical, dental & vision insurance
  • Work from home flexibility
  • Unlimited PTO
  • Commuter benefits
  • Free lunches
  • Paid parental leave
  • 401(k)
  • Employee assistance program
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Data Infrastructure

LMArena is seeking a Software Engineer to join our team and build the data pipel...
Location
Location
United States , Bay Area
Salary
Salary:
Not provided
arena.ai Logo
Arena Intelligence, Inc.
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience in software engineering, with a dedicated focus on data engineering and big data technologies
  • Proficiency in SQL and at least one programming language commonly used for data analysis (Python (preferred), Scala, R)
  • Hands-on experience with data processing and pipeline frameworks (Apache Spark, Ray Data, etc.) and at least one popular big data analytics platform (Databricks, Snowflake)
  • Demonstrated experience in designing, implementing, optimizing, and debugging production data pipelines
Job Responsibility
Job Responsibility
  • Design and build robust data pipelines to ingest, process, and transform user vote data to features essential for model performance evaluation
  • Collaborate with researchers and product leadership to understand product goals and necessary data
  • Design and implement solutions to generate result dashboards and reports, providing useful information for the public, model providers, and researchers
  • Ensure the integrity, data quality, and reliability of the pipelines
  • Scale our data infrastructure to accommodate increasing data volumes and evolving analytical needs
What we offer
What we offer
  • Comprehensive health and wellness benefits, including medical, dental, vision, and additional support programs
  • The opportunity to work on cutting-edge AI with a small, mission-driven team
  • A culture that values transparency, trust, and community impact
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - Data Engineering

We are looking for a bright and exceptional Data Engineer to join our Technology...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
arcesium.com Logo
Arcesium
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in computer science, math, or other technical fields
  • 2-4 years of relevant experience
  • Experience building pipelines and ingesting data from APIs, SFTP, Databases, etc.
  • Strong proficiency in Python and PySpark for data engineering tasks
  • Ability to write complex queries for data transformations and analysis
  • Basic understanding of data modeling and data warehousing
  • Solid understanding of CI/CD, version control, and DevOps practices
  • Excellent problem-solving and troubleshooting skills
  • Proven track record for working in an agile and collaborative environment
Job Responsibility
Job Responsibility
  • Build and support connectors and data pipelines for data ingestion and processing on the in-house data platform to meet business requirements
  • Demonstrate strong SQL skills to write complex transformations
  • Demonstrate deep knowledge in Python and PySpark to code ETL blocks for the data pipelines
  • Troubleshoot and optimize ETL pipelines to minimize execution overheads
  • Drive improvements in performance, reliability, and scalability of data pipelines
  • Mentor other team members
Read More
Arrow Right

Senior Software Engineer, AI Data Platform (CoreAI)

Join Microsoft’s CoreAI – AI Platform team in Bay Area/Redmond to build the AI D...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years
Job Responsibility
Job Responsibility
  • Design and build scalable data pipelines and services to automate the dataset lifecycle (ingestion, registration, validation, PII handling, discovery, sharing, lineage), including intelligent agent-driven automation for key stages
  • Develop secure and reliable infrastructure for data access, entitlement management, and operational support across global time zones
  • Implement governance and compliance tooling to ensure data integrity, auditability, and adherence to regulatory standards
  • Create user-facing tools and APIs that make datasets easily discoverable and reusable
  • Contribute to strategic extensions such as continuous feedback loops, human-in-the-loop workflows, and data intelligence services for internal and external stakeholders
  • Collaborate with cross-org partners to align priorities and deliver company-wide impact
  • Fulltime
Read More
Arrow Right

Senior Data Engineer Lead / Architect - Senior Vice President

At Citi Services - Global Trade Technology Organization, we are on a mission to ...
Location
Location
India , Pune, Maharashtra, India, Chennai, Tamil Nadu, India
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of professional experience in data engineering, with a proven track record of designing and building large-scale data systems
  • 3+ years in a technical leadership or architect role, with experience mentoring junior and senior engineers
  • Expert-level proficiency in at least one programming language (Python or Scala preferred) and exceptional SQL skills
  • Proven hands-on experience with Python or Scala for data manipulation, scripting, machine learning, and backend development
  • Deep, hands-on experience with a major cloud platform (AWS, GCP, or Azure) and its data ecosystem (e.g., S3/GCS, Redshift/BigQuery, EMR/Dataproc, Kinesis/Dataflow)
  • Extensive hands-on experience with modern big data technologies and Data streaming (like Hadoop, Hive, Impala, Apache Spark, Kafka, or Flink)
  • Proficiency with workflow orchestration tools such as Airflow, Dagster, or Prefect
  • Proficiency in designing and implementing microservices architectures, RESTful APIs, and event-driven systems with 'Data as a Product' Principle
  • Solid understanding of data modeling concepts and database design for both analytical (OLAP) and transactional (OLTP) workloads
  • Deep understanding and hands-on experience with relational databases (e.g., PostgreSQL, Oracle), NoSQL databases (e.g., MongoDB, Cassandra), data warehousing, and big data technologies (e.g., Spark, Kafka)
Job Responsibility
Job Responsibility
  • Architect & Design: Design, architect, and oversee the development of robust, scalable, and reliable data infrastructure, including data lakes, data warehouses, and real-time streaming platforms on the cloud
  • Build & Code: Act as a senior individual contributor and hands-on technical leader. Write clean, maintainable, and high-performance code for data ingestion, transformation, and serving layers (e.g., using Python, Scala, SQL, and Spark)
  • Lead & Mentor: Lead a team of data engineers, providing technical guidance, mentorship, and career development support. Foster a collaborative and inclusive team environment
  • Champion Culture: Define, document, and champion data engineering best practices across the organization, including CI/CD, data quality, testing frameworks, observability, and code review standards
  • Drive Strategy: Partner with leadership, product managers, data scientists, and analysts to understand data needs and develop a long-term data strategy and roadmap
  • Innovate & Evaluate: Stay at the forefront of data engineering technologies. Evaluate, prototype, and recommend new tools and frameworks to continuously improve our data platform
  • Ensure Governance: Implement and enforce robust data governance, security, and privacy policies in partnership with our security and compliance teams
  • Fulltime
Read More
Arrow Right