CrawlJobs Logo

Senior Software Engineer - Data Products

India, Bengaluru · Job Posted June 09, 2026
Apply Position
Job Link Share

Job Description

We are seeking a highly skilled Senior Software Engineer with deep expertise in big data technologies, including Apache Spark and Apache Airflow. This hybrid position bridges software engineering and data engineering, requiring the ability to design, build, and maintain scalable systems for both application development and large-scale data processing. In this role, you will collaborate with cross-functional teams to architect and manage robust, production-grade data products that power critical analytics and measurement capabilities. You will work with technologies such as Apache Spark, Apache Airflow, Trino, Druid, Spring Boot, StarRocks, and Looker to deliver reliable, high-performance solutions. The ideal candidate is a proactive, self-motivated professional with a strong track record in building high-scale data services and a dedication to delivering exceptional results.

Job Responsibility

  • Design and build APIs and backend services using Spring Boot to support data products and analytics workflows.
  • Write clean, maintainable, and efficient code, ensuring adherence to best practices through code reviews.
  • Design, develop, and maintain data pipelines and ETL workflows using Apache Spark and Apache Airflow.
  • Optimize data storage, retrieval, and processing systems to ensure reliability, scalability, and performance.
  • Develop and fine-tune complex queries and analytics solutions using Druid, Trino, and StarRocks for large-scale datasets.
  • Monitor, troubleshoot, and improve data systems to minimize downtime and maximize efficiency.
  • Partner with data scientists, software engineers, and other teams to deliver integrated, high-quality solutions.
  • Provide technical guidance and mentorship to junior engineers, promoting best practices in software and data engineering.

Requirements

  • Bachelor's degree in computer science, Engineering, or a related field (or equivalent experience).
  • 8+ years of experience in software and/or data engineering with expertise in big data technologies such as Apache Spark, Apache Airflow.
  • Expertise with at least one of the following Apache Druid, StarRocks, and Trino.
  • Strong understanding of SOLID principles and distributed systems architecture.
  • Proven experience in distributed data processing, data warehousing, and real-time data pipelines.
  • Advanced SQL skills, with expertise in query optimization for large datasets.
  • Exceptional problem-solving abilities and the capacity to work independently or collaboratively.
  • Excellent verbal and written communication skills.
  • Experience with cloud platforms such as AWS, GCP, or Azure, and containerization tools like Docker and Kubernetes. (preferred)
  • Familiarity with additional big data technologies, including Hadoop and Kafka.
  • Experience in AdTech, in advertising data platforms and campaign measurement. (preferred)
  • Strong programming skills in Python, Java, or Scala. (preferred)
  • Knowledge of CI/CD pipelines, DevOps practices, and infrastructure-as-code tools (e.g., Terraform). (preferred)
  • Expertise in data modeling, schema design, and data visualization tools.
  • Experience with building Agentic AI systems to automate decision-making and enhance analytics workflows. (preferred)

Nice to have

  • Experience with cloud platforms such as AWS, GCP, or Azure, and containerization tools like Docker and Kubernetes.
  • Familiarity with additional big data technologies, including Hadoop and Kafka.
  • Experience in AdTech, in advertising data platforms and campaign measurement.
  • Strong programming skills in Python, Java, or Scala.
  • Knowledge of CI/CD pipelines, DevOps practices, and infrastructure-as-code tools (e.g., Terraform).
  • Experience with building Agentic AI systems to automate decision-making and enhance analytics workflows.

What we offer

  • Global access to mental health and financial wellness support and resources.
  • Local benefits including statutory and voluntary benefits which may include healthcare (medical, dental, and vision), life, accident, disability, commuter, and retirement options (401(k)/pension).
  • Time off in accordance with local leave policies.

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Senior Software Engineer - Data Products

8 matching positions

Software engineer 2 / Senior Software engineer - Azure Data

Microsoft's Azure Data engineering team is leading the transformation of analyti...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 3+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • OR equivalent experience
  • Experience with the Azure stack including Storage, Compute, Networking, Fabric, Purview, Synapse, AKS, DevOps, Data Factory, or Power BI
  • Experience with big data technologies such as Spark, Kafka, Hadoop, or HBase
  • Experience building data lake or data engineering products, tools, or pipelines
  • Familiarity with container-based architectures (Docker, Kubernetes)
  • Ability to debug complex distributed systems on Linux and/or Windows platforms
Job Responsibility
Job Responsibility
  • Write extensible, maintainable code in C#, Java, Scala, or Python for Fabric Materialized Lake View services and HDInsight components
  • Use AI tools and coding best practices across the development lifecycle
  • Design data refresh, scheduling, and query optimisation features with minimal supervision
  • Review code from teammates for correctness, test coverage, security risks, and adherence to team standards
  • Coach junior engineers through code reviews
  • Debug complex issues in distributed systems running on Azure, Linux, and Windows
  • Run live site operations on a rotational, on-call basis
  • Integrate logging and instrumentation to gather telemetry on system health, performance, reliability, and security
  • Work with product managers, technical leads, and partners across geographies to define customer requirements for Materialized Lake View features
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Data Products

As a Senior Software Engineer, you will play a pivotal role in the development o...
Location
Location
United States , Los Angeles
Salary
Salary:
143000.00 - 180000.00 USD / Year
foxcorporation.com Logo
Fox Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience working in Software Engineering, Data Science, ML Engineering
  • Strong background in live media streaming and handling VOD content
  • Expertise in working with live media streaming
  • Experience working with Vector Database
  • Strong understanding of generative AI technologies and their underlying mechanisms
  • Good grasp of distributed system design
  • Experience with TensorFlow, PyTorch etc.
  • REST or GraphQL API Design Experience
  • Proficient with building batch and streaming data pipelines on cloud platforms
Job Responsibility
Job Responsibility
  • Design and implement novel and scalable AI solutions for real business problems
  • Design and implement workflows to generate and manage assets for live streaming and VOD
  • Build workflow orchestrations that can be readily extended to perform new analyses
  • Prototype new approaches and productionize solutions at scale for hundreds of millions of active users
  • Maintain high-level craftsmanship while delivering meaningful results
  • Mentor junior engineers on the team
  • Collaborate with peers, engineering leadership, and product management
What we offer
What we offer
  • Annual discretionary bonus
  • Medical/dental/vision insurance
  • 401(k) plan
  • Paid time off
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Data Products

As a Senior Software Engineer, you will play a pivotal role in the development o...
Location
Location
United States , Los Angeles
Salary
Salary:
143000.00 - 180000.00 USD / Year
foxnews.com Logo
Fox News Media
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience working in Software Engineering, Data Science, ML Engineering
  • Strong background in live media streaming and handling VOD content
  • Expertise in working with live media streaming
  • Experience working with Vector Database
  • Strong understanding of generative AI technologies and their underlying mechanisms
  • Good grasp of distributed system design
  • Experience with TensorFlow, PyTorch etc.
  • REST or GraphQL API Design Experience
  • Proficient with building batch and streaming data pipelines on cloud platforms
Job Responsibility
Job Responsibility
  • Design and implement novel and scalable AI solutions for real business problems
  • Design and implement workflows to generate and manage assets for live streaming and VOD
  • Build workflow orchestrations that can be readily extended to perform new analyses
  • Prototype new approaches and productionize solutions at scale for hundreds of millions of active users
  • Maintain high-level craftsmanship while delivering meaningful results
  • Mentor junior engineers on the team
  • Collaborate with peers, engineering leadership, and product management
What we offer
What we offer
  • Annual discretionary bonus
  • Medical/dental/vision insurance
  • 401(k) plan
  • Paid time off
  • Fulltime
Read More
Arrow Right

Full-Stack Senior Software Engineer, GenAI Data Products and Platform (VP)

This is your chance to build the foundational systems for 'Citi Assist', a Gener...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong experience building and deploying production applications across the full stack
  • Proficiency in multiple languages, including Python and TypeScript/JavaScript (experience with Go or Java is a plus)
  • Deep experience working with data—whether that's building data pipelines, designing analytics systems, or creating data-driven products
  • Experience building user-facing features with modern frontend frameworks like React, Vue, or Angular
  • Strong SQL skills and experience with relational databases like Postgres
  • Experience building backend services and APIs that handle data at scale
  • Comfort with containerised environments and cloud infrastructure (we use OpenShift/Kubernetes)
  • Strong understanding of CI/CD pipelines, testing frameworks, and automation
  • Experience with data visualisation tools and techniques
Job Responsibility
Job Responsibility
  • Build the tools that make Assist great
  • Work with data across the full stack
  • Own your features end to end
  • Build with safety and quality in mind
  • Set the technical direction
  • Be a great teammate
What we offer
What we offer
  • 27 days annual leave (plus bank holidays)
  • A discretional annual performance related bonus
  • Private Medical Care & Life Insurance
  • Employee Assistance Program
  • Pension Plan
  • Paid Parental Leave
  • Special discounts for employees, family, and friends
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Agentic Data Products

Scale GP is Scale’s enterprise Generative AI platform, providing APIs and infras...
Location
Location
United States , San Francisco
Salary
Salary:
216200.00 - 270250.00 USD / Year
scale.com Logo
Scale
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of full-time software engineering experience
  • 0-1 product build experience
  • Familiarity with LLMs, embeddings, vector databases, or modern AI data products/tools
  • Experience with distributed systems and cloud-based architectures
  • Prior experience mentoring or leading team
Job Responsibility
Job Responsibility
  • Own major full-stack product areas, driving features from concept and design through production deployment
  • Build intuitive, high-performance frontend experiences using React + TypeScript
  • Develop reliable backend services in Python, working with distributed systems, data pipelines, and AI/ML infrastructure
  • Integrate LLMs, vector databases, and agentic frameworks to power intelligent workflows and decision-making systems
  • Ship quickly through tight experimentation loops while maintaining high quality and reliability
  • Help define the technical direction and architecture of a brand-new team and product surface
  • Adapt across the stack and learn new tools as needed to solve real problems end-to-end
What we offer
What we offer
  • Comprehensive health, dental and vision coverage
  • retirement benefits
  • a learning and development stipend
  • generous PTO
  • Fulltime
Read More
Arrow Right

Full-stack Senior Software Engineer, GenAI Data Products and Platform

This is your chance to build the foundational systems for 'Citi Assist', a Gener...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong experience building and deploying production applications across the full stack
  • Proficiency in multiple languages, including Python and TypeScript/JavaScript (experience with Go or Java is a plus)
  • Deep experience working with data—whether that's building data pipelines, designing analytics systems, or creating data-driven products
  • Experience building user-facing features with modern frontend frameworks like React, Vue, or Angular
  • Strong SQL skills and experience with relational databases like Postgres
  • Experience building backend services and APIs that handle data at scale
  • Comfort with containerised environments and cloud infrastructure (we use OpenShift/Kubernetes)
  • Strong understanding of CI/CD pipelines, testing frameworks, and automation
  • Experience with data visualisation tools and techniques
  • A data-driven mindset
Job Responsibility
Job Responsibility
  • Build the tools that make Assist great
  • Work with data across the full stack
  • Own your features end to end
  • Build with safety and quality in mind
  • Set the technical direction
  • Be a great teammate
What we offer
What we offer
  • 27 days annual leave (plus bank holidays)
  • A discretional annual performance related bonus
  • Private Medical Care & Life Insurance
  • Employee Assistance Program
  • Pension Plan
  • Paid Parental Leave
  • Special discounts for employees, family, and friends
  • Fulltime
Read More
Arrow Right

Software Developer - Senior / Senior Data Engineer

Imagine being at the forefront of protecting financial integrity and enabling cr...
Location
Location
United States , Southlake, TX or Austin, TX
Salary
Salary:
55.92 - 62.13 USD / Hour
aquent.com Logo
Aquent
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field
  • 8+ years of hands-on experience in data management, including gathering data from diverse sources, consolidating it into centralized locations, and transforming it with business logic for consumption in visualization and data analysis
  • Strong expertise in BigQuery, Cloud Storage, Dataflow, Pub/Sub, Cloud Composer, and related cloud platform services
  • Proficiency in Python and SQL for data processing and automation
  • Extensive experience with ETL processes and data pipeline design
  • Excellent problem-solving skills and meticulous attention to detail
  • Strong communication and collaboration skills, with the ability to actively listen, dialogue freely, and verbalize ideas effectively
  • Ability to thrive in an Agile work environment, delivering incremental value to customers by effectively managing and prioritizing tasks
Job Responsibility
Job Responsibility
  • Design, build, and maintain robust and scalable data pipelines using cloud platform tools such as BigQuery, Cloud Storage, Dataflow (Apache Beam), Cloud Composer (Airflow), and Pub/Sub
  • Develop high-performance, production-grade Python and SQL code, optimizing queries for efficient data extraction, transformation, and loading (ETL) processes
  • Implement complex data models in BigQuery, leveraging partitioning, clustering, and materialized views to achieve optimal performance
  • Collaborate closely with cross-functional teams, including business customers and Subject Matter Experts, to gather data requirements and deliver impactful solutions
  • Implement and uphold best practices for data quality, data governance, and data security
  • Proactively monitor and troubleshoot data pipeline issues, ensuring high availability and performance of critical data flows
  • Contribute to strategic data architecture decisions, providing recommendations for continuous improvement of data pipelines
  • Stay current with emerging trends and technologies in cloud-based data engineering and cyber security to drive innovation
  • Lead investigation and resolution efforts for identified data issues, taking ownership to resolve them in a timely manner
  • Document processes and procedures thoroughly for producing accurate metrics and ensuring operational clarity
What we offer
What we offer
  • subsidized health, vision, and dental plans
  • paid sick leave
  • retirement plans with a match
Read More
Arrow Right

Software Engineer 2 / Senior Software Engineer

We are looking for an experienced Software Engineers for our Bangalore location ...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
komprise.com Logo
Komprise, Inc.
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Solid grasp of computer science fundamentals and especially data structures, algorithms, multi-threading
  • Ability to solve difficult problems with a simple elegant solution
  • Should have solid object-oriented programming background with impeccable design skills
  • Experience in developing management applications and performance management applications is ideal
  • Experience with object-based file systems and REST interfaces is a plus (e.g. Amazon S3, Azure, Google Cloud Service)
  • Should have a BE or higher in CS, EE, Math or related engineering or science field
  • At least 5+ years of experience in software deployment
  • Tech Stack: Java, Maven Virtualisation, SaaS, Github, Jira, Slack, Cloud Solutions and Hypervisors
Job Responsibility
Job Responsibility
  • Responsible for designing and developing features that powers Komprise data management platform to manage billions of files and petabytes of data
  • Responsible for designing of major components and systems of our product architecture, ensuring that Komprise data management platform is highly available and scalable
  • Responsible for writing performance code, evaluate feasibility, develop for quality and optimize for maintainability
  • Work in agile, customer focused and fast paced team with direct interaction with the customers
  • Responsible for analysing customer escalated issues and provide resolutions in a timely manner
  • Should be able to design and implement highly performant, scalable distributed systems
Read More
Arrow Right