CrawlJobs Logo

Senior Data Engineering Manager

yipitdata.com Logo

YipitData

Location Icon

Location:
India

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided

Job Description:

We’re scaling fast and need a hands-on Data Engineering Manager to join our dynamic Data Engineering team who can both lead people and shape data architecture. The ideal candidate possesses 3+ years of managing data engineers and 5+ years of experience working with PySpark, Python is a must. Data Bricks/ Snow Apache Iceberg/ Apache Flink/ and various orchestration tools, ETL pipelines, and data modeling. As our Data Engineering Manager, you will own the data-orchestration strategy end-to-end. You’ll lead and mentor a team of engineers while researching, planning, and institutionalizing best practices that boost our pipeline performance, reliability, and cost-efficiency. This is a hands-on leadership role for someone who thrives on deep technical challenges, enjoys rolling up their sleeves to debug or design, and can chart a clear, forward-looking roadmap for various data engineering projects.

Job Responsibility:

  • Lead, mentor, and grow a team of data engineers working on large-scale distributed data systems
  • Architect and oversee the development of end-to-end data solutions using AWS Data Services and Databricks
  • Hire, onboard, and develop a high-performing team—1-on-1s, growth plans, and performance reviews
  • Collaborate with cross-functional teams including data science, analytics, product, and business stakeholders to understand requirements and deliver impactful data products
  • Drive best practices in data engineering, coding standards, version control, CI/CD, and monitoring
  • Ensure high data quality, governance, and compliance with internal and external policies
  • Optimize performance and cost efficiency of data infrastructure in the cloud
  • Architect and evolve our data platform (batch & streaming) for scale, cost, and reliability
  • Own the end-to-end vision and strategic roadmap for various projects
  • Create documentation, architecture diagrams, and other training materials
  • Translate product and analytics needs into a clear data engineering roadmap and OKRs
  • Stay current with industry trends, emerging technologies, and apply them to improve system architecture and team capabilities

Requirements:

  • Bachelor’s or Master’s degree in Computer Science, STEM, or a related technical discipline
  • 8+ years in data engineering (or adjacent), including 2-3+ years formally managing 1-3 engineers
  • Proven hands-on experience with: Big Data ecosystems (Spark, Hive, Hadoop)
  • Databricks (including Delta Lake, MLFlow, Unity Catalog)
  • Robust programming experience in Python and PySpark
  • Deep understanding of data modeling, ETL/ELT processes using Streaming, and performance tuning
  • Experience managing Agile teams and delivering complex projects on time
  • Excellent problem-solving, leadership, and communication skills
  • Experience designing and implementing Agentic Models with Data Pipelines (Data Cleaning and creative Feature Engineering)
  • Practical LLM/RAG experience for search quality such as query understanding, semantic retrieval, reranker design
  • You are a self-starter who enjoys working with both internal and external stakeholders

Nice to have:

  • Familiarity with ML/AI workflows and collaboration with data science teams
  • Experience with Airflow, Docker, or equivalent

Additional Information:

Job Posted:
February 18, 2026

Employment Type:
Fulltime
Work Type:
Remote work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Senior Data Engineering Manager

Senior Data Engineer

Atlassian is looking for a Senior Data Engineer to join our Go-To Market Data En...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • A BS in Computer Science or equivalent experience
  • At least 5+ years professional experience as a Sr. Software Engineer or Sr. Data Engineer
  • Strong programming skills (Python, Java or Scala preferred)
  • Experience writing SQL, structuring data, and data storage practices
  • Experience with data modeling
  • Knowledge of data warehousing concepts
  • Experience building data pipelines, platforms, micro services, and REST APIs
  • Experience with Spark, Hive, Airflow and other streaming technologies to process incredible volumes of streaming data
  • Experience in modern software development practices (Agile, TDD, CICD)
  • Strong focus on data quality and experience with internal/external tools/frameworks to automatically detect data issues, anomalies
Job Responsibility
Job Responsibility
  • Help our stakeholder teams ingest data faster into our data lake
  • Make our data pipelines more efficient
  • Build micro-services, architect, design, and enable self-serve capabilities at scale
  • Work on an AWS-based data lake backed by open source projects such as Spark and Airflow
  • Identify ways to make our platform better and improve user experience
  • Apply strong technical experience building highly reliable services on managing and orchestrating a multi-petabyte scale data lake
What we offer
What we offer
  • Health coverage
  • Paid volunteer days
  • Wellness resources
  • Fulltime
Read More
Arrow Right

Senior Data Engineer

Atlassian is looking for a Senior Data Engineer to join our Go-To Market Data En...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • A BS in Computer Science or equivalent experience
  • At least 5+ years professional experience as a Sr. Software Engineer or Sr. Data Engineer
  • Strong programming skills (Python, Java or Scala preferred)
  • Experience writing SQL, structuring data, and data storage practices
  • Experience with data modeling
  • Knowledge of data warehousing concepts
  • Experience building data pipelines, platforms, micro services, and REST APIs
  • Experience with Spark, Hive, Airflow and other streaming technologies to process incredible volumes of streaming data
  • Experience in modern software development practices (Agile, TDD, CICD)
  • Strong focus on data quality and experience with internal/external tools/frameworks to automatically detect data issues, anomalies
Job Responsibility
Job Responsibility
  • Help our stakeholder teams ingest data faster into our data lake
  • Make our data pipelines more efficient
  • Build micro-services
  • Architect, design, and enable self-serve capabilities at scale
  • Apply your strong technical experience building highly reliable services
  • Manage and orchestrate a multi-petabyte scale data lake
  • Transform vague requirements into solid solutions
  • Solve challenging problems creatively
What we offer
What we offer
  • Health coverage
  • Paid volunteer days
  • Wellness resources
  • Fulltime
Read More
Arrow Right

Senior Data Engineering Manager

Data is a big deal at Atlassian. We ingest billions of events each month into ou...
Location
Location
United States , San Francisco
Salary
Salary:
168700.00 - 271100.00 USD / Year
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • stellar people management skills and experience in leading an agile software team
  • thrive when developing phenomenal people, not just great products
  • worked closely Data Science, analytics and platform teams
  • expertise in building and maintaining high-quality components and services
  • able to drive technical excellence, pushing for innovation and quality
  • at least 10 years experience in a software development role as an individual contributor
  • 4+ years of people management experience
  • deep understanding of data challenges at scale challenges and the eco-system
  • experience with solution building and architecting with public cloud offerings such as Amazon Web Services, DynamoDB, ElasticSearch, S3, Databricks, Spark/Spark-Streaming, GraphDatabases
  • experience with Enterprise Data architectural standard methodologies
Job Responsibility
Job Responsibility
  • build and lead a team of data engineers through hiring, coaching, mentoring, and hands-on career development
  • provide deep technical guidance in a number of aspects of data engineering in a scalable ecosystem
  • champion cultural and process improvements through engineering excellence, quality and efficiency
  • work with close counterparts in other departments as part of a multi-functional team, and build this culture in your team
What we offer
What we offer
  • health coverage
  • paid volunteer days
  • wellness resources
  • Fulltime
Read More
Arrow Right

Senior Microsoft Stack Data Engineer

Hands-On Technical SENIOR Microsoft Stack Data Engineer / On Prem to Cloud Senio...
Location
Location
United States , West Des Moines
Salary
Salary:
155000.00 USD / Year
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of DATA WAREHOUSE EXPERIENCE / Data Lake experience
  • Advanced SQL Server
  • Strong SQL experience, working with structured and unstructured data
  • Strong in SSIS ETL
  • Proficiency in SQL and SQL Queries
  • Experience with SQL Server and SQL Server
  • Knowledge of Data Warehousing and Data Warehousing
  • Data Warehouse experience: Star Schema and Fact & Dimension data warehouse structure
  • Experience with Azure Data Lake and Data lakes
  • Proficiency in ETL / SSIS and SSAS
Job Responsibility
Job Responsibility
  • Modernize, Build out a Data Warehouse, and Lead & Build out a Data Lake in the CLOUD
  • REBUILD an OnPrem data warehouse working with disparate data to structure the data for consumable reporting
  • ALL ASPECTS OF Data Engineering
  • Technical Leader of the team
What we offer
What we offer
  • Bonus
  • 2 1/2 day weekends
  • Medical, vision, dental, and life and disability insurance
  • 401(k) plan
  • Fulltime
Read More
Arrow Right

FLEX Senior Manager, Onsite Digital Data Management

This is a temporary position. We are looking for a Sr Manager, Owned Channels Di...
Location
Location
United States , Bethesda, MD
Salary
Salary:
52.06 - 82.45 USD / Hour
https://www.marriott.com Logo
Marriott Bonvoy
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4-year degree from an accredited university in Data Analytics, Computer Science, Engineering, Information Systems, or a similar quantitative discipline, with 4+ years of experience demonstrating progressive career growth and a history of exceptional performance in data management, analytics, and marketing technology OR 2+ years of experience in data management, analytics and marketing technology with a Master's degree
  • Proven expertise in designing and engineering complex data flows for customer data & audience development with proficiency in SQL
  • In-depth knowledge and hands-on experience with full funnel tag management and digital analytics platforms (e.g., Adobe Analytics, Google Analytics etc.)
  • Strong understanding of marketing and ad technologies, customer data platforms (CDP), and cloud data warehouses (e.g., Snowflake, AWS)
  • Proven ability to deliver impactful results while managing complex, high-stakes initiatives with ambiguity and cross-functional collaboration
  • Strong leadership skills with a track record of working closely with marketing, digital, and technology teams to drive data strategy and innovation
Job Responsibility
Job Responsibility
  • Provide leadership, consulting, and information expertise to achieve the goals for marketing & media
  • Work with MarTech and IT teams to develop and deploy an effective digital & customer data infrastructure
  • Contribute to data strategy, data management, and enhanced data products for use by the Marketing discipline
  • Contribute to the broader data strategy and effectively prioritize implementation tactics for onsite data to ensure they are in line with overall organizational goals and long-term business strategies
  • Work closely with the marketing and analytics teams to identify key performance indicators (KPIs) and data requirements necessary for optimizing segmentation strategies and enhancing ROI
  • Manage vendors/technical staff to build the infrastructure to support data & reporting solutions in AWS, Snowflake, Adobe etc
  • Oversee development of systems and technologies that reduce manual processes and data quality issues relating to data capture
  • Partner with Analytics and IT to build Data Management, Analytics and Reporting roadmaps for future years
  • Partner with business leadership to secure funding for critical Data Management & IT programs
What we offer
What we offer
  • Coverage for medical, dental, vision, health care flexible spending account, dependent care flexible spending account, life insurance, disability insurance, accident insurance, adoption expense reimbursements, paid parental leave, 401(k) plan, stock purchase plan, discounts at Marriott properties, commuter benefits, employee assistance plan, and childcare discounts
  • Fulltime
Read More
Arrow Right

Senior Product Manager – Security Data Integrations and Management

We are seeking a skilled, experienced Sales Engineering Director to manage a div...
Location
Location
United States
Salary
Salary:
182000.00 - 219000.00 USD / Year
https://corelight.com/ Logo
Corelight
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5-7 years of Product Management experience in security, observability, or data pipeline technologies
  • Strong knowledge of SIEMs, log analytics, and security data platforms (such as Splunk, Elastic, CrowdStrike)
  • Experience w/ Zeek, Suricata, and/or YARA is a plus
  • Experience with log routing and transformation, including tools such as vector.dev, Cribl, logstash, and Fluentd
  • Knowledge of network security, threat detection, compliance logging, and SIEM ingestion requirements
  • Ability to collaborate with engineering teams, security professionals, and customers to drive impactful solutions
Job Responsibility
Job Responsibility
  • Drive product planning, requirements gathering, and prioritization for the overall sensor management solution including device management and SIEM integrations while balancing customer needs with technical feasibility
  • Define and track key metrics to measure the success of SIEM integrations and drive data-informed decisions
  • Collaborate with engineering to ensure robust data manipulation capabilities for log enrichment, filtering, and transformation
  • Work with customers to understand pain points in security device management, log management, compliance, and security data pipelines
  • Ensure seamless integration with structured formats like JSON, and mappings to common schemas such as CIM, ECS, and OCSF
  • Partner with security and DevOps teams to align with SIEM ingestion best practices, detection rules, and compliance frameworks
What we offer
What we offer
  • Equity and additional benefits will also be awarded
  • Fulltime
Read More
Arrow Right

Senior Data Engineer

We are looking for a Senior Data Engineer (SDE 3) to build scalable, high-perfor...
Location
Location
India , Mumbai
Salary
Salary:
Not provided
https://cogoport.com/ Logo
Cogoport
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of experience in data engineering, working with large-scale distributed systems
  • Strong proficiency in Python, Java, or Scala for data processing
  • Expertise in SQL and NoSQL databases (PostgreSQL, Cassandra, Snowflake, Apache Hive, Redshift)
  • Experience with big data processing frameworks (Apache Spark, Flink, Hadoop)
  • Hands-on experience with real-time data streaming (Kafka, Kinesis, Pulsar) for logistics use cases
  • Deep knowledge of AWS/GCP/Azure cloud data services like S3, Glue, EMR, Databricks, or equivalent
  • Familiarity with Airflow, Prefect, or Dagster for workflow orchestration
  • Strong understanding of logistics and supply chain data structures, including freight pricing models, carrier APIs, and shipment tracking systems
Job Responsibility
Job Responsibility
  • Design and develop real-time and batch ETL/ELT pipelines for structured and unstructured logistics data (freight rates, shipping schedules, tracking events, etc.)
  • Optimize data ingestion, transformation, and storage for high availability and cost efficiency
  • Ensure seamless integration of data from global trade platforms, carrier APIs, and operational databases
  • Architect scalable, cloud-native data platforms using AWS (S3, Glue, EMR, Redshift), GCP (BigQuery, Dataflow), or Azure
  • Build and manage data lakes, warehouses, and real-time processing frameworks to support analytics, machine learning, and reporting needs
  • Optimize distributed databases (Snowflake, Redshift, BigQuery, Apache Hive) for logistics analytics
  • Develop streaming data solutions using Apache Kafka, Pulsar, or Kinesis to power real-time shipment tracking, anomaly detection, and dynamic pricing
  • Enable AI-driven freight rate predictions, demand forecasting, and shipment delay analytics
  • Improve customer experience by providing real-time visibility into supply chain disruptions and delivery timeline
  • Ensure high availability, fault tolerance, and data security compliance (GDPR, CCPA) across the platform
What we offer
What we offer
  • Work with some of the brightest minds in the industry
  • Entrepreneurial culture fostering innovation, impact, and career growth
  • Opportunity to work on real-world logistics challenges
  • Collaborate with cross-functional teams across data science, engineering, and product
  • Be part of a fast-growing company scaling next-gen logistics platforms using advanced data engineering and AI
  • Fulltime
Read More
Arrow Right

Senior Manager, Product Management Data and Analytics AdTech

We are seeking a Senior Manager, Product Management (Data & Analytics) to lead o...
Location
Location
United States , Bellevue
Salary
Salary:
151700.00 - 273700.00 USD / Year
https://www.t-mobile.com Logo
T-Mobile
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree Computer Science, Engineering, or Information Technology
  • Master's/Advanced Degree Business Administration or Technology Management
  • 4-7 years experience in product management experience, including at least 3+ years in a leadership or mentorship capacity
  • Strong SQL skills
  • demonstrated experience integrating machine learning or predictive modeling into product strategies
  • Proven success leading small product teams, driving alignment, and encouraging teamwork across cross-functional groups
  • Ability to frame business problems in analytical terms and translate insights into actionable product features
  • Familiarity with ad tech identity solutions (MAIDs, cookies, IDFVs), privacy regulations (GDPR, CCPA), and data governance
  • Capable of articulating sophisticated technical concepts to diverse audiences, including executives, engineers, and clients
  • Track record of delivering products
Job Responsibility
Job Responsibility
  • Platform Data Product Strategy & Vision Define and execute the platform data and analytics roadmap
  • Hands-on Technical Execution Use SQL and other data analysis tools to investigate product performance
  • Team Leadership & Development Lead, mentor, and develop a small team of product managers or analysts
  • Machine Learning Integration Collaborate with data science teams to embed ML models into product offering
  • Cross-Functional Collaboration Work closely with engineering, data science, sales, and marketing to align on product goals
  • Data-Driven Decision-Making Leverage product metrics, user feedback, and analytics to inform decisions
  • Client & Stakeholder Engagement Gather and prioritize feedback from internal and external stakeholders
  • Market & Competitive Analysis Stay informed on emerging industry trends, technologies, and competitor strategies
  • Ownership of Full Product Lifecycle Oversee products from ideation through launch and ongoing optimization
  • Compliance & Privacy Collaborate with legal and compliance teams to design privacy-first product features
What we offer
What we offer
  • medical, dental and vision insurance
  • flexible spending account
  • 401(k)
  • employee stock grants
  • employee stock purchase plan
  • paid time off
  • up to 12 paid holidays
  • paid parental and family leave
  • family building benefits
  • back-up care
  • Fulltime
Read More
Arrow Right