Data Engineer Sr (DBT) Job at NTT DATA (Bangalore)

Sr Data Engineer

As a Senior Data Engineer at Amgen, you will be responsible for managing and opt...

Location

India , Hyderabad

Salary:

Not provided

Amgen

Expiration Date

Until further notice

Requirements

Bachelor’s or Master's degree with 8 - 13 years of experience in Computer Science, IT or related field
Hands-on experience in data engineering technologies such as Databricks, PySpark, SparkSQL Apache Spark, AWS, Python, SQL, and Scaled Agile methodologies
Proficiency in workflow orchestration, performance tuning on big data processing
Strong understanding of AWS services
Experience with Data Fabric, Data Mesh, or similar enterprise-wide data architectures
Ability to quickly learn, adapt and apply new technologies
Strong problem-solving and analytical skills
Excellent communication and teamwork skills
Experience with Scaled Agile Framework (SAFe), Agile delivery practices, and DevOps practices
Excellent analytical and troubleshooting skills

Job Responsibility

Design, develop, and maintain scalable ETL/ELT pipelines to support structured, semi-structured, and unstructured data processing across the Enterprise Data Engineering for Biotech or Pharma functional knowledge of R&D
Implement real-time and batch data processing solutions, integrating data from multiple sources into a unified, governed data fabric architecture
Optimize big data processing frameworks using Apache Spark, Hadoop, or similar distributed computing technologies to ensure high availability and cost efficiency
Work with metadata management and data lineage tracking tools to enable enterprise-wide data discovery and governance
Ensure data security, compliance, and role-based access control (RBAC) across data environments
Optimize query performance, indexing strategies, partitioning, and caching for large-scale data sets
Develop CI/CD pipelines for automated data pipeline deployments, version control, and monitoring
Implement data virtualization techniques to provide seamless access to data across multiple storage systems
Collaborate with cross-functional teams, including data architects, business analysts, and DevOps teams, to align data engineering strategies with enterprise goals
Stay up to date with emerging data technologies and best practices, ensuring continuous improvement of Enterprise Data Fabric architectures

Fulltime

Data Engineer Sr (Databricks)

Location

India , Bangalore

Salary:

Not provided

NTT DATA

Expiration Date

Until further notice

Requirements

8+ years of experience in databricks with understanding complex legacy data models and getting that data into the cloud: Extracting DB2/AS400: Experience with Change Data Capture (CDC) or scheduled batch extractions from DB2 into cloud storage. Involves working through JDBC connections, mapping table dependencies, and re-platforming legacy SQL to distributed computing standards
Handling Guidewire Data: Integrating with Guidewire Cloud Data Access (CDA) or InsuranceSuite to replicate complex P&C (Property & Casualty) insurance schemas. Senior engineers parse these highly normalized operational databases and transform them into analytical-friendly schemas in the cloud
2. Architecture & Pipeline Development: The core of the experience involves transitioning these legacy, row-based stores into a scalable Medallion Architecture (Bronze, Silver, Gold layers)
Delta Lake Optimization: Using Databricks and Apache Spark to build ETL/ELT data pipelines with ACID transactions. Senior engineers handle schema evolution, upserts, and slowly changing dimensions (SCD Type 2)
Business Logic Refactoring: Translating rigid legacy procedural code (e.g., RPG/COBOL background logic, stored procedures) into scalable distributed patterns (PySpark, Spark SQL, and Scala)
3. Data Governance & Observability: A senior engineer is expected to govern vast amounts of incoming and generated data across the enterprise
Unity Catalog: Implementing strict data governance, lineage tracing, and table-level security
Data Quality: Automating data validation frameworks to ensure a seamless transition from legacy to modern systems without data loss or corruption
4. Integration with the Databricks Platform Ecosystem: Moving beyond basic storage to utilizing the full power of the Databricks Data Intelligence Platform
Serverless Compute: Managing Databricks serverless resources, ensuring optimal cluster sizing, and reducing compute costs

Job Responsibility

Develop Databricks notebooks, jobs, and workflows to replicate and enhance DB2/Guidewire-based pipelines and transformations
Implement Delta Lake tables and patterns (bronze/silver/gold, ACID, time travel, schema evolution) for migrated data
Integrate Databricks with AWS/S3 or Azure ADLS, ADF/Synapse, Key Vault, and Snowflake as required
Optimize Databricks clusters, jobs, and queries for performance and cost
Implement incremental loads, CDC patterns, and batch schedules for large datasets
Collaborate with Snowflake and dbt teams to ensure consistent data models and data contracts
Participate in data validation and reconciliation between DB2 400 / Guiderwire and Databricks outputs
Follow coding standards, version control, and CI/CD practices using Git/Azure DevOps
Provide defect fixes and support during SIT/UAT and post go-live stabilization

Sr. Data Engineer III

Piper Companies is seeking a Sr. Data Engineer III to join an enterprise technol...

Location

United States

Salary:

130000.00 - 150000.00 USD / Year

Piper Companies

Expiration Date

Until further notice

Requirements

5+ years of data engineering or related experience
Strong expertise in SQL, Python, dbt, and or C#
Experience with Azure Data Factory or similar orchestration tools
Hands-on experience with Microsoft Azure and Fabric
Experience with Databricks or Snowflake is acceptable
Strong understanding of data modeling and ETL concepts
Experience with Git and version control best practices
Ability to work independently and solve complex problems
Background working in small or mid-sized environments with leadership experience
Must be authorized to work in the United States

Job Responsibility

Design and maintain scalable data pipelines and ETL workflows
Develop and optimize advanced SQL queries for performance and reliability
Leverage Azure, Fabric, Spark, and Python to automate complex data processes
Build and support REST and SOAP APIs for system integrations
Monitor and troubleshoot data pipelines to ensure high availability
Identify and resolve bottlenecks in data workflows
Lead engineering efforts supporting AI and machine learning initiatives
Collaborate with stakeholders and cross-functional teams
Conduct code reviews and contribute to technical documentation
Provide Tier 3 support for reporting and integrated systems

What we offer

medical
dental
vision
401k
PTO
sick leave as required by law

Fulltime

Sr Data Engineer

The purpose of the Data Engineer position is to leverage their technical experti...

Location

India , Bengaluru

Salary:

Not provided

Randstad

Expiration Date

July 03, 2026

Requirements

Bachelor’s or Master’s degree in Computer Science, Data Science, Information Technology, or a related field with a focus on data engineering or data analytics
Strong proficiency in programming languages such as Python, SQL, Java, or Scala for data processing and analysis
Experience with data modeling, ETL processes, data warehousing, data integration, and data pipeline development
Proficiency in relational databases (e.g., SQL, PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra)
Working knowledge of cloud platforms such as Snowflake, AWS, Azure, or Google Cloud Platform for data storage and processing
Experience with data visualization tools (e.g., Power BI) to create meaningful insights from data
Understanding of data quality principles, data governance, and data validation processes
Ability to analyze complex data sets, identify trends, patterns, and insights to drive data-driven decision-making
Proficiency in troubleshooting data-related issues, identifying root causes, and implementing solutions
Familiarity with project management methodologies to contribute effectively to project planning and execution

Job Responsibility

Build pipelines using standard patterns for data pipelines and workflows utilizing Streamsets, Kestra, dbt, Git
Design and Implement data storage and processing solutions employing Snowflake
Utilize AWS services for cloud-based platform tooling infrastructure including but not limited to Lambda,ECS,MSK,RDS,EC2, Secrets Manager, ALB, Cloud Watch, Event Bridge
Utilize Terraform for AWS and Azure deployments
Leverage and integrate APIs for data access and manipulation
Write Python scripts for data common processing and automation tasks
Leverage Platform API’s and Web Applications to enforce Platform Security
Create and execute comprehensive test plans to ensure the pipelines functionality and performance
Develop unit tests, integration tests, and end-to-end tests for data pipelines and workflows
Ensure data accuracy and consistency through rigorous testing processes

What we offer

Commitment to your ongoing development, including on the job opportunities and formal programs
Inclusive parental leave entitlements for both parents
Values led culture
Flexible work options
Generous annual leave, sick leave and casual leave
Cultural and religious leave with flexible public holiday opportunities
A competitive remuneration package featuring performance based incentives with uncapped Employer Provident Fund

Fulltime

!

Sr. Data Engineer - Snowflake

Data Ideology is seeking a Sr. Snowflake Data Engineer to join our growing team ...

Location

Salary:

Not provided

Data Ideology

Expiration Date

Until further notice

Requirements

7+ years of experience in data engineering, data warehousing, or data architecture
3+ years of hands-on Snowflake experience (performance tuning, data sharing, Snowpark, Snowpipe, etc.)
Strong SQL and Python skills, with production experience using dbt
Familiarity with cloud platforms (AWS, Azure, or GCP) and modern data tooling (Airflow, Fivetran, Power BI, Looker, Informatica, etc.)
Prior experience in a consulting or client-facing delivery role
Excellent communication skills, with the ability to collaborate across technical and business stakeholders
SnowPro Core Certification required (or willingness to obtain upon hire)
advanced Snowflake certifications preferred

Job Responsibility

Design and build scalable, secure, and cost-effective data solutions in Snowflake
Develop and optimize data pipelines using tools such as dbt, Python, CloverDX, and cloud-native services
Participate in discovery sessions with clients to gather requirements and translate them into solution designs and project plans
Collaborate with engagement managers and account teams to help scope work and provide technical input for Statements of Work (SOWs)
Serve as a Snowflake subject matter expert, guiding best practices in performance tuning, cost optimization, access control, and workload management
Lead modernization and migration initiatives to move clients from legacy systems into Snowflake
Integrate Snowflake with BI tools, governance platforms, and AI/ML frameworks
Contribute to internal accelerators, frameworks, and proofs of concept
Mentor junior engineers and support knowledge sharing across the team

What we offer

Flexible Time Off Policy
Eligibility for Health Benefits
Retirement Plan with Company Match
Training and Certification Reimbursement
Utilization Based Incentive Program
Commission Incentive Program
Referral Bonuses
Work from Home

Fulltime

Sr. Qlik Data Engineer

Location

United States , Raleigh

Salary:

120000.00 USD / Year

Realign

Expiration Date

Until further notice

Requirements

Strong hands-on experience in building robust metadata-driven, automated data pipeline solutions leveraging modern cloud-based data technologies, tools for large data platforms
Strong experience leveraging data security, governance methodologies meeting data compliance requirements
Strong hands-on experience building medallion architecture automated ELT data pipelines, snow pipe frameworks leveraging Qlik Replicate, DBT Cloud, snowflake with CICD
Strong hands-on experience building data pipelines, data integrity solutions across multiple data sources and targets like SQL Server, Oracle, Mainframe-DB2, files, Snowflake
Experience working with various structured & semi-structured data files - CSV, fixed width, JSON, XML, Excel, and mainframe VSAM
Experience using S3, Lambda, SQS, SNS, Glue, RDS AWS services
Excellent proficiency in Python, Pyspark, advanced SQL for ingestion frameworks and automation
Hands-on data orchestration experience using DBT cloud, Astronomer Airflow
Experience in implementing logging, monitoring, alerting, observability, performance tuning techniques
Implement and maintain sensitive data protection strategies – tokenization, snowflake data masking policies, dynamic & conditional masking, and role based masking rules

Job Responsibility

Data Architecture and Strategy - Design and implement scalable, efficient data architectures. Lead the development of data strategy aligned with business objectives. Evaluate and integrate new technologies to enhance data capabilities
Hands-on Data Pipeline Development - Implement complex data pipelines for real-time and batch processing. Optimize data flows for high-volume, high-velocity data environments. Develop advanced ETL processes for diverse data sources
Data Governance and Quality Management - Establish and enforce data governance policies and best practices. Implement data quality frameworks and monitoring systems. Ensure compliance with data regulations and standards
Performance Optimization and Troubleshooting - Analyze and optimize system performance for large-scale data operations. Troubleshoot complex data issues and implement robust solutions
Mentorship and Knowledge Sharing - Mentor junior data engineers and provide technical guidance. Contribute to the development of best practices and standards. Collaborate with cross-functional teams to drive data literacy
Testing & Automation – Write unit test cases, validate the data integrity & consistency requirements, build automated data pipelines using GitLab, Git hub, CICD tools
Code Deployment & Release Management - Adopt release management processes to promote code deployment to various environments including production, disaster recovery, and support activities

Fulltime

Analytics Engineer / Sr. Business Intelligence Engineer

Tourism Economics, a subsidiary of Oxford Economics, is looking to hire an Analy...

Location

United States , Philadelphia

Salary:

Not provided

Oxford Economics

Expiration Date

Until further notice

Requirements

3–6 years as an analytics engineer, BI engineer, or data engineer in a data-driven SaaS or analytics-heavy environment
Solid understanding of data modeling principles (Kimball, Data Vault, or modern ELT patterns)
Strong proficiency in SQL and dbt (required)
Experience with modern cloud data warehouses (Snowflake, BigQuery, or Redshift)
Familiarity with orchestration tools such as Airflow, Prefect, or Dagster
Working knowledge of Python for transformation, validation, or automation
Experience with BI tools such as Looker, Tableau, or Power BI

Job Responsibility

Build and own the data modeling layer in Snowflake using dbt, transforming raw datasets into reliable, well-documented tables and metrics
Design scalable, future-proof data structures that support new product features, analytical use cases, and organizational growth
Create and maintain the semantic layer that standardizes metrics and enables self-service analytics
Automate, monitor, and optimize data pipelines to ensure reliability, version control, and transparent data lineage
Partner with BI engineers to deliver intuitive, performant datasets for dashboards and reporting in tools like Looker, Tableau, or Power BI
Work with business stakeholders to gather requirements and translate them into effective, technically sound data solutions
Collaborate with data, product, and analytics teams to identify opportunities for data-driven decision-making
Implement and maintain QA, testing, and data validation standards across all data models
Develop measurement and evaluation frameworks to ensure models meet performance and accuracy expectations
Maintain comprehensive documentation of data models, business logic, and metric definitions

Fulltime

Offshore ETL Sr Engineer

The Senior ETL Engineer at NTT DATA will be responsible for developing complex P...

Location

India , Bangalore

Salary:

Not provided

NTT DATA

Expiration Date

Until further notice

Requirements

7–10 years of data engineering experience
Strong Python expertise for batch data ingestion and processing
Advanced SQL skills (complex joins, window functions, aggregations)
Hands-on experience with dbt Core or dbt Cloud
Experience working with large-scale data migration or ETL modernization programs
Familiarity with Informatica PowerCenter concepts (preferred)
Experience mentoring junior engineers in offshore delivery models

Job Responsibility

Develop and maintain complex Python EL pipelines to land source data as-is into the target warehouse
Implement advanced dbt models, including: Incremental models, Snapshot-based logic (non-CDC), Complex joins and aggregations
Translate high-complexity Informatica mappings into dbt SQL under guidance from SMEs
Design and implement reusable dbt macros and common SQL patterns
Perform peer code reviews and provide technical guidance to Data Engineers
Troubleshoot performance issues and data discrepancies during SIT and UAT
Support wave-based migration execution across SWP and IMS in parallel
Contribute to technical documentation, runbooks, and handover materials

Fulltime

Select Country

Data Engineer Sr (DBT)

Job Responsibility

Requirements

Looking for more opportunities?