Senior Data Engineer - Platform Team Job at Microsoft Corporation (Multiple Locations)

Senior AWS Data Engineer / Data Platform Engineer

We are seeking a highly experienced Senior AWS Data Engineer to design, build, a...

Location

United Arab Emirates , Dubai

Salary:

Not provided

NorthBay

Expiration Date

Until further notice

Requirements

8+ years of experience in data engineering and data platform development
Strong hands-on experience with: AWS Glue
Amazon EMR (Spark)
AWS Lambda
Apache Airflow (MWAA)
Amazon EC2
Amazon CloudWatch
Amazon Redshift
Amazon DynamoDB
AWS DataZone

Job Responsibility

Design, develop, and optimize scalable data pipelines using AWS native services
Lead the implementation of batch and near-real-time data processing solutions
Architect and manage data ingestion, transformation, and storage layers
Build and maintain ETL/ELT workflows using AWS Glue and Apache Spark on EMR
Orchestrate complex data workflows using Apache Airflow (MWAA)
Develop and manage serverless data processing using AWS Lambda
Design and optimize data warehouses using Amazon Redshift
Implement and manage NoSQL data models using Amazon DynamoDB
Utilize AWS DataZone for data governance, cataloging, and access management
Monitor, log, and troubleshoot data pipelines using Amazon CloudWatch

Fulltime

Senior Staff Data Engineer- Data Platform

At Marktplaats, data is at the heart of everything we do, but Intelligence is wh...

Location

Netherlands , Amsterdam

Salary:

Not provided

Adevinta

Expiration Date

Until further notice

Requirements

10+ years of hands-on experience in Software Development or Data Engineering
at least 5+ years specifically focused on building Data Platforms
deep understanding of how Platform infra supports Analytics workloads
proven experience evolving complex platforms from legacy patterns to modern, cloud-native solutions
deep knowledge of Spark internals, JVM tuning, and performance optimization for high-scale batch and streaming datasets
deep expertise in Unity Catalog, Delta Lake internals, and optimizing high-volume workloads
strict software engineering discipline (CI/CD, Testing, OOP) applied to data pipelines
understanding of microservices architecture
understanding the needs of Analytics/DWH teams (Data Modeling, dbt)
strong background in building automated pipelines using Terraform/Terragrunt and ensuring system observability

Job Responsibility

Lead the evolution of our Data Platform and architect the "Data Exchange" strategy
define robust patterns for API-based ingestion, Event-Driven Architectures (Kafka), and Reverse ETL
ensure architectures are optimized for cost and performance on AWS
act as a catalyst for technical evolution
constantly scan the horizon for next-generation technologies
lead the implementation of new paradigms
design the strategy for Unity Catalog implementation and Data Contracts
champion FinOps, automating cost controls for our highest-volume workloads
build the underlying infrastructure that allows Analytics/DWH teams to run efficient transformations
elevate the technical bar of the team, mentoring Staff and Senior engineers

What we offer

An attractive Base Salary
Participation in our Short Term Incentive plan (annual bonus)
Work From Anywhere: Enjoy up to 20 days a year of working from anywhere
A 24/7 Employee Assistance Program for you and your family
a collaborative environment with an opportunity to explore your potential and grow
a range of locally relevant benefits

Fulltime

Senior Product Designer, Data Platform - Online Data Stores and Movement Platform

We're seeking a Senior Product Designer to shape the next generation of data mov...

Location

United States

Salary:

350000.00 - 520000.00 USD / Year

DesignX Community

Expiration Date

Until further notice

Requirements

5+ years as a Product Designer (or equivalent experience)
Experience designing orchestration and/or data movement tools, databases, developer/data platform tools, or analogous complex technical products
Strong portfolio showing system‑level product work and the impact of your design leadership, including initiatives that span multiple teams
Proven track record of leading design initiatives from concept to launch
Proficient in design tools (Figma, etc.) and prototyping technologies
Strong systems thinking and ability to design scalable solutions for complex data/analytics challenges
Successfully led design strategy for complex technical products
Proven track record of influencing and aligning cross-functional partners without direct authority
Ability to navigate ambiguity, create clarity, and move complex initiatives forward with partners
Excellence in communicating design decisions to technical audiences

Job Responsibility

Shape and own design for ODS and movement platform
creating intuitive workflows that enhance data practitioner and software engineer productivity and delight
Contribute to and help drive adoption of patterns and components for operational, data‑dense UIs (runs, pipelines, lineage, diffs) that scale across the platform
Identify opportunities for innovation through research, incident analysis, and technical insight
sequence delivery for measurable impact
Create cohesive, end‑to‑end experiences with guardrails over gates (paved paths, templates, defaults) to reduce errors
Partner with product and engineering to define and improve how we measure experiences
Anticipate emerging needs and help frame design problems early
Drive end‑to‑end design for interdependent workflows
Craft and validate prototypes that effectively communicate solutions

What we offer

Health Plans
Mental Health support
401(k) Retirement Plan with employer match
Stock Option Program
Disability Programs
Health Savings and Flexible Spending Accounts
Family-forming benefits
Life and Serious Injury Benefits
Paid leave of absence programs
Flexible time off

Fulltime

Senior Data Platform Engineer - Vice President

Fixed Income (FI) is the primary source of capital for corporations, governments...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

10+ years of demonstrable and highly hands-on experience in software development, with at least 3-5 years in a lead technical contributor or staff engineer role within a data-intensive environment
Deep Architectural & Implementation Expertise: Proven expertise in designing, building, and implementing Microservices and complex streaming/data pipeline architectures
Polyglot Proficiency & Technical Authority: Demonstrated ability to be a technical authority and actively contribute across multiple programming languages/frameworks, with strong hands-on experience in Java (Java 11+ preferred), and significant expertise in Python, along with practical knowledge of front-end technologies like Angular/React
Expertise in Flink Ecosystem: In-depth architectural understanding and hands-on experience with Apache Flink for real-time stream processing, including Flink SQL, DataStream API, and state management. Comprehensive knowledge of Flink infrastructure setup, maintenance, enhancement, migration strategies, and experience in evaluating/integrating alternative streaming frameworks
Mastery of Redis: In-depth knowledge and practical experience with Redis, encompassing not only data structures, caching patterns, and pub/sub mechanisms but also strategic expertise in Redis cluster setup, maintenance, enhancement, and migration strategies for mission-critical, high-performance applications
Distributed Systems & Data Engineering: Deep understanding of distributed systems concepts and extensive hands-on experience with data distribution platforms like Apache Kafka, and various big data storage/querying systems (e.g., Trino, Pinot, Druid, Ignite) for low-latency access in large-scale, distributed data pipelines
Cloud-Native & DevOps Contribution: Proven experience with the design, implementation, and operational aspects of cloud-native applications and deployment via Kubernetes / OpenShift / ECS, specifically for managing microservices, streaming, and data services
Technical Leadership, Coaching & Communication: Exceptional ability to provide technical direction, coach junior engineers through hands-on collaboration, and articulate complex technical concepts clearly to diverse audiences
Advanced Problem Solving & Strategic Technical Thinking: Excellent problem-solving skills, a data-driven approach to technical decision-making, and the ability to define and execute long-term technical strategies through direct implementation
Experience with Large Language Models (LLMs), including fine-tuning, prompt engineering, and integrating LLMs into applications, is a plus

Job Responsibility

Hands-on Technical Leadership & Coaching: Provide technical leadership through direct code contribution, architecture review, and hands-on guidance to junior and mid-level data engineers, fostering a culture of technical excellence and continuous learning across polyglot environments
Drive & Implement Architecture & Design: Directly contribute to and drive the technical design and hands-on development of highly scalable, low-latency, fault-tolerant data pipelines, real-time streaming platforms (e.g., Flink), and microservices architectures
Platform Infrastructure & Operations Deep Dive: Actively participate in the strategic planning, setup, enhancement, migration, and operational excellence of critical platform infrastructure, including distributed streaming (Flink) and high-performance caching solutions (Redis)
Open Source & Emerging Technologies Evaluation: Continuously research, evaluate, and prototype emerging open-source data and streaming technologies, leading their integration into Citi's data platform through direct implementation. This includes evaluating new streaming frameworks and caching solutions
Performance & Resilience Optimization: Proactively identify and implement architectural and systemic performance optimizations to ensure optimal efficiency, responsiveness, and resilience across the entire data platform, including Flink and Redis clusters
Cross-Functional Technical Collaboration: Collaborate closely with senior technical stakeholders across engineering, data science, and business teams to translate complex technical requirements into robust, high-impact implementations
Delivery & Code Quality Assurance: Drive the timely, high-quality, and secure delivery of data platform features, championing agile methodologies, engineering best practices, and ensuring accountability for code quality and robust testing
Technical Problem Solving: Act as a primary escalation point for complex technical challenges, providing expert diagnosis and hands-on resolution
Operational Flexibility: May occasionally work a non-standard shift including nights and/or weekends and/or have on-call responsibilities to support critical platform operations

Fulltime

Senior Data Platform Engineer

As a Senior Data Platform Engineer, you will design and deliver innovative, reli...

Location

New Zealand , Wellington

Salary:

Not provided

HSO

Expiration Date

Until further notice

Requirements

5+ years’ experience developing production‑grade data platform and reporting solutions
A degree in Information Technology or a related field
Strong experience with modern cloud technologies and data platforms
Proven experience leading projects and technical teams end to end
In‑depth knowledge of source control, testing frameworks and development methodologies
Experience applying DevOps practices across complex development lifecycles
Strong understanding of data security, privacy obligations and risk mitigation
Excellent problem‑solving and organisational skills, with the ability to manage competing priorities
High emotional intelligence and confidence working with diverse clients and stakeholders
A passion for learning emerging technologies and sharing knowledge with others

Job Responsibility

Lead the design, implementation, testing, support and ongoing maintenance of modern data platform solutions
Apply architectural patterns and best practices to create secure, scalable and maintainable solutions
Deliver end‑to‑end data platform projects, either independently or by leading project teams
Develop and continuously improve data platform lifecycles using DevOps principles
Work directly with customers to scope requirements, identify opportunities and deliver solutions that meet business needs
Clearly communicate complex technical concepts to non‑technical stakeholders
Deliver client workshops to upskill teams and support adoption of new technologies
Act as a technical leader and coach across the Aware Group, supporting team members to achieve project, customer and development goals
Proactively identify risks, issues and mitigation strategies throughout project delivery
Contribute to best practice, knowledge sharing and continuous improvement across HSO

Fulltime

Senior Platform Engineer - Data Reliability

The Feedzai Platform Data Reliability play a pivotal role in managing core data ...

Location

Portugal

Salary:

Not provided

Feedzai

Expiration Date

Until further notice

Requirements

A bachelor's degree in Computer Science, Information Systems, or the equivalent combination of education, experience, and training
4+ years of experience in data reliability, platform engineering, or operating data services at scale
Proficiency in programming languages such as Go, Java, or similar
Hands-on experience with Container Technologies and Orchestration (e.g., Docker, Kubernetes)
Valuable experience with data streaming and messaging platforms, like Kafka, Elasticsearch, RabbitMQ
Familiarity with CI/CD pipelines and tools such as Jenkins, Gitlab, or similar
Demonstrated commitment to staying updated with industry trends and emerging technologies, showcasing a proactive approach to continuous learning
Demonstrated knowledge of best practices in security, ensuring the implementation of secure coding standards
Experience working with Cloud Providers, with a preference for AWS Cloud
Expertise in utilizing monitoring and observability stacks like Grafana and Prometheus

Job Responsibility

Build and maintain Kubernetes Operators, including deployment, monitoring, operations, and analytics tools developed by the team
Engage in development tasks using Go, Java, or similar languages
Operate services such as Kafka, Elasticsearch, RabbitMQ, Redis, Relational databases and Couchbase at an enterprise scale
Contribute to the self-healing capabilities of applications in our enterprise environments
Develop playbooks associated with actionable alerts to streamline response procedures
Work with AI-assisted development tools (e.g. Cursor) as part of your daily workflow to ship faster and iterate effectively
Maintain and enhance our Infrastructure as Code (IaC) to efficiently manage end-to-end lifecycle operations (monitoring, alerting, security, cost optimization, configuration, backup, etc.) in production environments
Utilize your experience and problem solving skills to help prevent and investigate production issues

Senior Data Platform Engineer

As a Data Platform Engineer, you will be responsible for managing one of the lar...

Location

Netherlands , Amsterdam

Salary:

Not provided

Adyen

Expiration Date

Until further notice

Requirements

Familiar with infrastructure automation and configuration management tools (e.g. Ansible, Terraform, Puppet)
Strong system administrator/platform engineer background with experience in large-scale distributed systems
Experienced with infrastructure and private cloud systems (on-premise)
A team player with strong communication skills, able to work closely with diverse stakeholders (analysts, data scientists, data engineers, infrastructure, and security teams)
Experience developing and maintaining: Distributed data and compute systems (Hadoop, Druid, Trino, etc)
Kubernetes (k8s, Docker)
Hadoop ecosystems (Hive, YARN, HDFS, Kerberos)

Job Responsibility

Design, develop, operate, and maintain scalable, reliable, fault-tolerant, and high-performance big data platforms
Work with distributed systems in all shapes and flavors (databases, file systems, compute, etc.)
Shape and maintain continuous improvements and deployment of new clusters
Implement observability solutions (logging, monitoring, alerting) to ensure system reliability and performance
Optimize performance of large-scale distributed data platforms with appropriate cluster configurations, and resource management
Practice sustainable incident response and blameless postmortems, with a focus on automation and root cause analysis
Improve security and compliance by implementing access controls, encryption, and best practices for data governance
Design and build automation tools to reduce toil and enhance the reliability of data platforms
Self-service infrastructure to empower data teams while maintaining platform stability

Senior Data Platform Engineer

In 2025, the Data Platform Engineering team is going to implement various data p...

Location

Singapore , Singapore

Salary:

Not provided

Airwallex

Expiration Date

Until further notice

Requirements

5+ years of relevant experience as a data engineer
Bachelor's or Master's degree in CS/CE/CIS (or equivalent experience)
Knowledge of Kotlin / Java / Scala / Python / SQL
Knowledge of Kafka, Data Lake, Spark, Flink, DBT, Spring Boot, Hadoop, BigQuery, and Snowflake is preferred
Ability to take ownership of designing, building, and operating distributed systems and establishing overarching data architecture
Possess strong working knowledge of Real-time/Batch processing systems
Have experience with working data at PB scale

Job Responsibility

Engage in technical decision-making, custom development, integration, and performance optimization at the foundational level, extending to the development of service layer infrastructure components, application layer, and data processing, including third-party data integration
Develop and automate large-scale, high-performance distributed data processing systems (batch/streaming)
Advocate for high-quality software engineering practices to establish scalable data infrastructure and pipelines
Contribute to defining shared data engineering tool standards
Provide guidance to junior engineers
Take the lead in system design
Drive medium and large-scale projects that create an impact across the entire team
Provide dotted-line guidance to Data Platform Engineer II(s) to support them in driving project delivery

Fulltime

Select Country

Senior Data Engineer - Platform Team

Job Description

Job Responsibility

Requirements

Looking for more opportunities?