CrawlJobs Logo
Altamira Technologies Logo Altamira Technologies · IT - Software Development

Data Engineer (Kafka)

United States, Dayton, OH · Job Posted March 04, 2026
Apply Position
Job Link Share

Job Description

Altamira is seeking a Data Engineer to design, build, and operate high-performance data pipelines and event-driven systems supporting mission-critical platforms. This role focuses on implementing and managing Apache Kafka–based messaging architectures and integrating real-time data streams with cloud-native applications and analytics platforms. The ideal candidate brings strong experience in distributed systems, data streaming technologies, and cloud environments, and is comfortable working in secure, high-reliability environments.

Job Responsibility

  • Design, deploy, and operate Apache Kafka clusters in classified and hybrid environments
  • Build and maintain reliable, scalable, and secure data streaming pipelines
  • Develop and optimize producers, consumers, and stream processing applications
  • Configure and manage topics, partitions, replication, and retention policies
  • Monitor, tune, and troubleshoot Kafka performance, availability, and latency
  • Integrate streaming platforms with databases, storage systems, and analytics tools
  • Implement data governance, retention, and access control policies
  • Automate deployment and management of streaming infrastructure
  • Collaborate with platform, infrastructure, and application teams to support data requirements
  • Support system accreditation, compliance, and security requirements
  • Participate in architecture design and technical planning activities

Requirements

  • Active TS/SCI clearance
  • Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent experience)
  • Experience in data engineering, distributed systems, or backend engineering roles
  • Hands-on experience with Apache Kafka in production environments
  • Experience building and supporting real-time data pipelines
  • Strong proficiency in Java, Python, Scala, or similar programming languages
  • Experience working in AWS or hybrid cloud environments
  • Strong Linux systems administration and troubleshooting skills
  • Ability to work effectively in secure, mission-focused environments

Nice to have

  • Experience with Kafka Connect, Kafka Streams, or similar frameworks
  • Experience with stream processing platforms (Flink, Spark Streaming, etc.)
  • Experience with PostgreSQL, Redis, ArangoDB, or other data platforms
  • Experience with object storage systems such as MinIO or S3
  • Familiarity with Kubernetes-based deployments
  • Experience implementing data security and compliance controls
  • Prior experience supporting DoD or Intelligence Community programs
Altamira Technologies - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Data Engineer (Kafka)

8 matching positions

Kafka Data Engineer

The Kafka Data Engineer role requires expertise in Kafka and hands-on experience...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong expertise in Kafka (4-5 years), with hands-on experience designing and operating large-scale, highly available event-streaming platforms, including partitioning strategies, consumer group optimization, schema management, and performance tuning
  • Strong hands-on pulling data from REST/GraphQL APIs with auth (OAuth2, API keys), pagination, rate limits, retries/backoff, and webhooks
  • strong Python skills to normalize/enrich data and land it cleanly into S3 (schema, partitioning, Parquet)
  • Comfortable building/operating S3-based lakes with layered zones (raw → harmonized → conformed → modeled), Glue Data Catalog, IAM/Secrets Manager, VPC endpoints, encryption, lifecycle/versioning, and cost/perf best practices (file sizing, compaction)
  • Designs and optimizes Glue jobs using PySpark/DynamicFrames, bookmarks for incremental loads, dependency packaging, robust error handling, logging/metrics, and unit tests
  • knows how to tune jobs for scale and cost
  • Writes clean, parameterized, idempotent DAGs (sensors, SLAs, retries, alerts), manages dependencies across pipelines, and uses Git-based CI/CD to promote changes safely
  • Builds ELT models (staging/ODS/marts), tunes performance (warehouse sizing, clustering, micro-partitions, caching), uses Streams/Tasks/Snowpipe for CDC, and follows solid RBAC and data governance practices
Read More
Arrow Right

Data Engineer - Security (Kafka Experience)

The Data Engineer - Security role focuses on designing and operating large-scale...
Location
Location
India , Remote
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Kafka-Strong expertise in Kafka (4-5 years), with hands-on experience designing and operating large-scale, highly available event-streaming platforms, including partitioning strategies, consumer group optimization, schema management, and performance tuning
  • API-first data ingestion. Strong hands-on pulling data from REST/GraphQL APIs with auth (OAuth2, API keys), pagination, rate limits, retries/backoff, and webhooks
  • strong Python skills to normalize/enrich data and land it cleanly into S3 (schema, partitioning, Parquet)
  • AWS data lake, end to end. Comfortable building/operating S3-based lakes with layered zones (raw → harmonized → conformed → modeled), Glue Data Catalog, IAM/Secrets Manager, VPC endpoints, encryption, lifecycle/versioning, and cost/perf best practices (file sizing, compaction)
  • AWS Glue + PySpark expert. Designs and optimizes Glue jobs using PySpark/DynamicFrames, bookmarks for incremental loads, dependency packaging, robust error handling, logging/metrics, and unit tests
  • knows how to tune jobs for scale and cost
  • Airflow orchestration. Writes clean, parameterized, idempotent DAGs (sensors, SLAs, retries, alerts), manages dependencies across pipelines, and uses Git-based CI/CD to promote changes safely
  • Snowflake proficiency. Builds ELT models (staging/ODS/marts), tunes performance (warehouse sizing, clustering, micro-partitions, caching), uses Streams/Tasks/Snowpipe for CDC, and follows solid RBAC and data governance practices
Job Responsibility
Job Responsibility
  • Designing and operating large-scale event-streaming platforms using Kafka
  • API-first data ingestion
  • Building/operating S3-based lakes
  • Designing and optimizing Glue jobs using PySpark/DynamicFrames
  • Writing clean, parameterized, idempotent DAGs
  • Building ELT models in Snowflake
  • Fulltime
Read More
Arrow Right

Data Engineer - Python AND Kafka AND (Hadoop OR HDFS OR Hive) AND Snowflake AND apache AND (iceberg

The Data Engineer role at NTT DATA requires a Bachelor’s or Master’s degree in C...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field
  • 5–7 years of professional hands-on coding experience in collaborative, team-based environments
  • strong troubleshooting skills in SQL and scripting
  • proficiency in Python or Java
  • deep familiarity with SDLC, CI/CD best practices, and Kubernetes deployment
  • expertise in temporal data modeling (e.g., SCD Type 2)
  • schema management with a focus on schema evolution (Iceberg Apache)
  • performance optimization through data partitioning and clustering
  • architectural theory involving normalization/denormalization and natural vs. surrogate keys
  • experience with Python
Job Responsibility
Job Responsibility
  • Designing and implementing data solutions
  • optimizing performance
  • collaborating within a team
  • Fulltime
Read More
Arrow Right

Data Engineer - Python AND Kafka AND (Hadoop OR HDFS OR Hive) AND Snowflake AND apache AND (iceberg

The Data Engineer will play a crucial role in migrating data from on-prem DataLa...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field
  • Minimum of 3-5 years of professional hands-on-keyboard coding experience in a collaborative, team-based environment
  • Ability to troubleshoot SQL and basic scripting experience
  • Professional proficiency in Python or Java
  • Deep familiarity with the full Software Development Life Cycle (SDLC) and CI/CD best practices
  • K8s deployment experience
  • Sophisticated understanding of Temporal Data Modeling, Schema Management, Performance Optimization, and Architectural Theory
  • Experience with Kafka, ANSI SQL, FTP, Apache Spark
  • Experience with JSON, Avro, Parquet
  • Experience with Hadoop (HDFS/Hive), Snowflake, Apache Iceberg, Sybase IQ
Job Responsibility
Job Responsibility
  • Perform end-to-end datastore migration from on-prem DataLake to AWS hosted LakeHouse
  • Pipeline Migration - Refactoring and migrating extraction logic and job scheduling from legacy frameworks to the new Lakehouse environment
  • Data Transfer - Executing the physical migration of underlying datasets while ensuring data integrity
  • Stakeholder Engagement - Acting as a technical liaison to internal clients, facilitating handoff and sign-off conversations with data owners to ensure migrated assets meet business requirements
  • Consumption Pattern Migration - Translating and optimizing legacy SQL and Spark-based consumption patterns for compatibility with Snowflake and Iceberg
  • Usage analysis to understand usage patterns and deliver required data products
  • Data Reconciliation and Quality - Work with reconciliation frameworks to build confidence that migrated data is functionally equivalent to that already used within production flows
  • Fulltime
Read More
Arrow Right

Data Engineer - Streaming (WkStream 2 - Kafka)

The Data Engineer - Streaming role involves designing and implementing PySpark S...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Apache Kafka – Producer & Consumer: 4+ years of hands-on experience with Apache Kafka, including both producer and consumer development in PySpark, Java, or Scala
  • Deep understanding of Kafka internals: topics, partitions, consumer groups, offsets, rebalancing, and exactly-once delivery semantics
  • Experience with Confluent Kafka: schema registry, Avro/JSON serialisation, and Confluent Cloud or on-prem cluster configuration
  • Proven ability to build ingestion pipelines without relying on unsupported or third-party sink connectors — using only native Kafka consumer APIs and Spark integration
  • Familiarity with Kafka Connect architecture to evaluate trade-offs and articulate why application-level ingestion is preferred in constrained environments
  • PySpark Structured Streaming: Strong practical experience with PySpark Structured Streaming: Kafka source, file source, foreachBatch, output modes (append/update/complete), and checkpoint management
  • Experience tuning streaming micro-batch trigger intervals, watermarking, and late data handling for production workloads
  • Hands-on experience writing streaming data directly to Apache Iceberg tables using the Iceberg Spark runtime
  • Ability to implement robust error handling: dead-letter queues, parse error isolation, and recovery from checkpoint failures
  • Data Engineering & Iceberg: Working knowledge of Apache Iceberg: catalog configuration, schema definition, append writes, and partition strategy for event and log data
Job Responsibility
Job Responsibility
  • Design and implement a PySpark Structured Streaming application that reads from Confluent Kafka topics, parses JSON and Avro payloads, applies schema mappings, and writes atomically to Iceberg tables using the Iceberg Spark runtime and foreachBatch micro-batch pattern
  • Ensure all functionality relies exclusively on public Apache-supported APIs — Apache Spark, Apache Kafka, and Apache Iceberg — with no unsupported Confluent connectors or proprietary sinks
  • Configure Kafka source parameters: bootstrap servers, consumer group IDs, offset management (startingOffsets, failOnDataLoss), checkpoint paths, and trigger intervals
  • Implement PII detection and Protegrity tokenization hooks within the ingestion pipeline before data lands in the Iceberg Bronze layer
  • Write comprehensive unit and integration tests: row count validation, schema conformance checks, Kafka offset commit verification, and data comparison against the source topic
  • Support PNC UAT — walk PNC engineers through the code, demonstrate no unsupported connectors are used, and address review findings
  • Fulltime
Read More
Arrow Right

Senior Data Engineer - Data Platform

We are looking for a Senior Data Engineer - Data Platform to join our Data & AI ...
Location
Location
France , Paris
Salary
Salary:
Not provided
doctolib.fr Logo
Doctolib
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • More than 7 years of experience as Site Reliability Engineer, Data Ops, Data Platform Engineer or in a similar role, with a proven track record of building and maintaining complex data infrastructures
  • Strong proficiency in data engineering and infrastructure tools and technologies, such as stream and events processing (Kafka, PubSub, Firehose) and Kubernetes
  • Expertise in programming languages like Python
  • Familiar with cloud infrastructure and services, preferably AWS, Azure, or GCP, and have experience with infrastructure-as-code tools such as Terraform
  • Excellent problem-solving skills with a focus on identifying and resolving data infrastructure bottlenecks and performance issues
Job Responsibility
Job Responsibility
  • Design and implement a scalable and reliable data infrastructure that supports the collection, processing, storage, and analysis of large-scale datasets while pushing security and privacy best practices
  • Build and maintain data pipelines that efficiently extract, transform, and load data from various sources into our data warehouse
  • Implement automation and orchestration tools to streamline infrastructure provisioning, data workflows, reduce manual effort, and improve operational efficiency
  • Monitor data platform for performance and reliability, identify and troubleshoot issues, and implement proactive solutions to ensure data quality and availability
  • Streamline and monitor platform costs, identify optimizations and saving opportunities while collaborating with data engineers, data scientists, and other stakeholders
What we offer
What we offer
  • Free comprehensive health insurance for you and your children
  • Parent Care Program: receive one additional month of leave on top of the legal parental leave
  • Free mental health and coaching services through our partner Moka.care
  • For caregivers and workers with disabilities, a package including an adaptation of the remote policy, extra days off for medical reasons, and psychological support
  • Work from EU countries and the UK for up to 10 days per year, thanks to our flexibility days policy
  • Up to 14 days of RTT
  • A subsidy from the work council to refund part of the membership to a sport club or a creative class
  • Lunch voucher with Swile card
  • Fulltime
Read More
Arrow Right

Senior Staff Data Engineer- Data Platform

At Marktplaats, data is at the heart of everything we do, but Intelligence is wh...
Location
Location
Netherlands , Amsterdam
Salary
Salary:
Not provided
adevinta.com Logo
Adevinta
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of hands-on experience in Software Development or Data Engineering
  • at least 5+ years specifically focused on building Data Platforms
  • deep understanding of how Platform infra supports Analytics workloads
  • proven experience evolving complex platforms from legacy patterns to modern, cloud-native solutions
  • deep knowledge of Spark internals, JVM tuning, and performance optimization for high-scale batch and streaming datasets
  • deep expertise in Unity Catalog, Delta Lake internals, and optimizing high-volume workloads
  • strict software engineering discipline (CI/CD, Testing, OOP) applied to data pipelines
  • understanding of microservices architecture
  • understanding the needs of Analytics/DWH teams (Data Modeling, dbt)
  • strong background in building automated pipelines using Terraform/Terragrunt and ensuring system observability
Job Responsibility
Job Responsibility
  • Lead the evolution of our Data Platform and architect the "Data Exchange" strategy
  • define robust patterns for API-based ingestion, Event-Driven Architectures (Kafka), and Reverse ETL
  • ensure architectures are optimized for cost and performance on AWS
  • act as a catalyst for technical evolution
  • constantly scan the horizon for next-generation technologies
  • lead the implementation of new paradigms
  • design the strategy for Unity Catalog implementation and Data Contracts
  • champion FinOps, automating cost controls for our highest-volume workloads
  • build the underlying infrastructure that allows Analytics/DWH teams to run efficient transformations
  • elevate the technical bar of the team, mentoring Staff and Senior engineers
What we offer
What we offer
  • An attractive Base Salary
  • Participation in our Short Term Incentive plan (annual bonus)
  • Work From Anywhere: Enjoy up to 20 days a year of working from anywhere
  • A 24/7 Employee Assistance Program for you and your family
  • a collaborative environment with an opportunity to explore your potential and grow
  • a range of locally relevant benefits
  • Fulltime
Read More
Arrow Right
New

Data Engineer (Business Data)

As a Data Engineer on the R&D Team, you will help FreshBooks build and evolve hi...
Location
Location
Canada , Toronto
Salary
Salary:
102400.00 - 128000.00 CAD / Year
freshbooks.com Logo
FreshBooks
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2+ years of experience working in data engineering, analytics engineering, or a related field
  • Experience building and maintaining data models and transformation pipelines (e.g., dbt or similar tools)
  • Strong SQL skills and proficiency in Python (or similar language)
  • Solid understanding of data modeling concepts (e.g., dimensional modeling, normalization, data warehousing patterns)
  • Experience working with a cloud data warehouse (e.g., BigQuery, Snowflake, Redshift)
  • Familiarity with orchestrators such as Airflow, GCC, Dagster, Prefect (or similar tools)
  • Basic understanding or exposure to streaming/event-driven systems (e.g., Pub/Sub, Kafka, Kinesis, Dataflow)
  • Understanding of data quality, testing, and validation practices
  • Ability to work cross-functionally and communicate clearly with both technical and non-technical stakeholders
Job Responsibility
Job Responsibility
  • Architect, design, and develop clean, high-performance datasets using modern tools like dbt and BigQuery, focusing on usability and scalability for analytical consumption
  • Be a key contributor to our domain-oriented data architecture, defining how core business entities (e.g., customers, payments) are modeled, governed, and exposed across the organization
  • Build and maintain robust batch and streaming data pipelines that transform raw data into trusted, analytics-ready assets to support both near real-time and traditional use cases
  • Collaborate closely with Analytics, Product, and Machine Learning teams to translate complex requirements into reusable, well-governed data models and contracts
  • Champion data quality, reliability, and documentation by implementing rigorous testing, validation, and monitoring practices
  • Leverage cutting-edge tools, including AI/agentic workflows, to accelerate development, enhance productivity, and improve data exploration and lineage
  • Participate in code reviews, contribute to improving engineering standards, and partner with platform teams to ensure our data solutions meet ambitious performance, cost, and scalability goals
What we offer
What we offer
  • Comprehensive health and wellness benefits
  • Generous time off including a flexible vacation plan
  • Retirement savings program or pension plan matched to your local office
  • Stock options for every full-time employee
  • Parental leave and new parent support
  • Annual healthy living credit
  • Comprehensive medical and dental benefits
  • Fertility and gender-affirming benefits
  • Peer Recognition Program
  • Employee Assistance Program
  • Fulltime
Read More
Arrow Right