CrawlJobs Logo

Dataops Engineer - Security Monitoring & Analytics

India, Pune · Job Posted June 29, 2026
Apply Position
Job Link Share

Job Description

We are seeking a DataOps Engineer to implement, operate, and continuously improve security data onboarding, transformation, and validation pipelines within the Security Monitoring & Analytics (SMA) ecosystem. This role plays a critical part in ensuring data quality, integrity, lineage, and compliance with governance standards. The individual will collaborate closely with cross-functional teams, including onboarding engineers, Site Reliability Engineering (SRE), DevOps, architects, and security operations, to maintain reliable end-to-end data flows and support data-driven cyber defence capabilities.

Job Responsibility

  • Design, onboard, transform, validate, and deliver telemetry, log, and event data into SMA platforms using scalable and reusable ingestion patterns
  • Monitor and maintain data quality, ensuring accuracy, reliability, completeness, and detection of anomalies such as event loss or unexpected log volumes
  • Configure, troubleshoot, and optimise data ingestion pipelines and tools to ensure efficiency and performance
  • Document data schemas, mappings, transformations, and end-to-end data flow processes in alignment with governance standards
  • Implement and maintain data lineage, ownership tagging, and metadata to support governance controls and regulatory requirements
  • Support incident response by analysing data-related issues, performing root cause analysis, and driving remediation and continuous improvement initiatives

Requirements

  • Experienced in building and managing data pipelines for ingestion, transformation, and validation within a DataOps or data engineering environment
  • Proficient in scripting languages such as Python and Bash, with the ability to automate workflows and operational checks
  • Skilled in implementing automated data quality validation and monitoring mechanisms to ensure reliable data delivery
  • Comfortable working in a cyber security context, with an understanding of reliability expectations and support for production issue resolution
  • Effective at collaborating with diverse stakeholders across engineering, operational, and business teams

Nice to have

  • Familiarity (desirable) with cloud platforms (e.g., GCP), data solutions (e.g., Elastic Stack), and visualisation tools (e.g., Kibana)
  • Exposure (desirable) to DevSecOps or IT operations environments supporting security data platforms

What we offer

  • Opportunity to work in a global cyber security function supporting critical 24x7 defence operations
  • Exposure to large-scale data ecosystems and advanced analytics platforms
  • A collaborative environment that encourages continuous improvement and automation
  • Engagement with cross-functional teams across engineering, security, and operations domains
  • A role that contributes directly to strengthening organisational security and data governance

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Dataops Engineer - Security Monitoring & Analytics

8 matching positions

Dataops Engineer (Database)

DataOps Engineer Experience: 3–5 Years Location: Chennai | Madurai | Coimbator...
Location
Location
India , Chennai City Corporation
Salary
Salary:
Not provided
optisolbusiness.com Logo
OptiSol Business Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong experience in Oracle, SQL Server, or PostgreSQL administration
  • Expertise in performance tuning, query optimization, and indexing
  • Proficiency in T-SQL scripting and database automation
  • Hands-on experience troubleshooting database issues
  • Knowledge of cloud platforms (Azure SQL, AWS RDS)
  • Strong analytical, problem-solving, and communication skills
Job Responsibility
Job Responsibility
  • Manage and administer Oracle, SQL Server, or PostgreSQL databases
  • Optimize performance through query tuning, indexing, and execution plans
  • Troubleshoot issues like deadlocks, blocking, and slow queries
  • Implement backup, restore, and disaster recovery strategies
  • Ensure high availability using AAGs, failover clusters, and replication
  • Monitor and maintain database health and performance
  • Automate DBA tasks using T-SQL, PowerShell, and DataOps practices
  • Manage and monitor cloud databases (Azure SQL, AWS RDS)
  • Collaborate with engineering teams for scalable data solutions
  • Support CI/CD and database deployment processes
What we offer
What we offer
  • Opportunity to work on modern DataOps and database systems
  • Exposure to cloud, AI, and advanced data technologies
  • Collaborative and growth-focused work environment
  • Learning, certifications, and career development opportunities
  • Fulltime
Read More
Arrow Right

DataOps Engineer (AWS)

We are looking for a skilled DevOps + Data Engineer (AWS) to design, build, and ...
Location
Location
India
Salary
Salary:
3800000.00 - 4100000.00 INR / Year
northbaysolutions.com Logo
NorthBay
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5–7 years of experience in DevOps, Data Engineering, or Cloud Engineering
  • Strong hands-on experience with AWS services (EC2, S3, RDS, Lambda, EMR, Glue, Redshift, etc.)
  • Solid experience with Terraform for infrastructure automation
  • Experience building and maintaining ETL/data pipelines
  • Proficiency in Python and/or Shell scripting
  • Experience with CI/CD tools (GitHub Actions, Jenkins, GitLab CI, etc.)
  • Strong understanding of Linux systems and networking fundamentals
  • Experience with monitoring tools (CloudWatch, Prometheus, Grafana, ELK, etc.)
Job Responsibility
Job Responsibility
  • Design, deploy, and manage AWS cloud infrastructure using Infrastructure as Code (Terraform)
  • Build, maintain, and optimize ETL/data pipelines for large-scale data processing
  • Automate infrastructure provisioning, deployment, and monitoring
  • Develop and manage CI/CD pipelines for data and application workloads
  • Ensure high availability, scalability, security, and cost optimization of AWS environments
  • Support data ingestion, transformation, and orchestration workflows
  • Monitor systems, troubleshoot issues, and improve system reliability
  • Implement logging, monitoring, and alerting for infrastructure and data pipelines
  • Collaborate with Data Engineers, Analytics teams, and DevOps teams
  • Enforce security best practices, IAM policies, and compliance standards
  • Fulltime
Read More
Arrow Right

Azure Dataops Data Engineer – Ii

We are seeking an Azure DataOps Data Engineer – II with strong hands-on experien...
Location
Location
India , Gurgaon
Salary
Salary:
Not provided
rackspace.com Logo
Rackspace
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3–5 years of experience in Data Engineering / DataOps roles
  • Strong hands-on experience with: Azure Databricks (PySpark, Spark SQL, Delta Lake)
  • Azure Data Factory (ADF) – pipelines, triggers, parameters, monitoring
  • Azure Data Lake Storage (ADLS Gen2)
  • Good understanding of ETL/ELT frameworks, batch and incremental processing
  • Strong SQL skills for data analysis and troubleshooting
  • Experience with production support, incident management, and SLA-driven environments
  • Familiarity with monitoring tools (Azure Monitor, Log Analytics, alerts)
  • Understanding of Azure security concepts (RBAC, Managed Identity, Key Vault)
  • Willingness to work in a rotational shift / on-call support model as part of a global operations team
Job Responsibility
Job Responsibility
  • Support production data platforms, ensuring high availability, reliability, and performance
  • Monitor data pipelines and jobs, proactively identifying and resolving failures, performance issues, and data discrepancies
  • Perform root cause analysis (RCA) for incidents and implement preventive measures
  • Implement DataOps best practices including automation, monitoring, alerting, and operational dashboards
  • Collaborate with cross-functional teams to support reporting, analytics, and downstream consumption
  • Maintain documentation for pipelines, operational runbooks, and support procedures
  • Participate in on-call and rotational shift support, including weekends or night shifts as required
  • Fulltime
Read More
Arrow Right

Senior Data Engineer

In this vital role as Specialist / Senior Data Engineer at Amgen you will be res...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
amgen.com Logo
Amgen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Master's/Bachelor's degree and 8 to 13 years of Computer Science, IT or related field experience
  • Experience managing a team of data engineers
  • Experience architecting and building data and analytics solutions that extract, transform, and load data from multiple source systems
  • Demonstrated hands-on experience with cloud platforms (AWS) and the ability to architect cost-effective and scalable data solutions
  • Proficiency in Python, PySpark, SQL
  • Experience with dimensional data modeling
  • Experience working with Apache Spark, Apache Airflow
  • Experienced with software engineering best-practices, including but not limited to version control (Git, Subversion, etc.), CI/CD (Jenkins, Maven etc.), automated unit testing, and Dev Ops
  • Experienced with AWS or GCP or Azure cloud services
  • Understanding of end to end project/product life cycle
Job Responsibility
Job Responsibility
  • Lead and mentor a team of data engineers, fostering a culture of innovation, collaboration, and continuous learning for solving complex problems of R&D division
  • Oversee the development of data extraction, validation, and transformation techniques, ensuring ingested data is of high quality and compatible with downstream systems
  • Guide the team in writing and validating high-quality code for data ingestion, processing, and transformation, ensuring resiliency and fault tolerance
  • Drive the development of data tools and frameworks for managing and accessing data efficiently across the organization
  • Oversee the implementation of performance monitoring protocols across data pipelines, ensuring real-time visibility, alerts, and automated recovery mechanisms
  • Coach engineers in building dashboards and aggregations to monitor pipeline health and detect inefficiencies, ensuring optimal performance and cost-effectiveness
  • Lead the implementation of self-healing solutions, reducing failure points and improving pipeline stability and efficiency across multiple product features
  • Oversee data governance strategies, ensuring compliance with security policies, regulations, and data accessibility best practices
  • Guide engineers in data modeling, metadata management, and access control, ensuring structured data handling across various business use cases
  • Collaborate with business leaders, product owners, and cross-functional teams to ensure alignment of data architecture with product requirements and business objectives
  • Fulltime
Read More
Arrow Right

Data Architect

At Codvo, we are committed to building scalable, future-ready data platforms tha...
Location
Location
Qatar , Doha
Salary
Salary:
Not provided
Codvo AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or master’s degree in computer science, Data Engineering, Information Systems, or related field
  • 10+ years of experience in data architecture, data engineering, or enterprise analytics roles
  • Strong experience designing enterprise-scale data platforms in cloud and hybrid environments
  • Expertise in data modelling (conceptual, logical, physical) and data integration patterns
  • Deep understanding of data lakes, data warehouses, Lakehouse architectures, and real-time streaming platforms
  • Hands-on experience with cloud data services (Azure preferred)
  • Strong knowledge of SQL, data transformation frameworks, and metadata management
  • Familiarity with DevOps and DataOps practices
  • Strong communication skills with the ability to engage both technical and business stakeholders
Job Responsibility
Job Responsibility
  • Define and maintain the enterprise data architecture blueprint, including data lakes, warehouses, streaming platforms, and analytics layers
  • Design logical, physical, and conceptual data models to support operational, analytical, and AI use cases
  • Ensure architecture supports scalability, performance, and high availability
  • Architect cloud-native data platforms using Azure (ADF, Synapse, Databricks, Fabric) and/or GCP equivalents
  • Design secure hybrid architectures integrating on-premises systems with cloud data platforms
  • Define and enforce data governance frameworks, including metadata management, data lineage, data ownership, and stewardship models
  • Establish data quality standards, validation rules, and monitoring mechanisms
  • Ensure compliance with ISO 27001, NIA Qatar, data privacy, and internal cybersecurity policies
  • Define data security architecture including encryption, access control, and role-based data access
  • Design data architectures that support advanced analytics, AI/ML, and digital twin initiatives
  • Fulltime
Read More
Arrow Right

Senior Data Engineer (Contractor)

As a Senior Data Engineer, you’ll design, build, and operate scalable, reliable ...
Location
Location
Poland , Warsaw
Salary
Salary:
Not provided
mypos.com Logo
myPOS
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience)
  • 6+ years of experience as a Data Engineer, building and maintaining production-grade pipelines and datasets
  • Strong Python and SQL skills with a solid understanding of data structures, performance, and optimization strategies
  • Hands-on experience with orchestration (like Airflow, Dagster, Databricks Workflows) and distributed processing in a cloud environment
  • Experience with analytical data modeling (star and snowflake schemas), DWH, ETL/ELT patterns, and dimensional concepts
  • Experience building reliable incremental data ingestion pipelines from DBs and APIs.
  • Familiarity with at least one major cloud provider (GCP, AWS, Azure) and deploying data solutions in the cloud
  • Familiarity with CI/CD for data pipelines, IaC (Terraform), and/or DataOps practices
  • Strong troubleshooting mindset: ability to debug issues across data, infra, pipelines, and deployments
  • Collaborative mindset and clear communication across engineering, analytics, and business stakeholders
Job Responsibility
Job Responsibility
  • Build and maintain data pipelines for ingestion, transformation, and export across multiple sources and destinations
  • Develop and evolve scalable data architecture to meet business and performance requirements
  • Partner with analysts and data scientists to deliver curated, analysis-ready datasets and enable self-service analytics
  • Implement best practices for data quality, testing, monitoring, lineage, and reliability
  • Optimize workflows for performance, cost, and scalability (e.g., tuning Spark jobs, query optimization, partitioning strategies)
  • Ensure secure data handling and compliance with relevant data protection standards and internal policies
  • Contribute to documentation, standards, and continuous improvement of the data platform and engineering processes
  • Ensure secure, compliant handling of data and models, including access controls, auditability, and governance practices
  • Build and maintain MLOps automation: CI/CD for ML, environment management, artifact handling, versioning of data/models/code
What we offer
What we offer
  • Vibrant international team operating in hi-tech environment
  • Excellent compensation package
  • myPOS Academy for upskilling and training
  • Unlimited access to courses on LinkedIn Learning
  • Refer a friend bonus as we know that working with friends is fun
  • Teambuilding, social activities and networks on a multi-national level
  • Fulltime
Read More
Arrow Right

Senior Data Engineer (Contractor)

As a Senior Data Engineer, you’ll design, build, and operate scalable, reliable ...
Location
Location
Romania , Bucharest
Salary
Salary:
Not provided
mypos.com Logo
myPOS
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience)
  • 6+ years of experience as a Data Engineer, building and maintaining production-grade pipelines and datasets
  • Strong Python and SQL skills with a solid understanding of data structures, performance, and optimization strategies
  • Hands-on experience with orchestration (like Airflow, Dagster, Databricks Workflows) and distributed processing in a cloud environment
  • Experience with analytical data modeling (star and snowflake schemas), DWH, ETL/ELT patterns, and dimensional concepts
  • Experience building reliable incremental data ingestion pipelines from DBs and APIs
  • Familiarity with at least one major cloud provider (GCP, AWS, Azure) and deploying data solutions in the cloud
  • Familiarity with CI/CD for data pipelines, IaC (Terraform), and/or DataOps practices
  • Strong troubleshooting mindset: ability to debug issues across data, infra, pipelines, and deployments
  • Collaborative mindset and clear communication across engineering, analytics, and business stakeholders
Job Responsibility
Job Responsibility
  • Build and maintain data pipelines for ingestion, transformation, and export across multiple sources and destinations
  • Develop and evolve scalable data architecture to meet business and performance requirements
  • Partner with analysts and data scientists to deliver curated, analysis-ready datasets and enable self-service analytics
  • Implement best practices for data quality, testing, monitoring, lineage, and reliability
  • Optimize workflows for performance, cost, and scalability (e.g., tuning Spark jobs, query optimization, partitioning strategies)
  • Ensure secure data handling and compliance with relevant data protection standards and internal policies
  • Contribute to documentation, standards, and continuous improvement of the data platform and engineering processes
  • Ensure secure, compliant handling of data and models, including access controls, auditability, and governance practices
  • Build and maintain MLOps automation: CI/CD for ML, environment management, artifact handling, versioning of data/models/code
What we offer
What we offer
  • Vibrant international team operating in hi-tech environment
  • Excellent compensation package
  • myPOS Academy for upskilling and training
  • Unlimited access to courses on LinkedIn Learning
  • Refer a friend bonus as we know that working with friends is fun
  • Teambuilding, social activities and networks on a multi-national level
  • Fulltime
Read More
Arrow Right

Senior Data Engineer

Are you passionate about shaping the future of AI and empowering millions of use...
Location
Location
Canada , Vancouver
Salary
Salary:
114400.00 - 203900.00 CAD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
Job Responsibility
Job Responsibility
  • Compliance: Anticipates the need for data governance and designs data modeling and data handling procedures, with direct support and partnership with Corporate, External, and Legal Affairs (CELA), to ensure compliance with applicable laws and policies across all aspects of the development process
  • Tags data based on categorization (e.g., personally identifiable information [PII], pseudo-anonymized, financial)
  • Documents data type, build data dictionary, classifications, and lineage to ensure traceability
  • Governs accessibility of data within assigned data pipelines
  • Provides guidance on contributions to the data glossary to document the origin, usage, and format of data for each program
  • Independently implements data governance and privilege of least access practices leveraging security tools
  • Builds responsible AI-compliant data products and/or applications
  • Data Management and Transformation: Plans and creates efficient techniques and operations (e.g., inserting, aggregating, joining) to transform raw data into a form (e.g., dimensional data model) that is compatible with downstream data consumers, databases, and formats that support applications, analytics and reporting
  • Independently uses software, query languages, and computing tools (e.g., cloud-based) to transform raw data across end-to-end pipelines
  • Evaluates data to ensure data quality and completeness using queries, data wrangling, and statistical techniques
  • Fulltime
Read More
Arrow Right