CrawlJobs Logo
Randstad Logo Randstad · IT - Software Development

Level 2 Data Engineer

India, Bellandur · Job Posted May 26, 2026
Apply Position
Job Link Share

Job Responsibility

  • Develop pipelines using the standard patterns for data pipelines and workflows utilizing Kestra, dbt, Git
  • Design and Implement data storage and processing solutions employing Snowflake
  • Utilize AWS services for cloud-based platform tooling infrastructure including but not limited to: Lambda, ECS, MSK, RDS, EC2, Secrets Manager, ALB, Cloud Watch, Event Bridge
  • Utilize Terraform for AWS and Azure deployments
  • Leverage and integrate APIs for data access and manipulation
  • Write Python scripts for data common processing and automation tasks
  • Leverage Platform API's and Web Applications to enforce Platform Security
  • Development experience with SQL, Python, C#, .net, JavaScript, shell scripts & container platforms like Docker
  • Create and execute comprehensive test plans to ensure the pipelines functionality and performance
  • Develop unit tests, integration tests, and end-to-end tests for data pipelines and workflows
  • Ensure data accuracy and consistency through rigorous testing processes
  • Leverage automated testing processes to enhance efficiency
  • Monitor and manage the platform to ensure optimal performance and uptime
  • Conduct regular maintenance tasks such as updates, patches, and backups
  • Resolve any issues or incidents related to the platform in a timely manner
  • Continuously improve platform operations through automation and optimization
  • Implement security best practices throughout the pipeline development and deployment process
  • Conduct regular security reviews and vulnerability assessments
  • Ensure data encryption, access control, and other security measures are enforced
  • Use credential management platforms like Thycotic Secret Server, AWS Secrets Manager
  • Assist in troubleshooting and resolving intricate technical issues

Requirements

  • Bachelor's or Master's degree in Computer Science, Data Science, Information Technology, or a related field with a focus on data engineering or data analytics
  • Strong proficiency in programming languages such as Python, SQL, Java, or Scala for data processing and analysis
  • Experience with data modeling, ETL processes, data warehousing, data integration, and data pipeline development
  • Proficiency in relational databases (e.g., SQL, PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra)
  • Working knowledge of cloud platforms such as Snowflake, AWS, Azure, or Google Cloud Platform for data storage and processing
  • Experience with data visualization tools (e.g., Power BI) to create meaningful insights from data
  • Understanding of data quality principles, data governance, and data validation processes
  • Ability to analyze complex data sets, identify trends, patterns, and insights to drive data-driven decision-making
  • Proficiency in troubleshooting data-related issues, identifying root causes, and implementing solutions
  • Familiarity with project management methodologies to contribute effectively to project planning and execution
  • Strong verbal and written communication skills to collaborate with cross-functional teams and communicate technical concepts to non-technical stakeholders
  • Willingness to stay updated with the latest data technologies, tools, and industry trends to enhance data engineering skills
  • Prior experience in data engineering, data analytics, or related roles with a track record of successful data project delivery
  • Ability to make informed, strategic decisions that align technology with business objectives, while balancing short-term and long-term trade-offs
  • Deep understanding of customer needs and how to translate them into effective technical solutions that drive business value
  • Encourages collaboration across teams and stakeholders, breaking down silos and ensuring alignment
  • Strong analytical and problem-solving skills, capable of addressing complex technical challenges and delivering innovative solutions
  • Ability to lead innovation in technology while maintaining an eye on product-market fit and user experience
  • Adaptability in a fast-moving environment, with a mindset focused on delivering high-impact solutions quickly and iteratively

What we offer

  • Commitment to your ongoing development, including on the job opportunities and formal programs
  • Inclusive parental leave entitlements for both parents
  • Values led culture
  • Flexible work options
  • Generous annual leave, sick leave and casual leave
  • Cultural and religious leave with flexible public holiday opportunities
  • A competitive remuneration package featuring performance based incentives with uncapped Employer Provident Fund
Randstad - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Level 2 Data Engineer

8 matching positions

Senior Data Engineer

We’re growing our team at ELEKS in partnership with a company, the UK’s largest ...
Location
Location
Salary
Salary:
Not provided
eleks.com Logo
ELEKS
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience in Data Engineering, SQL, ETL(data validation + data mapping + exception handling) 4+ years
  • Hands-on experience with Databricks 2+ years
  • Experience with Python
  • Experience with AWS (e.g. S3, Redshift, Athena, Glue, Lambda, etc.)
  • At least an Upper-Intermediate level of English
Job Responsibility
Job Responsibility
  • Building Databases and Pipelines: Developing databases, data lakes, and data ingestion pipelines to deliver datasets for various projects
  • End-to-End Solutions: Designing, developing, and deploying comprehensive solutions for data and data science models, ensuring usability for both data scientists and non-technical users. This includes following best engineering and data science practices
  • Scalable Solutions: Developing and maintaining scalable data and machine learning solutions throughout the data lifecycle, supporting the code and infrastructure for databases, data pipelines, metadata, and code management
  • Stakeholder Engagement: Collaborating with stakeholders across various departments, including data platforms, architecture, development, and operational teams, as well as addressing data security, privacy, and third-party coordination
What we offer
What we offer
  • Close cooperation with a customer
  • Challenging tasks
  • Competence development
  • Ability to influence project technologies
  • Team of professionals
  • Dynamic environment with low level of bureaucracy
Read More
Arrow Right

Data Engineer, Solutions Architecture

We are seeking a talented Data Engineer to design, build, and maintain our data ...
Location
Location
United States , Scottsdale
Salary
Salary:
90000.00 - 120000.00 USD / Year
clearwayenergy.com Logo
Clearway Energy
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2-4 years of hands-on data engineering experience in production environments
  • Bachelor's degree in Computer Science, Engineering, or a related field
  • Proficiency in Dagster or Airflow for pipeline scheduling, dependency management, and workflow automation
  • Advanced-level Snowflake administration, including virtual warehouses, clustering, security, and cost optimization
  • Proficiency in dbt for data modeling, testing, documentation, and version control of analytical transformations
  • Strong Python and SQL skills for data processing and automation
  • 1-2+ years of experience with continuous integration and continuous deployment practices and tools (Git, GitHub Actions, GitLab CI, or similar)
  • Advanced SQL skills, database design principles, and experience with multiple database platforms
  • Proficiency in AWS/Azure/GCP data services, storage solutions (S3, Azure Blob, GCS), and infrastructure as code
  • Experience with APIs, streaming platforms (Kafka, Kinesis), and various data connectors and formats
Job Responsibility
Job Responsibility
  • Design, deploy, and maintain scalable data infrastructure to support enterprise analytics and reporting needs
  • Manage Snowflake instances, including performance tuning, security configuration, and capacity planning for growing data volumes
  • Optimize query performance and resource utilization to control costs and improve processing speed
  • Build and orchestrate complex ETL/ELT workflows using Dagster to ensure reliable, automated data processing for asset management and energy trading
  • Develop robust data pipelines that handle high-volume, time-sensitive energy market data and asset generation and performance metrics
  • Implement workflow automation and dependency management for critical business operations
  • Develop and maintain dbt models to transform raw data into business-ready analytical datasets and dimensional models
  • Create efficient SQL-based transformations for complex energy market calculations and asset performance metrics
  • Support advanced analytics initiatives through proper data preparation and feature engineering
  • Implement comprehensive data validation, testing, and monitoring frameworks to ensure accuracy and consistency across all energy and financial data assets
What we offer
What we offer
  • generous PTO
  • medical, dental & vision care
  • HSAs with company contributions
  • health FSAs
  • dependent daycare FSAs
  • commuter benefits
  • relocation
  • a 401(k) plan with employer match
  • a variety of life & accident insurances
  • fertility programs
  • Fulltime
Read More
Arrow Right

Data Engineering Lead

Embark on an exciting journey into the realm of software product development wit...
Location
Location
India
Salary
Salary:
Not provided
3pillarglobal.com Logo
3Pillar Global
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of experience in Data Engineering or related field, including 2+ years in a lead role
  • Expert-level proficiency with AWS data services (e.g., Glue, EMR, Lambda, Redshift, S3, Kinesis, Step Functions)
  • Strong Python skills for data processing, automation, and pipeline development
  • Experience building batch and streaming pipelines (Spark, PySpark, Kafka, Kinesis, etc.)
  • Strong SQL expertise and experience with relational and NoSQL databases
  • Hands-on experience with IaC (Terraform, CloudFormation, CDK)
  • Familiarity with DevOps tools for CI/CD (e.g., GitHub Actions, GitLab CI, Jenkins)
  • Understanding of data modeling, data warehousing concepts, and distributed systems
  • Fulltime
Read More
Arrow Right

Data Engineer

We are looking for an experienced Data Engineer with deep expertise in Databrick...
Location
Location
Salary
Salary:
Not provided
coherentsolutions.com Logo
Coherent Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Engineering, Mathematics, or a related field
  • 5+ years of experience in data engineering, with at least 2 years of hands-on experience with Databricks (including Spark, Delta Lake, and MLflow)
  • Strong proficiency in Python and/or Scala for data processing
  • Deep understanding of distributed data processing, data warehousing, and ETL concepts
  • Experience with cloud data platforms (Azure Data Lake, AWS S3, or Google Cloud Storage)
  • Solid knowledge of SQL and experience with large-scale relational and NoSQL databases
  • Familiarity with CI/CD, DevOps, and infrastructure-as-code practices for data engineering
  • Experience with data governance, security, and compliance in cloud environments
  • Excellent problem-solving, communication, and leadership skills
  • English: Upper Intermediate level or higher
Job Responsibility
Job Responsibility
  • Lead the design, development, and deployment of scalable data pipelines and ETL processes using Databricks (Spark, Delta Lake, MLflow)
  • Architect and implement data lakehouse solutions, ensuring data quality, governance, and security
  • Optimize data workflows for performance and cost efficiency on Databricks and cloud platforms (Azure, AWS, or GCP)
  • Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver actionable insights
  • Mentor and guide junior engineers, promoting best practices in data engineering and Databricks usage
  • Develop and maintain documentation, data models, and technical standards
  • Monitor, troubleshoot, and resolve issues in production data pipelines and environments
  • Stay current with emerging trends and technologies in data engineering and Databricks ecosystem
What we offer
What we offer
  • Technical and non-technical training for professional and personal growth
  • Internal conferences and meetups to learn from industry experts
  • Support and mentorship from an experienced employee to help you professional grow and development
  • Internal startup incubator
  • Health insurance
  • English courses
  • Sports activities to promote a healthy lifestyle
  • Flexible work options, including remote and hybrid opportunities
  • Referral program for bringing in new talent
  • Work anniversary program and additional vacation days
Read More
Arrow Right

Data Engineer

We are looking for a Data Engineer to join a data-intensive real estate/proptech...
Location
Location
Portugal
Salary
Salary:
Not provided
ascendixtech.com Logo
Ascendix Tech
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Snowflake – Advanced level, including User-Defined Functions (UDFs), stored procedures, and performance optimization
  • Python – Proficient in data manipulation, automation, and scripting (pandas, SQL connectors)
  • SQL – Expert level, including query optimization
  • Data Modeling – Dimensional modeling (star/snowflake schemas), normalization (3NF)
  • OLTP & OLAP – Understanding of transactional vs. analytical database design
  • Data Quality – Data cleansing, validation, consistency checks
  • Real estate data (property types, comparable analysis)
  • Multi-source data integration and harmonization
  • KPI reporting and dashboards
  • Real estate or property data systems (preferred)
Job Responsibility
Job Responsibility
  • Design scalable data models for high-volume property data (2,600-6,500 properties/month)
  • Build and maintain data integration pipelines from multiple sources
  • Implement complex entity relationships and dynamic grouping
  • Optimize for both transactional and analytical workloads
  • Develop Snowflake UDFs and stored procedures for business logic
What we offer
What we offer
  • Health insurance
  • Vacation days: 22 days per year
  • Paid Time off Benefits
  • Meal Card
  • Friendly and calm atmosphere in the company
  • Individual development plan
  • Technical leads and mentors
  • Open management and well-established processes
  • Regular performance reviews
  • Free access to the company accounts on educational platforms (Udemy, Pluralsight)
  • Fulltime
Read More
Arrow Right

Data Engineer

PulsePoint Data Engineering team plays a key role in our technology company that...
Location
Location
Salary
Salary:
Not provided
pulsepoint.com Logo
PulsePoint
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of data engineering experience
  • Strong skills in and current experience with SQL and Python
  • Strong recent Spark experience (3+ years)
  • Experience working in on-prem environments
  • Hadoop and Hive experience
  • Proficiency in Linux
  • Strong understanding of RDBMS and query optimization
  • Passion for engineering and computer science around data
  • East Coast U.S. hours 9am-6pm EST
  • Notice period needs to be less than 2 months (or 2 months max)
Job Responsibility
Job Responsibility
  • Design, build, and maintain reliable and scalable enterprise-level distributed transactional data processing systems for scaling the existing business and supporting new business initiatives
  • Optimize jobs to utilize Kafka, Hadoop, Presto, Spark, and Kubernetes resources in the most efficient way
  • Monitor and provide transparency into data quality across systems (accuracy, consistency, completeness, etc)
  • Increase accessibility and effectiveness of data (work with analysts, data scientists, and developers to build/deploy tools and datasets that fit their use cases)
  • Collaborate within a small team with diverse technology backgrounds
  • Provide mentorship and guidance to junior team members
Read More
Arrow Right

Data Engineer

We are looking for a Data Engineer to join a data-intensive real estate/proptech...
Location
Location
Salary
Salary:
Not provided
ascendixtech.com Logo
Ascendix Tech
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Snowflake – Advanced level, including User-Defined Functions (UDFs), stored procedures, and performance optimization
  • Python – Proficient in data manipulation, automation, and scripting (pandas, SQL connectors)
  • SQL – Expert level, including query optimization
  • Data Modeling – Dimensional modeling (star/snowflake schemas), normalization (3NF)
  • OLTP & OLAP – Understanding of transactional vs. analytical database design
  • Data Quality – Data cleansing, validation, consistency checks
  • Real estate data (property types, comparable analysis)
  • Multi-source data integration and harmonization
  • KPI reporting and dashboards
  • Real estate or property data systems (preferred)
Job Responsibility
Job Responsibility
  • Design scalable data models for high-volume property data (2,600-6,500 properties/month)
  • Build and maintain data integration pipelines from multiple sources
  • Implement complex entity relationships and dynamic grouping
  • Optimize for both transactional and analytical workloads
  • Develop Snowflake UDFs and stored procedures for business logic
What we offer
What we offer
  • 20 paid vacation
  • 5 paid sickness
  • B2B and remote cooperation
  • Career development and professional growth
  • Technical performance reviews and 360 Evaluation
  • Collaboration with a personal Direct Manager, Technical leads, and mentors
  • Udemy and Pluralsight subscription
  • Internal knowledge-sharing opportunities
  • Online and offline events & team-buildings
  • Friendly and supportive atmosphere
  • Fulltime
Read More
Arrow Right

Systems Engineer 2

SE2s work directly with our clients, as well as other system engineers, to provi...
Location
Location
Salary
Salary:
Not provided
atlasps.com Logo
Atlas Professional Services
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Mid-level IT engineer with minimum 2+ years of experience
  • Must have IT certifications, network plus or equivalent
  • Able to work in a fast paced, fast-growing environment
  • Able to work on support tickets independently and update status accurately
  • Knows when to escalate to higher level support
  • Able to learn and adapt quickly
  • Intermediate knowledge and experience with operating systems, such as Microsoft Windows and Active Directory
  • Intermediate knowledge and experience with authentication and directory services (Active Directory), etc.
  • Intermediate knowledge and experience with Windows system management tools, philosophies, and methods
  • Intermediate knowledge and experience with Windows automation (scripting, imaging, installation, management)
Job Responsibility
Job Responsibility
  • Able to work in a team environment focusing on assigned support service requests keeping customer informed on status
  • Handle daily technical support activities on desktop support, data network and server management and ability to multi-task with several Customers
  • Monitoring, maintaining, supporting and optimizing all networked software and associated operating systems in various customer environments
  • Maintaining network security, business continuity, records and documentation, and vendor relationships
  • Manage Active Directory Domain
  • including WINS, DHCP, IIS, DNS including user accounts/rights/permissions, group policies, etc.
  • Troubleshoot client production issues and analyze failures to minimize outages and interruptions
  • Review and troubleshoot monitored backup failures
  • Perform hardware/software updates /upgrades as required to maintain a secure and healthy network
  • Interface with support vendors to resolve network and system issues
Read More
Arrow Right