CrawlJobs Logo
Randstad Logo Randstad · IT - Software Development

Data Engineer – Cloud and Analytics Platforms

Canada, Calgary · Job Posted June 16, 2026
Apply Position
Job Link Share

Job Description

As a Data Engineer, you will be the foundational backbone of our data analytics delivery stream. You will design and implement high-performance data pipelines capable of handling both real-time streaming and batch processing. Beyond writing complex ETL/ELT logic and tuning SQL queries for maximum efficiency, you will collaborate with Data Scientists, ML Developers, and Cloud Architects to evolve our data platform environment. This role is perfect for a developer with a data-backend focus who champions clean code, automated data quality tracking, and scalable cloud architecture.

Job Responsibility

  • Build, improve, and maintain data infrastructure for ingesting, storing, and transforming data across advanced analytical workflows
  • Implement robust, scalable data pipelines supporting both stream and batch processing
  • Perform end-to-end ETL/ELT processes to extract and load data from a wide variety of structured and unstructured sources
  • Partner with enterprise architects and Information Services (IS) teams to design and evolve the overarching data platform environment in AWS
  • Write complex, automated SQL queries to manipulate data extracts and manage large-scale data flows
  • Perform continuous query optimization, performance analysis, and database tuning to minimize processing latency and cost
  • Assist with logical and physical data modeling, implementing optimized data structures (e.g., star and snowflake schemas)
  • Automate cloud deployments, data flows, and automated data quality validation checks
  • Collaborate closely with Data Scientists and ML developers to build, test, and operationalize automated ML pipelines
  • Design rigorous testing processes, formulating and executing comprehensive test cases for data validation
  • Conduct thorough code reviews, troubleshoot system defects, and provide constructive feedback to elevate code quality across the distributed engineering team

Requirements

  • Minimum of 3+ years in a dedicated Data Engineering role or a Software Developer role with a strong focus on data-backend development and complex transformations
  • Excellent, production-level knowledge of Python
  • Strong knowledge of SQL paired with a deep understanding of performance analysis and query optimization techniques
  • Rigid grasp of computer science fundamentals, including modularity, abstraction, data structures, and algorithms
  • Solid understanding of core data modeling concepts (normalized vs. denormalized data architectures, conceptual/logical/physical models, and star/snowflake schemas)
  • Proven experience building automated, production-grade ETL/ELT pipelines
  • Solid project experience leveraging various data storage technologies (including RDBMS, NoSQL, and Graph Databases)
  • Direct experience with cloud infrastructure provisioning and deployment automation on AWS

Nice to have

  • BSc. degree in Computer Science, Engineering, Mathematics, Physics, Statistics, or an equivalent quantitative field
  • In-depth knowledge of the broader AWS ecosystem combined with exposure to Microsoft Fabric
  • Experience working within fast-paced Agile development methodologies
Randstad - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Data Engineer – Cloud and Analytics Platforms

8 matching positions

Cloud Solution Architect (data platforms and analytics)

We’re looking for a Cloud Solution Architect (CSA) who specializes in data platf...
Location
Location
South Korea , Seoul
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science, Information Technology, Engineering, Business, or related field AND 7+ years experience in cloud/infrastructure technologies, information technology (IT) consulting/support, systems administration, network operations, software development/support, technology solutions, practice development, architecture, and/or consulting OR equivalent experience
  • Fluent Korean language skills to interact with local enterprise customers
  • English skills to communicate with Global Customer Success and Product Group
Job Responsibility
Job Responsibility
  • Engage with customer IT Executives and business leaders to understand their data estate, priorities, and success measures, and design secure, scalable Data & AI solutions that deliver measurable business value
  • Lead architecture design sessions, develop Data & Analytics roadmaps, and drive Proof of Concepts (POCs) and Minimum Viable Products (MVPs) to accelerate adoption and ensure long-term technical viability
  • Own the end-to-end technical delivery results, ensuring completeness and accuracy of consumption and customer success plans in collaboration with the CSAM
  • Deliver repeatable intellectual property (IP) to achieve targeted outcomes, accelerate Azure Consumed Revenue (ACR), and contribute to centralized IP development initiatives
  • Provide delivery oversight and escalation support for key Factory engagements across Data & Analytics workloads
  • Drive technical excellence by leading the health, resiliency, security, and optimization of mission-critical data workloads, ensuring readiness for production-scale AI use cases
  • Identify and resolve technical blockers, share customer feedback with engineering teams, and influence product improvements through Voice of the Customer insights
  • Maintain deep technical expertise and stay current with market trends, competitive insights, and Microsoft’s evolving data and AI capabilities
  • Be accredited and certified to deliver with advanced and expert-level proficiency in priority workloads including Microsoft Fabric, Azure Databricks, Microsoft Purview, Azure SQL, PostgreSQL, MySQL, and Cosmos DB
  • Demonstrate a growth mindset by continuously aligning your skills to team and customer needs, contributing to technical communities, and mentoring others to accelerate customer outcomes
  • Fulltime
Read More
Arrow Right

Cloud Solution Architect and Senior Cloud Solution Architect - Data and AI

We are looking for Cloud Solution Architect (CSA) and Senior Cloud Solution Arch...
Location
Location
United States , Multiple Locations
Salary
Salary:
85100.00 - 169800.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science, Information Technology, Engineering, Business, Liberal Arts, or related field AND 2+ years experience in cloud/infrastructure technologies, information technology (IT) consulting/support, systems administration, network operations, software development/support, technology solutions, practice development, architecture, and/or consulting OR equivalent experience
  • Authorization to work in the United States that does not now or in the future require employer sponsorship
  • 1+ years of experience with AI/ML and/or Generative AI technology
  • 2+ years of customer facing experience providing recommendations to, or collaborating with, mid-to-senior level executives to address and advance technology transformation initiatives, entablements, and outcomes, including Data and AI solutions
  • 2+ years of enterprise experience in ANY of the following: Microsoft Fabric, Azure Databricks, Microsoft Purview, Azure SQL, PostgreSQL, MySQL, and Cosmos DB
Job Responsibility
Job Responsibility
  • Seek to understand customers’ overall data estate Business and IT priorities and success measures to design Data & Analytics solutions that drive business value and drive positive Customer Satisfaction & become a trusted advisor
  • Ensure that solution exhibits high levels of performance, security, scalability, maintainability, repeatability, appropriate reusability, and reliability upon deployment and provide feedback and insights from customers/partners
  • Develop opportunities to drive Customer Success business results & help Customers get value from their Microsoft investments and identify resolutions to Customer blockers by leveraging SA subject matter expertise
  • Deliver according to MS best practices & using repeatable Intellectual Property (IP)
  • Apply technical knowledge to architect and design solutions that meet business and IT needs, create AI roadmaps, drive Proof of Concepts (POC) and Minimal Viable Product (MVP), and ensure long term technical viability of new deployments, infusing key AI technologies where appropriate
  • Be the Voice of Customer to share insights and best practices, connect with Engineering team to remove key blockers and drive product improvements
  • Maintain technical skills and knowledge, keep up to date with market trends and competitive insights
  • collaborate and share with the AI technical community while educating customers on Azure platform
  • Accelerate customer outcomes - Share expertise, contribute to IP creation & re-use to accelerate customer outcomes and obtain relevant accreditations and certifications
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Data Platforms and Services - Minecraft

At Mojang Studios, the creators of Minecraft, we are on a mission to build a bet...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 3+ years experience in business analytics, data science, software development, data modeling, or data engineering OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years experience in business analytics, data science, software development, data modeling, or data engineering OR equivalent experience.
Job Responsibility
Job Responsibility
  • Build and operate scalable data services and APIs powering personalization, analytics, and player insights across Mojang
  • Own end-to-end cloud infrastructure (Azure + Terraform + CI/CD) for reliable, repeatable deployments across environments
  • Ensure production reliability (ICM/DRI ownership, monitoring, incident response) for critical data systems and services
  • Drive performance, cost efficiency, and security compliance across data pipelines, services, and infrastructure
  • Build and maintain shared C++ telemetry libraries used across gameplay systems and services.
  • Fulltime
Read More
Arrow Right

Principal Data And Analytics Engineer

The Principal Data and Analytics Engineer holds comprehensive responsibility for...
Location
Location
United States
Salary
Salary:
108086.00 - 180144.00 USD / Year
oreillyauto.com Logo
O'Reilly Auto Parts
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven experience architecting enterprise-scale data platforms and ecosystems, including hybrid and cloud-native environments (e.g., GCP BigQuery, Snowflake, Iceberg, Advanced SQL, Erwin, dbt, Kafka, Alation, Collibra)
  • Deep expertise in designing and scaling highly available, secure, and fault-tolerant batch and streaming pipelines with strong emphasis on cost optimization, observability, and latency control
  • Advanced proficiency in semantic modeling, reusable data asset design, and cross-functional data product delivery aligned to medallion architecture
  • Leadership in implementing CI/CD-enabled pipelines, RBAC frameworks, schema evolution strategies, and interoperable data exchange using Iceberg or equivalent table formats
  • Ownership of organization-wide metrics store and semantic layers, ensuring consistency, governance, and performance across reporting, AI, and ML use cases
  • Advanced expertise in programming languages such as Python, Scala, with the ability to architect complex data solutions
  • Demonstrated leadership in designing and overseeing the implementation of scalable, idempotent workflows using orchestration frameworks such as Airflow and Prefect
  • Demonstrated ability to translate business transformation goals into scalable data solutions and reusable patterns
  • Deep understanding of business processes, KPIs, and capability maps across functions such as supply chain, customer, store ops, and finance
  • Proven experience in driving cross-functional data product prioritization, influencing senior stakeholders, and quantifying impact of data initiatives
Job Responsibility
Job Responsibility
  • Help define and evolve enterprise data engineering blueprints, including data mesh, medallion architecture, and hybrid cloud data platforms
  • Set strategic direction for data platforms, tools, and services (e.g., Snowflake, GCP BigQuery, dbt, Kafka, Airflow/Prefect) in alignment with future-state architecture and business priorities
  • Architect and design highly scalable, resilient, cost optimal and secure data platforms
  • Lead the design and implementation of next-generation data platforms, ensuring fault tolerance, high availability, and optimal performance for petabyte-scale data
  • Establish and enforce organization-wide best practices for data pipeline development, CI/CD for data workflows, automated deployment playbooks, and robust rollback strategies
  • Lead technology evaluation and adoption, proactively researching, evaluating, and championing the integration of cutting-edge data technologies, frameworks, and methodologies
  • Define and scale enterprise knowledge management frameworks that ensure consistent documentation, discoverability, and reusability of data assets across domains
  • Establish and govern standards for metadata management, data lineage, architectural diagrams, and runbooks
  • Lead the design of federated governance models that empower domain-aligned teams to operate autonomously while conforming to centralized policies, frameworks and playbooks
  • Collaborate with data governance, compliance, and security teams to operationalize policy-as-code frameworks for data retention, access control, and PII handling
What we offer
What we offer
  • Competitive Wages & Paid Time Off
  • Stock Purchase Plan & 401k with Employer Contributions Starting Day One
  • Medical, Dental, & Vision Insurance with Optional Flexible Spending Account (FSA)
  • Team Member Health/Wellbeing Programs
  • Tuition Educational Assistance Programs
  • Opportunities for Career Growth
  • Fulltime
Read More
Arrow Right

Senior Data Engineer - AI and Analytics

We're building a world of health around every individual — shaping a more connec...
Location
Location
United States , Buffalo Grove
Salary
Salary:
101970.00 - 203940.00 USD / Year
https://www.cvshealth.com/ Logo
CVS Health
Expiration Date
June 24, 2026
Flip Icon
Requirements
Requirements
  • 3-5+ years of experience with SQL, NoSQL
  • 3-5+ years of experience with Python
  • 3+ years of experience with Data warehouses (such as data modeling and technical architectures) and infrastructure components
  • 3+ years of experience with ETL/ELT, and building high-volume data pipelines
  • 3+ years of experience with reporting/analytic tools
  • 3+ years of experience with Query optimization, data structures, transformation, metadata, dependency, and workload management
  • 3+ years of experience with Big data and cloud architecture
  • 3+ years of hands-on experience building modern data pipelines within a major cloud platform (preferably GCP, open to AWS or Azure)
  • 3+ years of experience with deployment/scaling of apps on containerized environment (i.e. Kubernetes, AKS)
  • 3+ years of experience with real-time and streaming technology (i.e. Azure Event Hubs, Azure Functions, Kafka, Spark Streaming)
Job Responsibility
Job Responsibility
  • Design, develop, and maintain optimal data pipelines to assemble large and intricate datasets
  • Cater to the business requirements of various CVS lines of business
  • Collaborate closely with teams to craft tools to provide actionable insights and integrate them with consumer touchpoints
  • Solve problems associated with large scale complex, structured and unstructured data
What we offer
What we offer
  • Medical, dental, and vision coverage
  • paid time off
  • retirement savings options
  • wellness programs
  • bonus, commission or short-term incentive program
  • Fulltime
!
Read More
Arrow Right

Cloud Solution Architect - Data Platforms & Analytics

We are looking for a Cloud Solution Architect (CSA) who specializes in data plat...
Location
Location
Norway , Oslo
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science, Information Technology, Engineering, Business or related field AND a couple years experience in cloud/infrastructure technologies, information technology (IT) consulting/support, systems administration, network operations, software development/support, technology solutions, practice development, architecture, and/or consulting OR equivalent experience
  • Data platforms and analytics experience
  • Norwegian and English Fluency
Job Responsibility
Job Responsibility
  • Build and maintain trusted relationships with key customer IT and business stakeholders, developing a deep understanding of their data landscape, strategic priorities, and success criteria
  • Lead the design of secure, scalable Data & AI solutions—primarily leveraging Power BI within Microsoft Fabric—while progressively broadening your influence across areas such as Azure Databricks, databases, data governance, and AI
  • Take a leading role in the design and implementation of Power BI solutions, contributing expertise across complementary data technologies including Azure Databricks, databases, and governance frameworks
  • Contribute to and at times guide architecture discussions, helping shape Data & Analytics roadmaps that align technology decisions with business objectives
  • Drive the execution of Proof of Concepts (POCs) and Minimum Viable Products (MVPs) to accelerate customer adoption and demonstrate the value of Power BI and Microsoft Fabric
  • Partner closely with internal teams to ensure successful delivery outcomes, proactively managing risks and maintaining high customer satisfaction
  • Promote best practices and reusable assets across engagements to maximize customer value and improve repeatability
  • Oversee and continuously refine Power BI solutions to ensure they meet performance, reliability, and governance standards
  • Lead troubleshooting efforts for complex issues that could impact go-live, supporting technical conversations and enabling clear decision-making
  • Stay ahead of updates across Power BI and Microsoft’s broader Data & AI ecosystem, integrating new capabilities into customer solutions where relevant
  • Fulltime
Read More
Arrow Right

Cloud Solution Architect (CSA) - Data Platforms & Analytics

On this team, you will help some of Canada’s most complex and highimpact organiz...
Location
Location
Canada , Ottawa
Salary
Salary:
96000.00 - 177600.00 CAD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science, Information Technology, Engineering, Business, or related field AND 4+ years experience in cloud/infrastructure technologies, information technology (IT) consulting/support, systems administration, network operations, software development/support, technology solutions, practice development, architecture, and/or consulting
  • OR equivalent experience
  • 4+ years experience working in a customer-facing role (e.g., internal and/or external)
  • 4+ years experience working on technical projects
  • Technical Certification in Cloud (e.g., Azure, Amazon Web Services, Google, security certifications)
  • Breadth of technical experience and knowledge in foundational Cloud Security and Infrastructure, Cloud Architecture Design, with Depth / Subject Matter Expertise in Azure data services such as Microsoft Fabric, Azure Databricks, Azure SQL, Synapse Analytics, Cloud Scale Analytics, Data Factory, and Cosmos DB
  • Experience with cloud-based data migration and modernization strategies across relational and non-relational databases
  • Strong understanding of data engineering and experience in data modeling, ETL/ELT pipelines, performance optimization, Business Intelligence and real-time analytics
  • Deep experience in data governance, security, and compliance frameworks
  • Hands-on experience with big data technologies and distributed computing
Job Responsibility
Job Responsibility
  • Engage with customer IT and business leaders to understand their data estate, priorities, and success measures, and design secure, scalable Data & AI solutions that deliver measurable business value
  • Lead architecture design sessions, develop Data & Analytics roadmaps, and drive Proof of Concepts (POCs) and Minimum Viable Products (MVPs) to accelerate adoption and ensure long-term technical viability
  • Own the end-to-end technical delivery results, ensuring completeness and accuracy of consumption and customer success plans in collaboration with the CSAM
  • Deliver repeatable intellectual property (IP) to achieve targeted outcomes, accelerate Azure Consumed Revenue (ACR), and contribute to centralized IP development initiatives
  • Provide delivery oversight and escalation support for key Factory engagements across Data & Analytics workloads
  • Drive technical excellence by leading the health, resiliency, security, and optimization of mission-critical data workloads, ensuring readiness for production-scale AI use cases
  • Identify and resolve technical blockers, share customer feedback with engineering teams, and influence product improvements through Voice of the Customer insights
  • Maintain deep technical expertise and stay current with market trends, competitive insights, and Microsoft’s evolving data and AI capabilities
  • Be accredited and certified to deliver with advanced and expert-level proficiency in priority workloads including Microsoft Fabric, Azure Databricks, Microsoft Purview, Snowflake Azure SQL, PostgreSQL, MySQL, and Cosmos DB
  • Demonstrate a growth mindset by continuously aligning your skills to team and customer needs, contributing to technical communities, and mentoring others to accelerate customer outcomes
  • Fulltime
Read More
Arrow Right

Data Engineer (Big Data, Cloud - AWS, Databricks) - Assistant Vice President

Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Scala, Spark/Pyspark is must, Hadoop ( BIG Data ), + AWS,Databricks
  • 8 to 11 years’ experience implementing data-intensive solutions using agile methodologies
  • Experience of relational databases and using SQL for data querying, transformation and manipulation
  • Experience of modelling data for analytical consumers
  • Ability to automate and streamline the build, test and deployment of data pipelines
  • Experience in cloud native technologies and patterns
  • A passion for learning new technologies, and a desire for personal growth, through self-study, formal classes, or on-the-job training
  • Excellent communication and problem-solving skills
  • An inclination to mentor
  • an ability to lead and deliver medium sized components independently
Job Responsibility
Job Responsibility
  • Developing and supporting scalable, extensible, and highly available data solutions
  • Deliver on critical business priorities while ensuring alignment with the wider architectural vision
  • Identify and help address potential risks in the data supply chain
  • Follow and contribute to technical standards
  • Design and develop analytical data models
  • Fulltime
Read More
Arrow Right