Cloud Big-data Engineer Job at PhasorSoft Group (Starkville)

Data Scientist (Big Data) III – Contractor

The Senior Data Scientist (Big Data) will support large-scale data science initi...

Location

United States , Philadelphia

Salary:

Not provided

Robert Half

Expiration Date

Until further notice

Requirements

Strong hands-on experience with PySpark, Python, and R for data analysis and machine learning
Experience working in AWS cloud environments
Proficiency with Databricks for large-scale data processing and analytics
Experience building and supporting large-scale data pipelines and ETL workflows
Demonstrated experience applying statistical and modeling techniques, including: Hypothesis testing, Supervised and unsupervised learning, Forecasting and regression, Dimensionality reduction and clustering
Experience working with relational databases, SQL, and large datasets
Ability to gather, interpret, and translate business requirements into technical solutions
Strong communication skills with the ability to present complex concepts to diverse audiences
Bachelor's degree in Computer Science, Mathematics, Statistics, Engineering, or a related quantitative field, or equivalent practical experience
Typically 5+ years of relevant professional experience in data science, analytics, or related roles

Job Responsibility

Lead complex, cross-functional data science initiatives delivering solutions across multiple technologies and platforms
Design, develop, and deploy data mining, statistical, machine learning, and graph-based algorithms for large-scale data sets
Partner with data engineering teams to ensure proper implementation, performance, and operational use of analytical solutions
Review and assess data science programs and models at an enterprise level to evaluate suitability, performance, and scalability
Build and maintain scalable big-data analytics solutions supporting accurate targeting, forecasting, and advanced insights
Develop and support end-to-end machine learning pipelines, including data preparation, training, testing, validation, and deployment
Establish performance metrics, monitoring, and evaluation procedures for models in production
Translate complex analytical findings into clear, actionable insights for technical and non-technical stakeholders
Provide mentorship and technical guidance to junior team members
Contribute to data strategy, methodology selection, and continuous improvement of analytics capabilities

What we offer

medical
vision
dental
life and disability insurance
401(k) plan

Fulltime

Senior Data Engineer

Microsoft Cloud Operations + Innovation (CO+I) is the engine that powers Microso...

Location

United States , Redmond

Salary:

119800.00 - 234700.00 USD / Year

Microsoft Corporation

Expiration Date

Until further notice

Requirements

Bachelor’s degree in computer science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years’ experience in business analytics, data science, data modeling, or data engineering work
OR master’s degree in computer science, Math, Software Engineering, Computer Engineering, or related field and 3+ years’ experience in business analytics, data science, data modeling, or data engineering work
Ability to meet Microsoft, customer and/or government security screening requirements
8+ years of experience in data engineering with coding and debugging skills in C#, Python, and/or SQL
Deploying solutions in Azure Services & Managing Azure Subscriptions
Understanding and knowledge about big data and writing queries with Kusto/KQL
Understanding and knowledge about extracting data via REST APIs
Strong analytical skills with a systematic and structured approach to software design
5+ years of experience in data science, analytics, or machine learning
4+ years of experience in developing solutions with Microsoft Power Platform, including Power BI, Fabric, Power Automate & M365 Dataverse

Job Responsibility

Apply modification techniques to transform raw data into compatible formats for downstream systems
Utilize software and computing tools to ensure data quality and completeness
Implement code to extract and validate raw data from upstream sources, ensuring accuracy and reliability
Writes efficient, readable, extensible code from scratch that spans multiple features/solutions
Develops technical expertise in proper modeling, coding, and/or debugging techniques such as locating, isolating, and resolving errors and/or defects
Leverages technical proficiency of big-data software engineering concepts, such as Hadoop Ecosystem, Apache Spark, continuous integration and continuous delivery (CI/CD), Docker, Delta Lake, MLflow, AML, and representational state transfer (REST) application programming interface (API) consumption/development
Acquires data necessary for successful completion of the project plan
Proactively detects changes and communicates to senior leaders
Develops usable data sets for modeling purposes
Contributes to ethics and privacy policies related to collecting and preparing data by providing updates and suggestions around internal best practices

Fulltime

Informatica Cloud Data Governance Catalog Specialist

We are looking for an experienced Informatica Cloud Data Governance Catalog Spec...

Location

United States , Torrance

Salary:

Not provided

Robert Half

Expiration Date

Until further notice

Requirements

Proficiency with tools like Informatica Cloud Data Governance Catalog and Cloud Data Quality
Hands-on experience in data modeling, metadata management, and large-scale data analysis
Familiarity with Collibra, Alation, and Glue Data Catalog
Strong understanding of entity-relationship modeling and data security practices
Expertise in business intelligence technologies such as Power BI and Tableau
Exceptional communication and presentation skills to effectively convey technical concepts
Analytical mindset with proven problem-solving abilities
Ability to work collaboratively as part of a team and build strong relationships with stakeholders

Job Responsibility

Create catalog quality reports to monitor and enhance data governance metrics across domains and sub-domains
Develop and showcase data governance dashboards tailored to different user roles, including Data Owners, Stewards, Engineers, and Privacy Officers
Collaborate with business and IT teams, including data stewards, catalog architects, and platform owners, to implement governance solutions
Execute profiling, sampling, and scanner setups using Informatica tools to ensure data quality
Apply expertise in metadata management, data modeling, and large-scale data analysis to support governance initiatives
Design and implement both traditional relational and modern big-data architectures based on organizational requirements
Utilize business intelligence tools such as Power BI and Tableau to create actionable insights and reports
Define compliance procedures and produce audit reports to meet regulatory requirements
Establish and support governance councils and operational frameworks using data catalog tools
Facilitate metadata ingestion and ensure adherence to data security and quality standards

What we offer

medical, vision, dental, and life and disability insurance
eligible to enroll in our company 401(k) plan

Data Engineer

Are you a Data Engineer based in Austin, Texas, who is inspired by working with ...

Location

United States , Austin

Salary:

100000.00 - 135000.00 USD / Year

Beezwax Datatools

Expiration Date

Until further notice

Requirements

4-6 years of hands-on data modeling and data engineering experience
Strong expertise in dimensional modeling and data warehousing
Database design and development experience with relational or MPP databases such as Postgres/ Oracle/ Teradata/ Vertica
Experience in design and development of custom ETL pipelines using SQL and scripting languages (Python/ Shell/ Golang)
Proficiency in advanced SQL and performance tuning
Hands on experience with Big-Data platforms like Spark, Dremio, Hadoop, MapReduce, Hive etc
Experience with Java, Scala and Python
Experience with cloud computing platforms like AWS, Google Cloud
Experience working with APIs
Ability to learn and adapt to new tools and technologies

What we offer

Competitive compensation
Retirement plan with employer matching
Excellent healthcare package with vision and dental
Support for productivity and continued learning in the forms of hardware, software, learning materials, training, and conferences

Fulltime

Senior Data Engineer

Senior Data Engineer – Dublin (Hybrid) Contract Role | 3 Days Onsite. We are see...

Location

Ireland , Dublin

Salary:

Not provided

Solas IT Recruitment

Expiration Date

Until further notice

Requirements

7+ years of experience as a Data Engineer working with distributed data systems
4+ years of deep Snowflake experience, including performance tuning, SQL optimization, and data modelling
Strong hands-on experience with the Hadoop ecosystem: HDFS, Hive, Impala, Spark (PySpark preferred)
Oozie, Airflow, or similar orchestration tools
Proven expertise with PySpark, Spark SQL, and large-scale data processing patterns
Experience with Databricks and Delta Lake (or equivalent big-data platforms)
Strong programming background in Python, Scala, or Java
Experience with cloud services (AWS preferred): S3, Glue, EMR, Redshift, Lambda, Athena, etc.

Job Responsibility

Build, enhance, and maintain large-scale ETL/ELT pipelines using Hadoop ecosystem tools including HDFS, Hive, Impala, and Oozie/Airflow
Develop distributed data processing solutions with PySpark, Spark SQL, Scala, or Python to support complex data transformations
Implement scalable and secure data ingestion frameworks to support both batch and streaming workloads
Work hands-on with Snowflake to design performant data models, optimize queries, and establish solid data governance practices
Collaborate on the migration and modernization of current big-data workloads to cloud-native platforms and Databricks
Tune Hadoop, Spark, and Snowflake systems for performance, storage efficiency, and reliability
Apply best practices in data modelling, partitioning strategies, and job orchestration for large datasets
Integrate metadata management, lineage tracking, and governance standards across the platform
Build automated validation frameworks to ensure accuracy, completeness, and reliability of data pipelines
Develop unit, integration, and end-to-end testing for ETL workflows using Python, Spark, and dbt testing where applicable

Data Engineer / ML Ops

As our Data Engineer, you will design, build, and maintain the data infrastructu...

Location

Germany , Berlin; Potsdam

Salary:

Not provided

Sensmore GmbH

Expiration Date

Until further notice

Requirements

3+ years of hands-on experience building production data pipelines in the cloud (AWS, GCP, or Azure)
Proficiency in Python, SQL, and at least one big-data framework
Familiarity with ML Ops tooling: DVC, MLflow, Kubeflow, or similar
Experience designing and operating data warehouses/data lakes (e.g., Redshift, Snowflake, BigQuery, Delta Lake)
Strong understanding of distributed systems, data serialization (Parquet, Avro), and batch vs. streaming paradigms
Excellent problem-solving skills and the ability to work in ambiguous, fast-paced environments

Job Responsibility

Build & operate data pipelines: Ingest, process, and transform multi-sensor telemetry (radar point-clouds, video frames, log streams) into analytics-ready and ML-ready formats
Design scalable storage: Architect high-throughput, low-latency data lakes and warehouses (e.g., S3, Delta Lake, Redshift/Snowflake)
Enable ML Ops workflows: Integrate DVC or MLflow, automate model training/retraining triggers, track data/model lineage
Ensure data quality: Implement validation, monitoring, and alerting to catch anomalies and schema changes early
Collaborate cross-functionally: Partner with Embedded Systems, Robotics, and Software teams to align on data schemas, APIs, and real-time requirements
Optimize performance: Tune distributed processing, queries, and storage layouts for cost-efficiency and throughput
Document & evangelize: Maintain clear documentation for data schemas, pipeline architectures, and ML Ops practices to uplift the whole team

What we offer

Attractive compensation package and stock options
Beverages on-site and regular social events
Engage with top-tier researchers, engineers, and thought leaders
Influence the future of robotic technologies and tackle significant technological challenges
Assistance with relocation to Berlin

Fulltime

Data Engineer 3

Responsible for designing, building and overseeing the deployment and operation ...

Location

India , Chennai

Salary:

Not provided

Comcast

Expiration Date

Until further notice

Requirements

Bachelor's Degree
5-7 Years Relevant Work Experience
In-depth experience, knowledge and skills in own discipline
Experience in designing, building and overseeing deployment and operation of technology architecture, solutions and software for data
Experience in developing data structures and pipelines
Experience with data acquisition, archive recovery, and database implementation
Experience with on-prem platforms like Kubernetes and Teradata
Experience with Cloud platforms like Databricks, AWS S3, Redshift
Understanding of data lineage and transformation rules
Understanding of data sensitivity and customer data privacy rules and regulations

Job Responsibility

Design, build and oversee deployment and operation of technology architecture, solutions and software to capture, manage, store and utilize structured and unstructured data
Establish and build processes and structures to channel data from multiple inputs
Develop technical tools and programming that leverage AI, machine learning and big-data techniques to cleanse, organize and transform data
Create and establish design standards and assurance processes for software, systems and applications development
Review internal and external business and product requirements for data operations
Work with data modelers/analysts to understand business problems and create or augment data assets
Develop data structures and pipelines to organize, collect, standardize and transform data
Ensure data quality during ingest, processing and final load
Create standard ingestion frameworks for structured and unstructured data
Create standard methods for end users to consume data (database views, extracts, APIs)

What we offer

Paid Time off
Physical Wellbeing benefits
Financial Wellbeing benefits
Emotional Wellbeing benefits
Life Events + Family Support benefits

Fulltime

Senior Cloud Software Engineer

Riverstone Enterprise Solutions, an Envision Innovative Solutions Company, deliv...

Location

United States , Annapolis Junction

Salary:

185000.00 - 215000.00 USD / Year

Riverstone Enterprise Solutions

Expiration Date

Until further notice

Requirements

Eight (8) years’ software engineering experience in programs and contracts of similar scope, type, and complexity is required
two (2) years of which must be in programs utilizing Big- Data Cloud technologies and/or Distributed Computing.
Bachelor’s degree in computer science or related discipline from an accredited college or university is required. Four (4) years of cloud software engineering experience on projects with similar Big- Data systems may be substituted for a bachelor’s degree.
Cloudera Certified Hadoop Developer certification may be substituted for one (1) year of Cloud experience.
Two (2) years of Cloud and Distributed Computing Information Retrieval (IR).
One (1) year of experience with implementing code that interacts with implementation of Cloud Big Table.
One (1) year of experience with implementing code that interacts with implementation of Cloud Distributed File System.
One (1) year of experience with implementing complex MapReduce analytics.
One (1) year of experience with implementing code that interacts with Cloud Distributed Coordination Frameworks.
Object Oriented Design and Programming, Java, Eclipse or similar development environment, MAVEN, RESTful web services

Job Responsibility

The Cloud Software Engineer develops, maintains, and enhances complex and diverse Big-Data Cloud systems based upon documented requirements.
Directly contributes to all stages of back-end processing, analyzing, and indexing. Provides expertise in Cloud Computing, Hadoop Eco-System including implementing Java applications, Distributed Computing, Information Retrieval (IR), and Object-Oriented Design.
Works individually or as part of a team. Reviews and tests software components for adherence to the design requirements and documents test results. Resolves software problem reports.
Utilizes software development and software design methodologies appropriate to the development environment.
Provides specific input to the software components of system design to include hardware/software trade-offs, software reuse, use of Commercial Off-the-shelf (COTS)/Government Off-the-shelf (GOTS) in place of new development, and requirements analysis and synthesis from system level to individual software components.

Fulltime

Select Country

Cloud Big-data Engineer

Job Description

Requirements

Looking for more opportunities?