CrawlJobs Logo
Vaniam Group Logo Vaniam Group · IT - Software Development

Data Engineer, Enterprise Data, Analytics and Innovation

United States 110000.00 - 125000.00 USD / Year · Job Posted December 25, 2025
Apply Position
Job Link Share

Job Description

Are you passionate about building robust data infrastructure and enabling innovation through engineering excellence? As our Data Engineer, your goal is to own and evolve the foundation of our data infrastructure. You will be central in ensuring data reliability, scalability, and accessibility across our lakehouse and transactional systems. This role is ideal for someone who thrives at the intersection of engineering and innovation, ensuring our data platforms are robust today while enabling the products of tomorrow.

Job Responsibility

  • Design, build, and operate reliable ETL and ELT pipelines in Python and SQL
  • Manage ingestion into Bronze, standardization and quality in Silver, and curated serving in Gold layers of our Medallion architecture
  • Maintain ingestion from transactional MySQL systems into Vaniam Core to keep production data flows seamless
  • Implement observability, data quality checks, and lineage tracking to ensure trust in all downstream datasets
  • Develop schemas, tables, and views optimized for analytics, APIs, and product use cases
  • Apply and enforce best practices for security, privacy, compliance, and access control, ensuring data integrity across sensitive healthcare domains
  • Maintain clear and consistent documentation for datasets, pipelines, and operating procedures
  • Lead the integration of third-party datasets, client-provided sources, and new product-generated data into Vaniam Core
  • Partner with product and innovation teams to build repeatable processes for onboarding new data streams
  • Ensure harmonization, normalization, and governance across varied data types (scientific, engagement, operational)
  • Collaborate with the innovation team to prototype and productionize analytics, predictive features, and decision-support tools
  • Support dashboards, APIs, and services that activate insights for internal stakeholders and clients
  • Work closely with Data Science and AI colleagues to ensure engineered pipelines meet modeling and deployment requirements
  • Monitor job execution, storage, and cluster performance, ensuring cost efficiency and uptime
  • Troubleshoot and resolve data issues, proactively addressing bottlenecks
  • Conduct code reviews, enforce standards, and contribute to CI/CD practices for data pipelines

Requirements

  • 5+ years of professional experience in data engineering, ETL, or related roles
  • Strong proficiency in Python and SQL for data engineering
  • Hands-on experience building and maintaining pipelines in a lakehouse or modern data platform
  • Practical understanding of Medallion architectures and layered data design
  • Familiarity with modern data stack tools, including: Spark or PySpark
  • Workflow orchestration (Airflow, dbt, or similar)
  • Testing and observability frameworks
  • Containers (Docker) and Git-based version control
  • Excellent communication skills, problem-solving mindset, and a collaborative approach

Nice to have

  • Experience with Databricks and the Microsoft Azure ecosystem
  • Expertise with Delta Lake formats, metadata management, and data catalogs
  • Familiarity with healthcare, scientific, or engagement data domains
  • Experience exposing analytics through APIs or lightweight microservices

What we offer

  • 100% remote environment with opportunities for local meet-ups
  • Positive, diverse, and supportive culture
  • Passionate about serving clients focused on Cancer and Blood diseases
  • Investment in you with opportunities for professional growth and personal development through Vaniam Group University
  • Health benefits – medical, dental, vision
  • Generous parental leave benefit
  • Focused on your financial future with a 401(k) Plan and company match
  • Work-Life Balance and Flexibility
  • Flexible Time Off policy for rest and relaxation
  • Volunteer Time Off for community involvement
  • Emphasis on Personal Wellness
  • Virtual workout classes
  • Discounts on tickets, events, hotels, child care, groceries, etc.
  • Employee Assistance Programs
Vaniam Group - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Data Engineer, Enterprise Data, Analytics and Innovation

8 matching positions

Principal Data And Analytics Engineer

The Principal Data and Analytics Engineer holds comprehensive responsibility for...
Location
Location
United States
Salary
Salary:
108086.00 - 180144.00 USD / Year
oreillyauto.com Logo
O'Reilly Auto Parts
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven experience architecting enterprise-scale data platforms and ecosystems, including hybrid and cloud-native environments (e.g., GCP BigQuery, Snowflake, Iceberg, Advanced SQL, Erwin, dbt, Kafka, Alation, Collibra)
  • Deep expertise in designing and scaling highly available, secure, and fault-tolerant batch and streaming pipelines with strong emphasis on cost optimization, observability, and latency control
  • Advanced proficiency in semantic modeling, reusable data asset design, and cross-functional data product delivery aligned to medallion architecture
  • Leadership in implementing CI/CD-enabled pipelines, RBAC frameworks, schema evolution strategies, and interoperable data exchange using Iceberg or equivalent table formats
  • Ownership of organization-wide metrics store and semantic layers, ensuring consistency, governance, and performance across reporting, AI, and ML use cases
  • Advanced expertise in programming languages such as Python, Scala, with the ability to architect complex data solutions
  • Demonstrated leadership in designing and overseeing the implementation of scalable, idempotent workflows using orchestration frameworks such as Airflow and Prefect
  • Demonstrated ability to translate business transformation goals into scalable data solutions and reusable patterns
  • Deep understanding of business processes, KPIs, and capability maps across functions such as supply chain, customer, store ops, and finance
  • Proven experience in driving cross-functional data product prioritization, influencing senior stakeholders, and quantifying impact of data initiatives
Job Responsibility
Job Responsibility
  • Help define and evolve enterprise data engineering blueprints, including data mesh, medallion architecture, and hybrid cloud data platforms
  • Set strategic direction for data platforms, tools, and services (e.g., Snowflake, GCP BigQuery, dbt, Kafka, Airflow/Prefect) in alignment with future-state architecture and business priorities
  • Architect and design highly scalable, resilient, cost optimal and secure data platforms
  • Lead the design and implementation of next-generation data platforms, ensuring fault tolerance, high availability, and optimal performance for petabyte-scale data
  • Establish and enforce organization-wide best practices for data pipeline development, CI/CD for data workflows, automated deployment playbooks, and robust rollback strategies
  • Lead technology evaluation and adoption, proactively researching, evaluating, and championing the integration of cutting-edge data technologies, frameworks, and methodologies
  • Define and scale enterprise knowledge management frameworks that ensure consistent documentation, discoverability, and reusability of data assets across domains
  • Establish and govern standards for metadata management, data lineage, architectural diagrams, and runbooks
  • Lead the design of federated governance models that empower domain-aligned teams to operate autonomously while conforming to centralized policies, frameworks and playbooks
  • Collaborate with data governance, compliance, and security teams to operationalize policy-as-code frameworks for data retention, access control, and PII handling
What we offer
What we offer
  • Competitive Wages & Paid Time Off
  • Stock Purchase Plan & 401k with Employer Contributions Starting Day One
  • Medical, Dental, & Vision Insurance with Optional Flexible Spending Account (FSA)
  • Team Member Health/Wellbeing Programs
  • Tuition Educational Assistance Programs
  • Opportunities for Career Growth
  • Fulltime
Read More
Arrow Right

Ai Technology And Innovation Engineer

The AI Solutions Engineer is an advanced subject matter expert, responsible for ...
Location
Location
South Africa , Johannesburg
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Advanced understanding of artificial intelligence, natural language processing (NLP), and machine learning principles
  • Advanced expertise in selecting, fine-tuning, and deploying large and small language models (LLMs/SLMs), such as OpenAI’s GPT series and open-source alternatives
  • Advanced proven experience with prompt engineering, prompt optimization, and AI model reliability and accuracy improvements
  • Advanced proficiency in Python programming, essential for rapid prototyping, integration, and model implementation. Python is the preferred language for AI
  • strong proficiency in Python is essential due to the extensive use of frameworks, libraries, and models
  • Advanced knowledge of additional programming languages (optional, but valuable): JavaScript / TypeScript: Helpful if building frontend interfaces or web integrations
  • Java / C#: Beneficial for integrations with enterprise backend systems (e.g., ERP, CRM)
  • Advanced familiarity with full-stack software development, including frontend and backend integration, user experience considerations, and system interoperability
  • Robust knowledge of data pipeline development, data engineering concepts, and handling of structured and unstructured data
  • Advanced proficiency in cloud computing platforms (Azure, AWS, GCP), particularly in deploying, scaling, and managing AI workloads
Job Responsibility
Job Responsibility
  • Develop, fine-tune, and deploy AI models, including large language models (LLMs) such as GPT-4 or open-source equivalents
  • Design and implement effective prompt engineering strategies and optimizations to enhance AI accuracy, consistency, and reliability
  • Engage with internal stakeholders and clients to understand business needs, translating them into actionable AI solutions
  • Rapidly prototype, test, and iterate AI applications using advanced Python programming and relevant frameworks
  • Integrate AI solutions securely with existing enterprise systems (CRM, ERP, HRIS, finance platforms, collaboration software) via API development and integration
  • Build, maintain, and optimize end-to-end data pipelines to ensure accurate and timely data delivery for AI models
  • Manage structured and unstructured datasets, leveraging vector databases and semantic search to enhance knowledge management capabilities
  • Deploy, manage, and scale AI solutions within cloud computing environments (Azure, AWS, GCP), ensuring high availability, performance, and cost efficiency
  • Implement DevOps and MLOps practices, including automated deployment, testing, monitoring, and version control, to efficiently manage the AI model lifecycle
  • Ensure AI solutions adhere to industry standards and compliance regulations (GDPR, HIPAA), emphasizing security and privacy best practices
  • Fulltime
Read More
Arrow Right

Senior Data and Application Engineer

The Senior Data and Application Engineer will participate and provide engineerin...
Location
Location
United States , St. Inigoes
Salary
Salary:
150000.00 - 225000.00 USD / Year
kairosinc.net Logo
KAIROS Inc
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Expert level experience in DoD NIPR, SIPR, and/or JWICS platform engineering processes
  • Databricks, Foundry, Qlik, Tableau, Python, SQL, PySpark, Databricks, and other data, software, and application development capabilities
  • Excellent project management skills, with the ability to manage cross-functional teams
  • Strong communication and interpersonal skills, capable of leading technical discussions and driving alignment across teams
  • Strong analytical and problem-solving skills, with the ability to diagnose and resolve complex technical issues in a fast-paced environment
  • Strong customer relations, analytics, documentation skills
  • Self-starter, highly motivated, strong work ethic with a commitment to quality
  • Microsoft office suite proficiency, i.e., Word, Excel, PowerPoint
  • Ability to work within a challenging, fast-paced, team-oriented environment
  • Ability to work independently
Job Responsibility
Job Responsibility
  • Data and application engineering using existing enterprise architecture: Provide data engineering, data automation, and automated data mapping capabilities that will be used to power a series of software applications
  • Provide application engineering support necessary to deliver enterprise applications and required data analytics to achieve customer decision advantage
  • Deliver secure, scalable, and modular platform architecture that is optimized for DoD enterprise data automation, AI model deployment, and continuous feature updates across a global network of priority data platforms and/or customer-owned systems
  • Drive process standardization and platform improvements based on data analytics, performance metrics, and industry best practices
  • Technology Leadership: Lead multi-disciplinary hardware, software, AI, and data engineering teams focused on delivering capabilities and features described on the latest KAIROS EPAD technology roadmap
  • Recommend and implement cutting-edge technologies and methodologies to improve KAIROS data automation processes and platform capabilities
  • Process Optimization and Automation: Identify areas for process improvement, focusing on automation powered by optimized software applications and data automation capabilities across all KAIROS technology stack components
  • Continuously innovate new software application data engineering, and AI capabilities focused on delivering a seamless, secure, scalable, and cost-effective suite of KAIROS software and data automation products
  • Cross-Functional Collaboration: Collaborate with engineering, manufacturing, and product teams to ensure successful design and implementation of enterprise platform and data-automation solutions across various applications
  • Work closely with supply chain and operations teams to ensure material availability, cost efficiency, and process sustainability
What we offer
What we offer
  • Medical Coverage
  • Employer Paid Dental, Vision, Basic Life/AD&D, Short-Term/Long-Term Insurance
  • Health Savings Account with Contribution by Employer
  • 401K Plan with Employer Matching
  • Annual Discretionary Bonuses
  • Paid Time Off
  • Eleven (11) Paid Holidays
  • Certification reimbursement program
  • Tuition Reimbursement Program
  • Paid Parental Leave
  • Fulltime
Read More
Arrow Right

Principal Engineer, AI Strategy and Innovation

Shape the architecture and execution of CLEAR’s AI platform strategy, from infra...
Location
Location
United States , New York
Salary
Salary:
250000.00 - 290000.00 USD / Year
clearme.com Logo
Clear
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 12+ years in software engineering and/or technical experience with deep expertise in AI systems, ML platforms, and data infrastructure
  • At least 5 years of experience with various AI technologies including GenAI, ML, Deep Learning, RPA or others
  • Proven ability to scale AI capabilities into high-throughput, low-latency environments
  • Strong technical background in cloud-native architectures (AWS or similar) and modern AI/ML stacks (TensorFlow/PyTorch, MLflow, RAG, MCP, etc.)
  • Experience leading AI strategy and platform adoption in enterprise-scale environments
  • Skilled at translating regulatory and compliance requirements into responsible AI practices
  • Track record of partnering closely with Product, Engineering, Analytics, and Security teams as well as business executives
  • Excellent communicator who can set a vision for AI, explain technical trade-offs, and influence executives, peers, and partners
  • Passionate about embedding AI into core products to deliver measurable impact for members and enterprise partners
Job Responsibility
Job Responsibility
  • Define and scale CLEAR’s AI strategy: spanning data pipelines, ML lifecycle management, and intelligent applications
  • Lead engineering execution for AI models (development, deployment, monitoring, retraining) with a focus on reliability, observability, and ethical AI practices
  • Modernize analytics and intelligence systems to deliver predictive insights and partner-facing transparency in real time
  • Operationalize trust in AI by embedding privacy, compliance, and security into all platforms and workflows
  • Influence cross-functional stakeholders across the business, fostering a culture of technical rigor, collaboration, and innovation, advising C Suite executives, leaders, and individual contributors
  • Lead the AI Governance group and drive best practices across business functions
  • Track and optimize KPIs on AI adoption, model performance, scalability, and business impact
What we offer
What we offer
  • Comprehensive healthcare plans
  • Family-building benefits (fertility and adoption/surrogacy support)
  • Flexible time off
  • Annual wellness stipend
  • Free OneMedical memberships for you and your dependents
  • A CLEAR Plus membership
  • A 401(k) retirement plan with employer match
  • Catered lunches every day
  • Fully stocked kitchens
  • Stipends and reimbursement programs for well-being and learning & development
  • Fulltime
Read More
Arrow Right

Data Engineer Lead (OT Data)

Data Engineer (OT Data) (Category - Engineer) Sector: Oil and Gas Location: Doha...
Location
Location
Qatar , Doha
Salary
Salary:
Not provided
Codvo AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's in engineering, Information Systems, or a related quantitative field
  • 5+ years of proven experience in a data engineering role
  • Experience within oil and gas industry is highly preferred
  • Demonstrable experience building and operationalizing large-scale data pipelines and applications
Job Responsibility
Job Responsibility
  • Architect & Build Data Pipelines: Design, construct, install, test, and maintain highly scalable data management systems and ETL/ELT pipelines
  • Integrate Diverse Data Sources: Develop processes to ingest and integrate high-volume, high-velocity data from SCADA systems, historians (like OSIsoft PI, Aspen InfoPlus.21), DCS, PLC, and IoT sensors
  • Cloud Data Platform Development: Implement and manage data solutions on the Microsoft Azure cloud platform, Leveraging services like Azure IoT Hub, Azure Event Hubs, and Azure Stream Analytics for real-time ingestion and processing of operational technology (OT) data
  • Data Modelling & Warehousing: Design and implement data models optimized for time-series data from industrial assets, supporting operational dashboards and real-time analytics
  • Enable Advanced AI: Build the data infrastructure to support AI/ML models for predictive maintenance, operational anomaly detection, and process optimization using real-time OT data
  • Champion Master Data Management (MDM): Design and implement MDM strategies and solutions to create a single, authoritative source of truth for critical data domains such as wells, equipment, and assets, ensuring data consistency across the enterprise
  • Ensure Data Quality & Governance: Implement robust data quality checks, validation rules, and monitoring to ensure the accuracy, consistency, and reliability of our data. Adhere to and help shape our data governance policies
  • Embrace Industry Standards: Champion and implement industry-specific data standards and models, such as the OSDU™ Data Platform, to ensure interoperability and a unified data view across the upstream lifecycle
  • Collaborate & Innovate: Work closely with a cross-functional team of geoscientists, drilling engineers, data scientists, and business analysts to understand their data needs and deliver effective solutions
  • Automate & Optimize: Identify opportunities for process automation and infrastructure optimization to improve data delivery, scalability, and cost-effectiveness
  • Fulltime
Read More
Arrow Right

Lead Data Engineer

Rapid7 is seeking a Data Engineer, Data Engineering & Analytics to join a high-p...
Location
Location
India , Pune
Salary
Salary:
Not provided
rapid7.com Logo
Rapid7
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of experience in data engineering, analytics, or business intelligence
  • 8+ years experience designing, implementing, operating, and extending enterprise dimensional data models
  • 3+ years experience building reports and dashboards in Tableau and/or other similar data visualization tools
  • Experience in DBT modeling and understanding modular, performant models
  • Solid understanding of Snowflake, SQL, and data warehouse management
  • Understanding of ETL/ELT processes, data pipelines, and cloud-based data architectures
  • Familiarity with modern data stacks (DBT, Airflow, Fivetran, Matillion, or similar tools)
  • Hands-on experience building and operating cloud infrastructure on AWS
  • Experience managing software installations, upgrades, and configuration in production environments
  • Ability to manage data governance, security, and compliance requirements (SOC 2, GDPR, etc.)
Job Responsibility
Job Responsibility
  • Implement data modeling best practices to enhance data accessibility and reporting capabilities
  • Ensure data integrity, security, and compliance with industry standards and regulations
  • Document plans and results in user-stories, issues, PRs, the team’s handbook - following the tradition of documentation first
  • Implement the Corp Data philosophy in everything you do
  • Craft code that meets our internal standards for style, maintainability, and best practices for a high-scale database environment
  • Maintain and advocate for these standards through code review
  • Collaborate with IT and DevOps teams to optimize cloud infrastructure and data governance policies
  • Manage and enhance the existing Tableau reporting suite, ensuring self-service analytics and actionable insights for stakeholders
  • Design, develop, and extend DBT code repository to extend the Enterprise Dimensional Warehouse capabilities and infrastructure
  • Develop and maintain a single source of truth for business metrics, ensuring consistency across reporting platforms
  • Fulltime
Read More
Arrow Right

Director, Product Management - Enterprise Data

Product Management at Capital One is a booming, vibrant craft that requires reim...
Location
Location
United States , San Francisco, California; McLean, Virginia
Salary
Salary:
230400.00 - 286900.00 USD / Year
capitalone.com Logo
Capital One
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • At least 7 years of experience working in Product Management
  • A Bachelor's Degree in a quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Computer Science, Computer Engineering, Software Engineering, Mechanical Engineering, Information Systems or a related quantitative field)
  • A Master's Degree in a quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Computer Science, Computer Engineering, Software Engineering, Mechanical Engineering, Information Systems or a related quantitative field) or an MBA with a quantitative concentration
Job Responsibility
Job Responsibility
  • Human Centered - Obsesses about internal and external customer needs to reimagine and innovate product solutions
  • Business Focused -Delivers game-changing outcomes by focusing on leverage and execution excellence
  • Technology Driven -Leverages technology to deliver innovative and resilient solutions that enable both near term and long term value
  • Integrated Problem Solving - Identifies and resolves complex problems to deliver outcomes while mitigating product risks
  • Transformational Leadership - Leads cross functional teams to solve customer problems and drive organizational alignment
What we offer
What we offer
  • performance based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI)
  • comprehensive, competitive, and inclusive set of health, financial and other benefits
  • Fulltime
Read More
Arrow Right

Lead Data Engineer

Rapid7 is seeking a Data Engineer, Data Engineering & Analytics to join a high-p...
Location
Location
India , Pune
Salary
Salary:
Not provided
rapid7.com Logo
Rapid7
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Ability to thrive in a fast-paced hybrid organization
  • Comfort working in a highly agile, intensely iterative environment
  • Demonstrated capacity to clearly and concisely communicate complex business activities, technical requirements, and recommendations
  • 8+ years of experience in data engineering, analytics, or business intelligence
  • 8+ years experience designing, implementing, operating, and extending enterprise dimensional data models
  • 3+ years experience building reports and dashboards in Tableau and/or other similar data visualization tools
  • Experience in DBT modeling and understanding modular, performant models
  • Solid understanding of Snowflake, SQL, and data warehouse management
  • Understanding of ETL/ELT processes, data pipelines, and cloud-based data architectures
  • Familiarity with modern data stacks (DBT, Airflow, Fivetran, Matillion, or similar tools)
Job Responsibility
Job Responsibility
  • Implement data modeling best practices to enhance data accessibility and reporting capabilities
  • Ensure data integrity, security, and compliance with industry standards and regulations
  • Document plans and results in user-stories, issues, PRs, the team’s handbook - following the tradition of documentation first
  • Implement the Corp Data philosophy in everything you do
  • Craft code that meets our internal standards for style, maintainability, and best practices for a high-scale database environment
  • Maintain and advocate for these standards through code review
  • Collaborate with IT and DevOps teams to optimize cloud infrastructure and data governance policies
  • Manage and enhance the existing Tableau reporting suite, ensuring self-service analytics and actionable insights for stakeholders
  • Design, develop, and extend DBT code repository to extend the Enterprise Dimensional Warehouse capabilities and infrastructure
  • Develop and maintain a single source of truth for business metrics, ensuring consistency across reporting platforms
  • Fulltime
Read More
Arrow Right