CrawlJobs Logo

Batch Processing System Architect

United States, Spring 92700.00 - 213500.00 USD / Year · Job Posted March 02, 2026
Apply Position
Job Link Share

Job Description

Researches, designs, develops, configures, integrates, tests and maintains existing and new business applications and/or information systems solutions including databases through integration of technical and business requirements. Applications and infrastructure solutions include both 3rd party software and internally developed applications and infrastructure. Responsibilities include, but are not limited to, analysis of business requirements, coding of modifications or new program, creation of documentation, testing and maintenance of applications, infrastructure, and information systems including database management systems. Works within the Information Technology function, obtaining resources and working in support of objectives and strategies. Provides required documentation and participates in architecture reviews to ensure that the solutions comply with standards and use approved technologies.

Job Responsibility

  • Manage and configure LSF Scheduler/Resource Manager and RTM monitoring, or similar workload management platforms, to optimize ASIC development workflows
  • Install and update ASIC tools from vendors such as Cadence, Synopsys, and Mentor, as well as RSYNC tools and projects from other HPE ASIC lab environments
  • Develop and maintain Python, Bash, and Perl scripts to manage the LSF environment and streamline system operations
  • Collaborate with the ASIC tools team to update and maintain LSF configurations to support evolving engineering workflows
  • Understand and manage Flexera Licensing and license file configurations to ensure proper operation of licensed EDA tools
  • Occasionally assist ASIC teams in investigating and debugging tool-related issues to ensure optimal productivity
  • Work with business units to coordinate system environment events, such as patching or planned downtime, ensuring minimal disruption to engineering workflows
  • Collaborate with business units to ensure the continuity of business services and compliance with operational standards
  • Manage Linux systems at scale
  • Develop and maintain scripts (Ansible, Bash, Python, Perl) for automated deployments, updates, and system configurations across multiple systems
  • Ensure all deployments and updates are thoroughly tested prior to rollout to minimize impact on production environments
  • Manage NFS servers, user file permissions, and associated storage infrastructure
  • Patch Linux environments using COLO/PC-provided scripts (yum, RPMs, and resolving dependencies)
  • Administer Linux user accounts and authentication through LDAP and local service accounts
  • Install and manage certificates for secure system operations
  • Configure and support secure remote access tools, such as NoMachine (NX) and graphical desktop environments (GNOME/KDE/MATE)
  • Familiarity with Linux hardening tools, such as SaltStack or VMware Aria, and ability to work with hardened environments
  • Measure and optimize system performance, including network, storage, and overall resource utilization
  • Create and maintain comprehensive documentation, including operational runbooks and automated processes
  • Collaborate with architecture teams to design best-of-breed hardware solutions for the ASIC environment
  • Use ServiceNow to submit, track, and manage incidents, provisioning requests, and decommissioning tickets
  • Create and manage ServiceNow change requests (CHG) to implement projects such as quarterly patching and infrastructure updates
  • Work with infrastructure and site networking teams to resolve DNS, routing, and firewall issues to ensure seamless system operation
  • Work with infrastructure teams and management to escalate and resolve any environment-related issues impacting performance

Requirements

  • Hands-on experience with LSF Scheduler/Resource Manager and RTM monitoring, or similar platforms
  • Familiarity with ASIC tools such as Cadence, Synopsys, and Mentor
  • Strong scripting skills (Python, Bash, Perl) to manage EDA and workload management environments
  • Understanding of Flexera Licensing and license file management
  • 10-15 years of experience managing RHEL systems (certifications preferred)
  • Proficiency in automation tools such as Ansible
  • Experience managing NFS servers, user file permissions, and Linux system patching
  • Familiarity with SaltStack, VMware Aria, or similar hardening tools
  • Strong troubleshooting skills related to networks, storage, and Linux performance
  • Experience with incident/change management platforms like ServiceNow
  • Ability to coordinate with architecture, networking, and infrastructure teams to resolve system-level issues
  • Familiarity with provisioning and decommissioning processes for Linux systems
  • Strong communication and collaboration skills to work with cross-functional teams
  • Strong documentation skills for runbooks, operational workflows, and automated processes
  • Ability to handle multiple priorities in a fast-paced environment

Nice to have

  • RHEL certifications or equivalent Linux certifications
  • Proven experience managing environments with large-scale deployments of Linux systems and EDA tools
  • Experience with performance tuning for ASIC workflows, storage, and networking
  • Experience with NoMachine (NX) or similar remote desktop applications

What we offer

  • Health & Wellbeing
  • Personal & Professional Development
  • Unconditional Inclusion

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Batch Processing System Architect

8 matching positions

Senior Software Engineer - AI Engineering and Productivity

The Role The AI Engineering and Productivity team in the Global Planning, Desig...
Location
Location
United States , Austin; Warren
Salary
Salary:
Not provided
gm.com Logo
General Motors
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s Degree in Computer Science, Software Engineering, Information Systems, Engineering, or a related field, OR equivalent experience
  • 6+ years of experience delivering enterprise or full stack software solutions using Java / JEE, Python , and preferably Angular
  • 3+ years of experience working with complex SQL queries, functions, and stored procedures, including performance tuning and optimization against large datasets
  • Experience building or supporting data pipelines , ETL/ELT processes, or datacentric applications on distributed or cloud platforms (e.g., Databricks, Spark , or similar)
  • 3+ years of experience with Kubernetes/Docker, Quarkus , and cloud platforms such as Azure, AWS , or GCP
  • Experience working in Agile/SCRUM development methodologies, including backlog refinement, sprint planning, and incremental delivery
  • Hands on experience with modern DevOps practices such as Git/GitHub, code reviews, automated builds, automated testing, and CI/CD pipelines (e.g., GitHub Actions)
  • Willingness and demonstrated ability to learn and apply AI concepts , including working with data and APIs that support AI/ML and LLM based solutions
  • Strong problem solving skills with the ability to break down complex technical and data challenges into clear, actionable steps and deliver high quality solutions
  • Excellent written and verbal communication skills with the ability to collaborate with both technical and nontechnical stakeholders
Job Responsibility
Job Responsibility
  • Design, develop, and maintain data driven and AI-enabled applications and services that support Product Development engineering teams
  • Write high-quality, performant SQL (queries, functions, stored procedures) for complex data transformations and modeling across enterprise data platforms (e.g., SQL Server, Oracle, PostgreSQL)
  • Build and optimize data pipelines and workflows in Databricks (DBX) and related tools to support batch and near realtime data processing
  • Develop backend services and APIs in Java and/or Python that integrate data, business rules, and user workflows into robust, reusable components
  • Develop enterprise grade applications using Kubernetes/Docker, Quarkus, Java, Angular, PostgreSQL, and other GM approved tools
  • Partner with data science and AI teams to productionize AI/ML and LLM based solutions, including feature pipelines, inference integrations, monitoring, and continuous improvement
  • Proactively identify and remediate issues related to code quality, patterns, performance, security, and data correctness, using code quality analysis tools and remediation techniques
  • Lead or contribute to solution design, including architecture, patterns, and technology choices aligned with GM standards and Statement of Technical Direction
  • Apply and champion software engineering best practices, including code reviews, automated testing, branching strategies, CI/CD pipelines (e.g., GitHub Actions), observability, and secure coding practices
  • Collaborate with cross-functional teams (product owners, data engineers, architects, business stakeholders) to refine requirements, define acceptance criteria, and deliver incremental value in an Agile/SCRUM environment
What we offer
What we offer
  • Relocation benefits
  • Fulltime
Read More
Arrow Right
New

Technical Lead – AI & IOT

We are seeking a Technical Lead with strong retail technology experience to driv...
Location
Location
India , Pune
Salary
Salary:
Not provided
Codvo AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8–12+ years in engineering or architecture roles, with at least 3–5 years in retail technology. Hands-on experience with Azure Services (Synapse, IoT Hub, Event Hub, Functions, APIM, AKS, AAD, Blob Storage, Monitor). Strong knowledge of microservices, API design, event-driven systems, DevOps, and cloud security.
  • Understanding of store operations, loss prevention, planogram execution, inventory workflows, demand forecasting, and supply chain processes.
  • Familiarity with AI/ML platforms, CV-based analytics, and streaming data architectures (Event Hub, Kafka, etc.). Experience with RAG/GenAI is a plus.
  • Proven ability to lead technical teams, collaborate with external partners, and deliver complex programs in fast-paced environments.
  • Azure Solutions Architect (AZ-305)(preferred, not mandatory)
  • Azure Developer (AZ-204) or Azure DevOps (AZ-400)(preferred, not mandatory)
  • Relevant retail technology or cloud certifications (preferred, not mandatory)
Job Responsibility
Job Responsibility
  • Own and drive the end-to-end technical execution of cloud, AI, and IoT-based retail modernization initiatives
  • Develop scalable, secure Azure-native architectures and oversee implementation through production rollout
  • Work closely with engineering partners and cross-functional teams to deliver prioritized retail use cases
  • Integrate data across POS, ERP, CRM, inventory, supply chain, and IoT devices using Azure services
  • Implement reliable real-time and batch data pipelines for analytics, monitoring, and automation
  • Ensure security, governance, and observability across environments
  • Convert retail workflows such as shrink reduction, planogram compliance, store operations optimization, footfall analytics, and inventory accuracy into scalable technical solutions
  • Enable data-driven decision making through automation, AI, and analytics
  • Act as the technical owner and primary point of contact for cloud and AI initiatives
  • Manage backlog, technical milestones, architecture reviews, and cross-team coordination
  • Fulltime
Read More
Arrow Right
New

Azure Data Platform Lead

Lead the design, implementation, and optimization of enterprise-scale Azure data...
Location
Location
India
Salary
Salary:
Not provided
Codvo AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7+ years Azure data platform experience (Data Factory, Synapse Analytics, SQL Database, Storage)
  • Expert-level SQL, Python, and experience with Apache Spark (PySpark/Scala)
  • Strong understanding of data architecture patterns, data modeling, and distributed systems
  • Experience with Azure DevOps, ARM templates, and infrastructure automation
  • Knowledge of data governance tools and frameworks (Microsoft Purview preferred)
  • Hands-on experience with modern data stack components and cloud-native architectures
  • RESTful API integration, real-time streaming, and batch processing systems
  • Enterprise data warehouse and analytics platform implementation
  • Performance tuning, cost optimization, and capacity planning experience
  • Proven ability to lead technical teams and mentor junior engineers
Job Responsibility
Job Responsibility
  • Design and implement scalable data platform architectures using Azure services (Synapse, Data Factory, Fabric, SQL Database)
  • Establish data lakehouse and data warehouse solutions with medallion architecture patterns
  • Define data governance frameworks, security protocols, and compliance standards
  • Lead platform evaluations, vendor selections, and technology roadmap planning
  • Architect data ingestion pipelines from diverse sources including databases, APIs, files, and streaming systems
  • Design and optimize ETL/ELT processes using SQL, Python, Spark, and Azure native services
  • Implement data quality frameworks, monitoring, and alerting systems
  • Establish CI/CD pipelines for data workflows and infrastructure as code
  • Design seamless integrations with BI tools, analytics platforms, and downstream systems
  • Optimize performance, cost, and scalability through automated resource management
  • Fulltime
Read More
Arrow Right
New

Enterprise Data Architect

Robert Half's client in Surprise, Arizona is seeking an Enterprise/Lead Data Arc...
Location
Location
United States , Surprise
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7+ years of experience in data architecture, data engineering, or enterprise architecture
  • Proven experience leading end-to-end data discovery and architecture design initiatives
  • Strong background in data modeling, system mapping, and enterprise data strategy
  • Hands-on experience with modern data platforms (e.g., Azure, AWS, GCP, Snowflake, Databricks, SQL Server)
  • Deep understanding of data governance frameworks and best practices
  • Experience working in regulated environments with HIPAA and PCI compliance requirements
  • Strong knowledge of data integration methodologies (ETL/ELT, APIs, middleware tools)
  • Ability to translate business needs into scalable technical architectures
Job Responsibility
Job Responsibility
  • Lead comprehensive discovery of the current data environment, including systems, data flows, and storage platforms
  • Assess existing architecture (data warehouses, lakes, operational systems, cloud/on-prem)
  • Identify gaps, inefficiencies, redundancies, and data silos
  • Evaluate current data quality, accessibility, and scalability
  • Define and design a long-term (5–10 year) data architecture roadmap aligned to business growth
  • Provide recommendations on where data should reside (data lake, warehouse, cloud platforms, etc.)
  • Design scalable, high-performance data solutions to support enterprise reporting, analytics, and future AI/ML initiatives
  • Advise on total data volume growth and infrastructure planning (full-scale/data capacity planning)
  • Map data flows across critical systems (ERP, CRM, HRIS, and other business applications)
  • Develop enterprise data models and architecture diagrams
What we offer
What we offer
  • Medical, vision, dental, and life and disability insurance
  • Company 401(k) plan
  • Free online training
  • Fulltime
Read More
Arrow Right

Senior Principal AI Infrastructure Architect

The Senior Principal AI Infrastructure Architect is a highly skilled and advance...
Location
Location
Italy , Milano
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Significant experience in a consulting, presales or architecture role within a large-scale (preferably multi-national) technology services environment, with a track record of leading AI infrastructure pursuits
  • Demonstrable experience designing and delivering production AI platforms — from single multi-GPU servers through to multi-rack training clusters and inference factories
  • Strong working knowledge of the AI hardware vendor landscape (NVIDIA, AMD, Intel, Dell, HPE, Lenovo, Supermicro, Cisco, Pure, VAST, WEKA, DDN, NetApp) and how to position partner ecosystems competitively
  • Proven ability to translate AI workload requirements (model size, parameter count, sequence length, throughput SLOs, latency targets) into accurate hardware bills of materials and sizing justifications
  • Significant client engagement and consulting experience, including client needs assessment, change management and the ability to identify whitespace for follow-on AI infrastructure and managed-services work
  • Significant business development and presales experience on infrastructure-led deals, ideally including sovereign AI, AI Factory or regulated-industry GenAI programmes
  • Strong understanding of how AI infrastructure integrates with business processes, applications, data platforms and existing enterprise architecture
  • Bachelor's degree or equivalent in Information Technology, Engineering, Computer Science or a related field
  • Deep, hands-on knowledge of AI hardware: GPU and accelerator portfolios (NVIDIA Hopper / Blackwell, AMD MI300/MI325, Intel Gaudi 3, emerging custom silicon), host CPU platforms (Intel Xeon, AMD EPYC, NVIDIA Grace), system topologies (HGX, DGX, MGX, OAM) and how each choice maps to specific AI workloads
  • Strong understanding of AI-class storage: parallel filesystems, all-flash NVMe platforms, S3-class object stores, checkpoint and dataset pipelines and the I/O patterns of large-scale training and inference (VAST, WEKA, DDN EXAScaler, Pure FlashBlade, NetApp ONTAP AI, Dell PowerScale)
Job Responsibility
Job Responsibility
  • Lead the end-to-end design of large, complex AI infrastructure solutions — covering accelerated compute (NVIDIA H100/H200/B200 and GB200 NVL72, AMD Instinct MI300X/MI325X, Intel Gaudi 3), CPU host platforms (Intel Xeon, AMD EPYC, NVIDIA Grace), high-throughput storage tiers and lossless AI fabric — for enterprise, sovereign AI and AI Factory clients
  • Architect reference designs built on NVIDIA DGX/HGX SuperPOD, Dell AI Factory with NVIDIA, Cisco Nexus HyperFabric AI, HPE / Lenovo / Supermicro accelerated compute and equivalent platforms, balancing single-node performance with cluster-scale efficiency
  • Size and validate GPU clusters against real workloads — foundation-model pre-training, distributed fine-tuning, RAG, real-time and batch inference — using the right combination of NVLink/NVSwitch domains, InfiniBand NDR/XDR or Ultra Ethernet / NVIDIA Spectrum-X fabrics and tiered NVMe and parallel storage (VAST, WEKA, DDN, Pure FlashBlade, NetApp ONTAP AI, Dell PowerScale)
  • Define the supporting datacenter design: high-density power (50–140 kW/rack), direct-to-chip and rear-door liquid cooling, structured cabling for AI fabrics and modular deployment models across on-prem, colo and sovereign-cloud footprints
  • Work closely with the sales team to drive the presales process for AI infrastructure pursuits — client discovery, technical workshops, proposal writing, executive presentations and bid defence
  • Translate clients' AI ambitions and business outcomes into a hardware and platform roadmap, positioning NTT DATA's end-to-end portfolio — silicon, systems, storage, fabric, MLOps stack and managed services — to land service-led AI solutions
  • Lead integration of compute, storage, networking, the AI software stack (CUDA, ROCm, Triton, NIM, NVIDIA AI Enterprise, Run:ai, Slurm, Kubernetes / Kubeflow) and managed-service operating models across multiple domains, delivery units and geographies
  • Build business cases, TCO and unit-economics models (cost per token, cost per training run, GPU-hour economics) and end-to-end transition roadmaps for cloud-to-private AI migrations and sovereign AI deployments
  • Define architectural principles for AI infrastructure — accelerator utilisation, data gravity, multi-tenancy, model lifecycle, energy efficiency — and apply them to influence architectural outcomes and governance
  • Develop As-Is, Vision, FMO and To-Be AI platform architectures, identify gaps and develop transition roadmaps
  • Fulltime
Read More
Arrow Right
New

Data Engineer

iPECS is looking for an experienced Data Engineer (Data Architect) to lead the d...
Location
Location
Korea, Republic Of , Seoul
Salary
Salary:
Not provided
ericsson.com Logo
Ericsson
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of experience in data‑focused roles such as Data Engineer, DBA, or backend engineering
  • Strong hands‑on experience in database engineering
  • Experience operating large‑scale RDBMS environments (MySQL, PostgreSQL, Oracle, etc.)
  • Solid experience in query optimization, schema design, indexing, sharding, and performance tuning
  • Deep understanding of SQL and experience designing complex queries and schemas
  • Experience designing Data Warehouses and Data Marts
  • Experience designing and operating ETL/ELT pipelines
  • Experience with database replication, clustering, and failover/fencing configurations
  • Experience diagnosing and tuning performance issues in large‑scale data environments
  • Experience establishing incident response and recovery processes for data pipelines
Job Responsibility
Job Responsibility
  • Design and evolve large‑scale database and data platform architectures
  • Design data flows and pipelines across operational databases, analytics databases, and Data Warehouses/Data Marts
  • Design and operate ETL/ELT pipelines, ensuring data quality and performance
  • Structure data for analytics and BI use cases and design serving layers
  • Define data standards, schemas, and governance policies in collaboration with DevOps, Cloud, and service teams
  • Participate in roadmap design for future data platform scalability
  • Manage schema versioning and migrations integrated with CI/CD pipelines
  • Build monitoring for query performance and transaction logs aligned with observability and logging systems
  • Establish data lifecycle management and optimization strategies
  • Define and implement data quality management frameworks and automated pipelines
Read More
Arrow Right
New

Associate Data Solutions Architect

At Boeing, we innovate and collaborate to make the world a better place. We’re c...
Location
Location
United States , Seattle; Hazelwood; Dallas; Berkeley
Salary
Salary:
95200.00 - 138000.00 USD / Year
boeing.com Logo
Boeing
Expiration Date
June 15, 2026
Flip Icon
Requirements
Requirements
  • 1+ years of experience building data pipelines, ETL/ELT processes, and/or data engineering tasks
  • 1+ years of experience with SQL
  • Experience designing queries and data models for analytics
  • Experience with scripting languages (Python or R programming) for data processing and automation
  • Experience with orchestration tools (Airflow, Prefect, Cloud Composer)
  • Experience with relational databases (Oracle, SQL Server, Postgres)
  • Experience with cloud data service (BigQuery, Redshift, Snowflake, and/or equivalent)
  • Experience with data quality, basic data governance, and logging/monitoring concepts
Job Responsibility
Job Responsibility
  • Implement, test, and maintain Extract, Transform, Load/Extract, Load Transform (ETL/ELT) pipelines to ingest data from on-prem systems and cloud sources into data lakes/warehouses
  • Develop data transformations and data models to support reporting, analytics, and Machine Learning (ML) workloads
  • Build and maintain batch and streaming data workflows using orchestration tools (Airflow, Cloud Composer, Prefect, etc.) and data integration platforms (Informatica, Talend, Fivetran, etc.)
  • Work with on-prem technologies (databases, file shares, middleware) and cloud services (BigQuery, Redshift, Cloud Storage, S3, Dataflow, Glue) to move and transform data
  • Collaborate with data owners to profile data, identify quality issues, and implement data validation and cleansing rules
  • Design and implement data cataloging, lineage, and basic governance controls for datasets
  • Monitor pipeline health, troubleshoot failures, and implement alerts and automated retries
  • Contribute to Continuous Integration and Continuous Delivery (CI/CD) for data pipelines and infrastructure-as-code patterns (Terraform, Cloud Deployment Manager, etc.)
  • Produce clear technical documentation and runbooks for operational support and handoffs
  • Support production handover, incident response, and post-incident retrospectives
What we offer
What we offer
  • health insurance
  • flexible spending accounts
  • health savings accounts
  • retirement savings plans
  • life and disability insurance programs
  • paid and unpaid time away from work
  • generous company match to your 401(k)
  • industry-leading tuition assistance program pays your institution directly
  • fertility, adoption, and surrogacy benefits
  • up to $10,000 gift match when you support your favorite nonprofit organizations
  • Fulltime
Read More
Arrow Right
New

Software Developer

Software Developer Permanent Position!\n\nJob Summary:\nTasks and Activities: As...
Location
Location
United States , Rancho Cucamonga
Salary
Salary:
55.76 - 56.00 USD / Hour
sar-tech.net Logo
SAR Tech
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Information Technology, Engineering, or a closely related field
  • Related experience in Microsoft .NET application development, enterprise application development, and backend systems implementation
  • Strong experience working with Microsoft .NET, C#, .NET Core, ASP.NET, RESTful APIs, and object-oriented programming concepts
  • Experience with front-end technologies such as HTML5, CSS3, JavaScript, TypeScript, Angular, or React is preferred
  • Strong knowledge of SQL Server, database design, query optimization, stored procedures, and relational database management systems
  • Experience with cloud platforms such as Microsoft Azure, Amazon Web Services (AWS), or Google Cloud Platform (GCP) is preferred
  • Experience working with RabbitMQ, messaging systems, and event-driven architecture is preferred
  • Knowledge of software development lifecycle (SDLC), Agile/Scrum methodologies, and version control systems such as Git and Azure DevOps
  • Experience with CI/CD pipelines, deployment automation, Docker, Kubernetes, and DevOps practices is preferred
  • Understanding of application security, authentication, authorization, secure coding practices, and data protection standards
Job Responsibility
Job Responsibility
  • Design, develop, and enhance backend and web-based application components using Microsoft .NET, C#, ASP.NET, Azure technologies to support healthcare business operations
  • Implement business logic, service layers, and APIs following object-oriented principles and industry best practices learned through formal academic training
  • Perform unit testing, debugging, and code refactoring to improve application performance, maintainability, and reliability
  • Collaborate with technical leads and architects to understand and design system architecture and apply structured development methodologies within the Software Development Life Cycle (SDLC)
  • Design, develop, and optimize SQL Server database objects including tables, views, stored procedures, and functions to support healthcare data processing
  • Analyze data requirements and write complex SQL queries to retrieve, validate, and transform healthcare-related data accurately and efficiently
  • Ensure data integrity, performance tuning, and adherence to security and compliance standards while working with large datasets
  • Support application functionality by integrating database logic with .NET application components
  • Assist in batch processing and scheduled job development for large-scale data operations, including claims processing, financial transactions, and reporting workflows
  • Design, develop and integrate web-based user interfaces using HTML, CSS, JavaScript, TypeScript and related web technologies in coordination with backend services
What we offer
What we offer
  • Medical Insurance
  • 401(k) Retirement Plan
  • Fulltime
Read More
Arrow Right