This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
We are seeking a highly skilled and experienced Senior/Architect Data Engineer to lead the end-to-end architecture of a Databricks-centric multi-agent processing engine. This role will leverage cutting-edge technologies such as Mosaic AI, Model Serving, MLflow, Unity Catalog, Delta Lake, and Feature Store to automate decoder processes at scale.
Job Responsibility:
Lead the design and implementation of a Databricks-centric multi-agent processing engine
Design governed data ingestion, storage, and real-time processing workflows using Delta Lake, Structured Streaming, and Databricks Workflows
Own the model lifecycle with MLflow, including experiment tracking, registry/versioning, A/B testing, drift monitoring, and automated retraining pipelines
Architect low latency model serving endpoints with auto-scaling and confidence-based routing for sub-second agent decisioning
Establish robust data governance practices with Unity Catalog, including access control, audit trails, data quality, and compliance
Drive performance and cost optimization strategies, including auto-scaling, spot usage, and observability dashboards
Define production release strategies (blue-green), monitoring and alerting mechanisms, operational runbooks, and Service Level Objectives (SLOs)
Partner with engineering, MLOps, and product teams to deliver human-in-the-loop workflows and dashboards
Lead change management, training, and knowledge transfer while managing a parallel shadow processing path
Plan and coordinate phased delivery, success metrics, and risk mitigation
Requirements:
Proven experience architecting solutions on the Databricks Lakehouse using Unity Catalog, Delta Lake, MLflow, Model Serving, Feature Store, AutoML, and Databricks Workflows
Expertise in real-time/low latency model serving architectures with auto-scaling, confidence-based routing, and A/B testing
Strong knowledge of cloud security and governance on Azure or AWS, including Azure AD/AWS IAM, encryption, audit trails, and compliance frameworks
Hands-on MLOps skills across experiment tracking, model registry/versioning, drift monitoring, automated retraining, and production rollout strategies
Proficiency in Python and Databricks native tooling, with practical integration of REST APIs/SDKs and Databricks SQL in analytics products
Familiarity with React dashboards and human-in-the-loop operational workflows for ML and data quality validation
Demonstrated ability to optimize performance, reliability, and cost for large-scale analytics/ML platforms with strong observability
Experience leading multi-phase implementations with clear success metrics, risk management, documentation, and training/change management
Domain knowledge in telemetry, time series, or industrial data (aerospace a plus) and prior work with agentic patterns on Mosaic AI
Databricks certifications and experience in enterprise deployments of the platform are preferred
Nice to have:
Domain knowledge in telemetry, time series, or industrial data (aerospace a plus)
Prior work with agentic patterns on Mosaic AI
Databricks certifications
Experience in enterprise deployments of the Databricks platform
What we offer:
Flexible working hours
Hybrid work model
Cafeteria system
Generous referral bonuses (up to PLN6,000)
Additional revenue sharing opportunities
Ongoing guidance from dedicated Team Manager
Tailored technical mentoring from assigned technical leader
Dedicated team-building budget for online and on-site team events
Opportunities to participate in charitable initiatives and local sports programs
Supportive and inclusive work culture
Fully funded training, certifications, and participation in technology conferences
Welcome to CrawlJobs.com – Your Global Job Discovery Platform
At CrawlJobs.com, we simplify finding your next career opportunity by bringing job listings directly to you from all corners of the web. Using cutting-edge AI and web-crawling technologies, we gather and curate job offers from various sources across the globe, ensuring you have access to the most up-to-date job listings in one place.
We use cookies to enhance your experience, analyze traffic, and serve personalized content. By clicking “Accept”, you agree to the use of cookies.