Job Description:
As a key member of our global development team, you will: Innovate & Develop: Partner closely with project managers, business stakeholders, and senior managers to translate complex business requirements into well-architected technical solutions. Consult with users and other technology groups, providing advanced programming insights and support. Drive cross-functional collaboration with diverse management teams to ensure seamless integration of functions, aligning efforts to achieve strategic organizational goals. Proactively identify, define, and implement necessary system enhancements to facilitate the successful deployment of new products and process improvements. Complex Problem Resolution: Lead the resolution of high-impact problems and critical projects through in-depth evaluation of intricate business processes, complex system architectures, and relevant industry standards. Employ advanced analytical and interpretive thinking to define issues, uncover root causes, and develop innovative, sustainable solutions. Consult with users, clients, and other technology groups on issues, and recommend programming solutions. Analyze complex technical and business challenges, and propose innovative solutions that enhance system functionality and business processes. Technical Architecture & Standards Leadership: Serve as a subject matter expert in application programming, ensuring that all application designs rigorously adhere to the overall architectural blueprint and strategic technology roadmap. Leverage an advanced understanding of system flow to develop and enforce robust standards for coding, testing, debugging, and implementation across development teams. Mentorship & Talent Development: Act as a trusted advisor and coach for mid-level developers and analysts, providing guidance, fostering skill development, and judiciously allocating work to maximize team potential and project success. Provide technical guidance, mentorship, and code reviews to junior data engineers, fostering a culture of excellence and continuous improvement. Operational Excellence: Ensure adherence to best practices and essential procedures. Autonomy & Ownership: Operate with a high degree of independence and judgment, taking ownership of critical initiatives and driving them to successful completion. Risk Management: Proactively assess and manage technical risks, demonstrating a strong commitment to regulatory compliance, ethical judgment, and transparent reporting of control issues. Design, develop, and maintain robust, scalable, and high-performance data pipelines using PySpark. Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality data solutions. Optimize and tune Spark jobs for performance and efficiency. Implement data quality checks and ensure data integrity across all data pipelines. Data Architecture & Design: Design, develop, and optimize data architectures, pipelines, and data models to support various business needs, including analytics, reporting, and machine learning. ETL/ELT Development (Python/PySpark Focus): Build, test, and deploy highly scalable and efficient ETL/ELT processes using Python and PySpark to ingest, transform, and load data from diverse sources into data warehouses and data lakes. Develop and optimize complex data transformations using PySpark. Data Quality & Governance: Implement best practices for data quality, data governance, and data security to ensure the integrity, reliability, and privacy of our data assets. Performance Optimization: Monitor, troubleshoot, and optimize data pipeline performance, ensuring data availability and timely delivery, particularly for PySpark jobs.
