Director, Data Integration & Workflows

About the Role: Grade Level (for internal use): 13 The Team: The Director of Data Integration & Workflows leads the team that serves as the backbone of enablement at S&P Dow Jones Indices (SPDJI), responsible for building, templating, and standardizing the data pipelines and workflow patterns that power our index and analytical solutions. Reporting to the Director of Data AI & Enablement, this role is critical to the Data Platform enablement strategy, accelerating delivery, reducing production risk, and ensuring that every pipeline and workflow is scalable, observable, and maintainable. Responsibilities and Impact: Strategic Leadership & Technical Vision Define and drive the data automation and transformation strategy for the Data Integration & Workflows group, setting standards and approaches for building production-ready pipelines and workflow automation across value streams Establish reference architectures and engineering standards for ETL/ELT, orchestration, error handling, observability, and performance/cost optimization with clear "definition of done" criteria for production readiness Collaborate with the PPD Group to build sustainable, transformational enhancements to the data platform and associated tools Foster a culture of technical excellence, craftsmanship, reusable component development, and continuous improvement in automation maturity Contribute to the broader Data AI & Enablement strategy, ensuring Data Integration & Workflows capabilities align with organizational strategic goals Delivery Through Enablement Lead delivery through enablement by assessing SME technical capability and selecting the right engagement model Partner with value stream SMEs to co-develop and review pipelines, adapting support based on SME technical capability and fostering their growth Oversee the design and implementation of robust, reusable data integration and workflow patterns for both batch and streaming use cases Partner with PPD on feasibility and planning, providing realistic estimates, identifying dependencies, and shaping technical scope to ensure delivery commitments are achievable and measurable Develop and implement training programs to enhance the technical proficiency of value stream SMEs in data engineering practices Quality Assurance & Production Readiness Run the code review and quality gate process for SME-built pipelines, ensuring consistency in maintainability, testing, logging, data validation, and documentation prior to IT handover Ensure all solutions are production-ready with comprehensive documentation, testing, error handling, and operational monitoring Coordinate seamless IT handover and production gateway readiness, ensuring complete deployment packages (runbooks, architecture notes, testing evidence, monitoring/alerting expectations) Partner with IT during QA to resolve issues quickly and ensure solutions meet enterprise standards for quality, security, and supportability Implement and maintain governance frameworks specific to data integration, ensuring compliance with organizational policies and industry standards Operational Excellence Provide L3 support for production business-logic issues (in collaboration with value stream SMEs), driving root-cause analysis and permanent fixes for recurring pipeline failures or data breaks Ensure implementation of strong data reliability controls including validation rules, reconciliation checks, anomaly detection, and completeness/timeliness controls that protect downstream index processes Drive performance and cost optimization through appropriate partitioning, caching, incremental processing patterns, and compute usage tuning—balancing speed, stability, and platform spend Establish and monitor operational metrics to track solution delivery timeliness, pipeline reliability, and platform performance Team Development & Collaboration Lead, mentor, and develop a high-performing team of Data Integration Leads and Experts Build data engineering capability across the organization through structured mentorship, knowledge sharing, and hands-on coaching Collaborate effectively with the other pillars within Data AI & Enablement (AI Solutions and Data Governance) to ensure cohesive platform enablement Foster strong partnerships with PPD teams, IT, Data Value Streams, and Data Services & Strategy to align technical enablement efforts with business priorities Shared Accountabilities With PPD: Collaborate on prioritization and alignment of data integration efforts with business requirements and strategic goals; provide realistic technical assessments to inform planning. With IT: Partner to ensure infrastructure readiness, smooth deployment of production-ready solutions, and operational excellence; coordinate handover processes and support production gateway requirements. With Data Value Streams: Engage with value stream SMEs to co-develop solutions, ensuring alignment with business logic and domain expertise; assess and develop SME technical capabilities. With Data Services & Strategy: Work together with Vendor Governance and Catalog teams to establish scalable data integration patterns and ensure proper metadata and lineage tracking. With AI Solutions & Data Governance Teams: Coordinate on cross-cutting concerns including data quality standards, AI data pipeline requirements, and governance compliance. Ownership Data Integration & Workflow Strategy: Own the technical strategy, standards, and execution approach for all data pipeline and workflow automation initiatives Production Readiness Framework: Responsible for defining and enforcing the "definition of done" for production-ready data solutions SME Enablement in Data Engineering: Own the capability development strategy for value stream SMEs in data integration and automation Quality Gates & Code Review Process: Maintain the quality assurance framework for all data pipelines prior to production deployment What Success Looks Like Ac