Senior Data Engineer

About ACD For more than a century, AIA Contract Documents (ACD) has supported architecture, engineering, and construction professionals by delivering a shared industry standard to align parties on a project. What began in 1888 with the development of standardized construction contracts has evolved into a comprehensive suite of contract tools and foundational workflows that not only shape how the industry works today, but uniquely position ACD to help firms navigate construction’s growing complexity. In a world driven by scale, fragmentation, and AI-generated decisions that prioritize speed over a clear understanding of risk, ACD serves as a trusted anchor, ensuring project participants can reduce disputes and negotiations, while achieving faster alignment and more predictable outcomes for the future. AIA Contract Documents is seeking a Senior Data Engineer to support the buildout of data infrastructure powering upcoming AI initiatives. This role will play a critical part in designing and structuring data systems that enable scalable, high-quality inputs for AI and machine learning models. The ideal candidate brings deep experience in modern data engineering practices, strong familiarity with Databricks, and the ability to operate independently in a fast-evolving environment with limited oversight. Key Responsibilities - Design and implement scalable data models optimized for AI/ML use cases, ensuring data is structured for effective model training and inference - Architect and manage data pipelines using Databricks, including orchestration, job scheduling, and workflow optimization - Develop and maintain robust ETL/ELT processes to support data ingestion, transformation, and delivery across systems - Leverage Databricks Asset Bundles to manage deployment of data assets (pipelines, jobs, notebooks, and files) across environments - Collaborate with cross-functional teams to align data architecture with AI initiative requirements and business objectives - Ensure data quality, integrity, and governance standards are met across all pipelines and datasets - Contribute to CI/CD practices using Azure DevOps (ADO), with a future transition to GitHub-based workflows - Participate in Agile development processes, including sprint planning, stand-ups, and iterative delivery - Other duties as assigned Qualifications - 5+ years of experience in Data Engineering or related field - Strong expertise in Databricks, including pipeline development, orchestration, and data architecture - Experience designing data models to support AI/ML applications - Proficiency in Python (PySpark) and SQL - Hands-on experience with Azure-based data environments - Experience with CI/CD tools such as Azure DevOps (ADO) - Ability to work independently with minimal oversight and ramp quickly in a fast-paced environment - Experience with Databricks Asset Bundles for deployment and environment management, preferred - Familiarity with transitioning CI/CD workflows to GitHub, preferred - Experience working in Agile development environments, preferred - Exposure to AI/ML workflows and data requirements for model development, preferred What Success Looks Like - Efficient, scalable data pipelines supporting AI initiatives - Well-structured, high-quality datasets optimized for machine learning - Streamlined deployment and orchestration of Databricks assets across environments - Strong collaboration with stakeholders to deliver data solutions aligned with business needs