Enterprise Data Platform & AI‑Enabled Innovation at NSW Treasury

Context

As technical lead for enterprise-scale data engineering, analytics, platform architecture, and AI‑enabled innovation, I support NSW Procurement and broader government reporting capabilities, architecting and delivering enterprise-grade data platforms for procurement analytics, supplier reporting, executive reporting, and whole‑of‑government decision‑making.

Platform Architecture & Engineering

The platform is built around a fully automated, scalable data ingestion pipeline that reliably processes data from multiple upstream systems and delivers trusted, analytics‑ready datasets. Data is sourced from multiple internal and external providers in varying formats and delivery schedules, securely landed on a centralised SFTP server, and standardised prior to ingestion, supporting both incremental and full loads with full historical traceability and auditability.

The pipeline is orchestrated end‑to‑end using Azure Data Factory (ADF), coordinating data movement, validation, processing, and reporting workflows. Automated data quality checks are applied early to validate schema consistency, completeness, and business rules, with automated notifications triggered to source data providers when issues are detected. Validated data is processed and stored using a medallion architecture (Bronze, Silver, Gold) to keep raw, curated, and analytics‑ready data clearly separated.

I design secure, scalable, fault‑tolerant integration patterns across multiple government agencies and technology environments, and establish metadata‑driven development approaches that improve platform consistency and reduce delivery complexity.

Snowflake Platform Governance

I lead Snowflake platform adoption, optimisation, and governance, introducing Git‑integrated schema management, automated DDL versioning, and repository‑based engineering practices across the platform. This includes standardising reference‑data management and metadata governance frameworks across multiple analytics platforms, using Snowpipe, Streams & Tasks, and Dynamic Tables to improve reliability, performance, and supportability through engineering standardisation.

Data Quality & Data Product Enablement

I design and expand enterprise data quality frameworks supporting automated validation, exception management, supplier feedback workflows, and operational reporting, delivering trusted data products that improve transparency, reporting accuracy, and business confidence in analytics outputs, while establishing scalable QA processes that significantly reduce manual intervention.

AI & Innovation Leadership

I drive AI experimentation and innovation initiatives focused on Microsoft AI Foundry and intelligent automation, evaluating AI use cases for data standardisation, enrichment, quality management, and operational efficiency. This includes championing adoption of agentic and generative AI concepts that improve engineering productivity, and helping define future‑state AI‑enabled analytics capabilities aligned with organisational strategy.

Team Leadership

I lead and mentor engineers and analysts across multiple workstreams, providing technical governance, architecture guidance, code reviews, and engineering direction. I deliver capability uplift programs covering Snowflake, Azure Data Factory, CI/CD, GitHub, Python, and modern engineering practices, fostering a high‑performance engineering culture focused on quality, collaboration, and continuous improvement.

Key Achievements

• Delivered an enterprise‑wide professional services spend reporting capability supporting whole‑of‑government stakeholders.
• Resolved complex cross‑agency integration challenges through scalable architecture solutions.
• Automated previously manual reporting and operational processes, driving significant productivity gains.
• Successfully modernised legacy workflows into cloud‑native data platforms.
• Received the Innovation Award for delivering a high‑impact automated solution supporting the Audit Office.

Technologies Used

Outcome

The platform significantly improved data reliability, reduced ingestion failures, and eliminated manual monitoring. Real‑time Power BI dashboards provide full transparency into pipeline health, while automated notifications keep engineering and product teams informed of ingestion status and downstream impacts, underpinned by governed Snowflake engineering practices and an emerging AI‑enabled innovation roadmap.