W
Skip to Content

Hal Wall.

Data Engineer at Chubb Insurance

Somewhere between Databricks and Dickens.

About

Data Engineer with over four years of experience designing production-grade pipelines and cloud-native architectures across insurance and finance. Currently architecting global datasets at Chubb, working across North America, EMEA, and APAC to enable data-driven underwriting and risk assessment at enterprise scale. Trained in Theoretical Physics at Imperial College London, bringing analytical depth to system design, platform governance, and the kind of engineering that earns trust over time.

Outside work, I build things: a literary app called The Daily Canon, an AI property tool called Renovision, and an annual leave planner called Timewell. I'm also a long-distance runner, a hiker and wildcamper, and an enthusiastic dog owner, often found exploring the wilder corners of the UK with Japhy, my Wirehaired Vizsla.

Experience

  1. Aug 2025 — Present

    Leading the design and build of Chubb's curated external company data asset across EMEA and Global, owning entity resolution, vendor data conflation, and data architecture across multiple commercial providers and millions of records. Built a probabilistic entity resolution pipeline with a 300x runtime improvement over the existing production system. Developed a full-stack automated pipeline dashboard using Next.js, FastAPI, and Tailwind.

    • Python
    • SQL
    • Databricks
    • PySpark
    • Azure
    • Next.js
    • FastAPI
  2. Nov 2024 — May 2025

    Refactored a 30,000-line legacy pipeline from monolithic notebooks to modular PySpark, reducing runtime from 270 minutes to 45 minutes. Delivered end-to-end data products supporting AI-driven risk modelling and virtual portfolio analysis. Shaped platform governance during Azure migration, implementing Unity Catalog and automating critical pipelines with Kafka and Databricks Workflows.

    • Python
    • SQL
    • Databricks
    • PySpark
    • Azure
    • Unity Catalog
    • Kafka
  3. Jun 2025 — Aug 2025

    Built an automated Deletion and Retention framework ensuring GDPR compliance across all layers of a medallion architecture, tested to guarantee zero accidental deletions. Created architectural documentation for complex metadata-driven infrastructure, adopted by both consultant and internal engineering teams.

    • Python
    • Databricks
    • Azure Data Factory
    • Azure DevOps
  4. Oct 2022 — Nov 2024

    Designed a custom PySpark CDC solution on Databricks that reduced data volumes by 85% and cut file transfer times from over an hour to 15 minutes. Architected a JSON-driven contract system enabling dynamic toggling of data file production without code changes. Led sprint planning and backlog grooming across cross-functional squads.

    • Databricks
    • Azure Data Factory
    • Python
    • SQL
    • PySpark
    • Power BI
  5. Mar 2021 — Apr 2022

    Owned analytics reporting across B2B marketing campaigns. Piloted and scaled employee advocacy program. Collaborated on diversity and inclusion initiatives while managing stakeholder relationships.

    • Analytics
    • Campaign Management
    • Stakeholder Engagement
  6. Oct 2019 — May 2021

    Founded a creative consultancy delivering photography, branding, and design work. Managed client relationships and design roadmaps, creating visual identities for businesses ranging from startups to established brands.

    • Photography
    • Graphic Design
    • Creative Direction
    • Branding

Projects

  • The Daily Canon

    One carefully chosen work of literature for each day of the year. Novels, short stories, poems, essays, plays, philosophy: great writing in all its forms, delivered one day at a time.

    thedailycanon.org
    • Next.js
    • Editorial
    • Curation
    • Literature
    Featured
  • Renovision

    An AI-powered property renovation visualization tool. Paste a Rightmove URL, select rooms, configure a renovation style, and see possibilities beyond avocado walls. Built during my partner and I's own property search.

    • Next.js
    • AI/ML
    • TypeScript
    • Tailwind CSS
    Featured
  • Timewell

    An annual leave planner for UK employees. Stripped of its usual HR clunkiness and dressed in clarity. It offers visual insight into your year ahead and calculates the ROI of your time off. Vanilla JavaScript, no build step required, just honest utility.

    • JavaScript
    • HTML/CSS
    • UX Design
    Featured