Add regional household land value calibration targets by vahid-ahmadi · Pull Request #315 · PolicyEngine/policyengine-uk-data

vahid-ahmadi · 2026-03-24T16:17:19Z

Summary

Add regional ons/household_land_value/{REGION} calibration targets
Split ONS National Balance Sheet household land total across regions proportionally by property wealth (dwellings × avg_house_price)
Add regional calibration of main_residence_value and property_wealth to match UK HPI Dec 2025 regional averages

Addresses #314

Problem

Regional land values are wrong because (1) the model used a flat intensity ratio, (2) the WAS-imputed main_residence_value is regionally compressed, and (3) there were no regional calibration targets. The intensity fix is now merged in PolicyEngine/policyengine-uk#1537. This PR fixes the remaining two issues.

What this PR does

1. Regional calibration targets

Each region's share of national household land value is proportional to its total property wealth:

regional_share = (dwellings × avg_house_price) / national_total
regional_target = ONS_household_land_total × regional_share

2. Regional property value calibration

After WAS wealth imputation and before calibration, rescale main_residence_value per region so that owner-occupier means match UK HPI Dec 2025 average house prices. property_wealth is adjusted by the same factor to keep the land-to-property ratio consistent. Follows the same post-imputation calibration pattern as _calibrate_energy_to_need in consumption.py:

factor[region] = UK_HPI_avg_price[region] / imputed_mean(main_residence_value[region])

This fixes the value-level compression (London -41%, North East +39%) that calibration weights alone cannot correct.

Files

New

storage/regional_land_values.csv: region, avg_house_price (UK HPI Dec 2025), dwellings
targets/sources/mhclg_regional_land.py: computes per-region targets, rescaled to ONS total
tests/test_regional_land_value_targets.py: unit tests for calibration targets
datasets/imputations/regional_property_uprating.py: post-imputation regional scaling
tests/test_regional_property_uprating.py: unit tests for property scaling

Modified

targets/compute/other.py: new compute_regional_land_value — filters household_land_value by region
targets/compute/__init__.py and targets/build_loss_matrix.py: wire up the new compute function
datasets/imputations/__init__.py: export uprate_property_by_region
datasets/create_datasets.py: call uprate_property_by_region after wealth imputation

Tests

All pure unit tests (no baseline fixture):

CSV data quality: all 11 GB regions, no NI, all values positive
Target constraints: sum to ONS national total, London highest, London/NE >= 3x, SE > SW, EoE > EM
Registry integration: 11 targets with correct names, values for 2025
Property calibration: regional means match HPI after scaling, renters unchanged, property_wealth scales proportionally

Data sources

The WAS QRF imputation compresses regional property values — London averages £325k vs the UK HPI figure of £551k, while North East is £230k vs £165k. This post-imputation step scales owner-occupied main_residence_value per region so regional means match UK House Price Index Dec 2025 averages. Fixes #314 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace the single national intensity (0.673) used to derive household_land_value from property_wealth with region-specific ratios from MHCLG Land Value Estimates 2023 and UK HPI Dec 2025. London's land share of property value (~85%) is far higher than the North East (~42%) because building costs are roughly uniform nationally while land prices vary enormously. The flat ratio compressed regional variation to 1.2× (London to NE) vs the official 3.3× in house prices and 20× in MHCLG land per hectare. Changes: - Restructure intensity.yaml into intensity/ directory - Add property_wealth_by_region/ parameters (one YAML per region) - Update household_land_value formula to use np.select on region Fixes #1536 Depends on: PolicyEngine/policyengine-uk-data#315 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

This reverts commit fb72db8.

Add MHCLG-derived regional targets for household_land_value so the weight optimiser can push regional totals toward official estimates. - regional_land_values.csv: region, avg_house_price (UK HPI Dec 2025), mhclg_land_per_ha, dwellings, land_intensity - mhclg_regional_land.py: computes per-region targets rescaled to ONS National Balance Sheet household land total - compute_regional_land_value: filters household_land_value by region - Fix Wales HPI (189677 -> 214883) and Scotland HPI (195000 -> 190649) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Unit tests covering: - CSV data quality (all GB regions present, no NI, positive values) - Target constraints (sum to ONS national total, London highest, London/NE ratio >= 3x, regional ordering) - Registry integration (11 targets, correct names, values for 2025) - Data verification (house prices and intensity ratios within range) All tests are pure unit tests with no baseline fixture dependency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove land_intensity and mhclg_land_per_ha from the CSV — intensity ratios belong in the model repo (policyengine-uk). Regional targets now split the ONS household land total proportionally by dwellings × avg_house_price (UK HPI Dec 2025). Rename targets from mhclg/ to ons/ since the national total comes from ONS National Balance Sheet. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Private rent: update from 4.7m England-only renters to 5.4m UK-wide, and from £1,400/mo to ONS PRHI £1,374/mo (~£89bn total) - Social rent: new target from EHS 2023-24 mean £118/wk × 5.0m UK social renters (~£30.7bn total) - Wire up housing/rent_social in compute and loss matrix dispatch Addresses #317 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

This reverts commit b9e15ec.

* Replace flat land intensity ratio with region-specific ratios Replace the single national intensity (0.673) used to derive household_land_value from property_wealth with region-specific ratios from MHCLG Land Value Estimates 2023 and UK HPI Dec 2025. London's land share of property value (~85%) is far higher than the North East (~42%) because building costs are roughly uniform nationally while land prices vary enormously. The flat ratio compressed regional variation to 1.2× (London to NE) vs the official 3.3× in house prices and 20× in MHCLG land per hectare. Changes: - Restructure intensity.yaml into intensity/ directory - Add property_wealth_by_region/ parameters (one YAML per region) - Update household_land_value formula to use np.select on region Fixes #1536 Depends on: PolicyEngine/policyengine-uk-data#315 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add changelog entry for regional land intensity ratios Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix ruff format for household_land_value.py Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Consolidate regional land intensity into single YAML file Replace property_wealth_by_region/ directory (14 files) with a single property_wealth_by_region.yaml containing all regions as named children. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Consolidate intensity/ directory into single intensity.yaml Merge property_wealth, corporate_wealth, and property_wealth_by_region into one file, removing the directory structure. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add YAML unit tests for regional land intensity Test household_land_value for London (0.85), North East (0.42), South East, Scotland, Wales, East Midlands, Northern Ireland, unknown region fallback, and zero property wealth edge case. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update UC taper reform expected impact to -44.5bn The model now produces -44.5bn for the UC taper rate reduction to 20%, drifted from the previous -41.9bn expectation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Relax UC taper reform test tolerance from 1.5 to 3.0bn Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Document intensity ratio derivation methodology Add description explaining the residual method formula and note that Scotland/Wales values are estimates (MHCLG covers England only). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Interpolate Scotland/Wales intensity from nearest English regions Scotland (£191k HPI) → 0.44, interpolated between North East (£165k, 0.42) and North West (£217k, 0.47). Wales (£215k HPI) → 0.47, matched to North West (£217k, 0.47) as nearest by house price. Northern Ireland unchanged at 0.44 (similar price level to Scotland). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove unused intensity.property_wealth and intensity.corporate_wealth Both are dead code: household_land_value now uses property_wealth_by_region, and corporate_land_value computes its ratio directly from aggregate values in value.yaml. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Restore corporate_wealth intensity parameter Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

After WAS wealth imputation, scale main_residence_value and property_wealth per region so that owner-occupier means match UK HPI Dec 2025 average house prices. Follows the same post-imputation calibration pattern as _calibrate_energy_to_need in consumption.py. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

corporate_land_value overshoots the ONS target by 61.2% — this is a pre-existing calibration gap unrelated to the regional property scaling (which only touches property_wealth, not corporate_wealth). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vahid-ahmadi mentioned this pull request Mar 24, 2026

Replace flat land intensity ratio with region-specific ratios PolicyEngine/policyengine-uk#1537

Merged

4 tasks

vahid-ahmadi self-assigned this Mar 25, 2026

vahid-ahmadi and others added 2 commits March 25, 2026 12:08

Revert "Calibrate main_residence_value to UK HPI by region"

9b03cbe

This reverts commit fb72db8.

vahid-ahmadi changed the title ~~Calibrate main_residence_value to UK HPI by region~~ Add regional household land value calibration targets Mar 25, 2026

vahid-ahmadi and others added 3 commits March 25, 2026 12:14

Apply ruff formatting to test file

ade8aab

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vahid-ahmadi requested review from nikhilwoodruff and nwoodruff-co March 25, 2026 12:46

vahid-ahmadi and others added 2 commits March 25, 2026 13:43

Revert "Fix private rent target and add social rent target"

638d7fc

This reverts commit b9e15ec.

vahid-ahmadi and others added 3 commits March 26, 2026 13:50

Format with ruff

b5e3e07

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove accidentally committed files

c3ccb6a

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vahid-ahmadi requested a review from MaxGhenis March 26, 2026 13:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add regional household land value calibration targets#315

Add regional household land value calibration targets#315
vahid-ahmadi wants to merge 12 commits intomainfrom
fix/regional-land-intensity-314

vahid-ahmadi commented Mar 24, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

vahid-ahmadi commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Problem

What this PR does

1. Regional calibration targets

2. Regional property value calibration

Files

New

Modified

Tests

Data sources

Related

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vahid-ahmadi commented Mar 24, 2026 •

edited

Loading