verification / datasets / SHORTLIST.md

Variable shortlist — the 100 that matter

Selected from the full register of 277 variables (README). Scored per variable on availability, understandability, efficiency signal, and uniqueness (0-5 each, total /20 — recorded in each variable file's frontmatter with a one-line justification). Near-duplicate variables were cut first, keeping the stronger twin (76 dropped as duplicates or prospective-only), then the top 100 by total, tie-broken by efficiency signal.

Not selected does not mean deleted — everything remains in the register; this is the working set for scoring councils.

# Variable Dataset Score A U E Q Note
1 debt_servicing_share Capital & debt 20 5 5 5 5 Best-in-set: pounds diverted from services to debt, per pound of service spend.
2 efs_awarded_in_year EFS & s114 20 5 5 5 5 Official GOV.UK distress flag, clean binary, the dataset's core signal.
3 household_recycling_rate_pct WasteDataFlow 20 5 5 5 5 The standard ex-NI192 outcome measure with a decade of history; the dataset's strongest variable.
4 dsg_deficit_cum SEND/DSG deficits 19 5 4 5 5 The headline outcome; sign convention needs care but it is the deficit signal itself.
5 lgsco_compliance_rate Ombudsman/LGSCO 19 5 4 5 5 Did-the-council-actually-comply — a hard outcome with per-council coverage duplicated nowhere else.
6 lgsco_uphold_rate_vs_similar Ombudsman/LGSCO 19 5 4 5 5 Type-controlled excess fault rate — the cleanest single efficiency comparison in the dataset.
7 asc_unit_cost_res_nursing_care Adult social care 19 4 5 5 5 The headline weekly care-home unit cost — the cleanest, strongest efficiency signal in the dataset.
8 tsm_non_decent_homes TSM 19 4 5 5 5 Decent Homes failure share — the hardest single read on stock-condition outcomes.
9 ta_bnb_children_over_6wk Temporary accommodation 19 5 5 5 4 Directly published statutory-breach count: unambiguous, high-signal, unique to H-CLIC.
10 efs_amount_in_year EFS & s114 18 4 4 5 5 Official amounts but regex-parsed and retroactively revised; strong severity outcome.
11 hn_independent_share SEND/DSG deficits 18 4 4 5 5 Reliance on expensive independent placements — a genuine efficiency/structure proxy.
12 debt_to_core_spend Capital & debt 18 5 4 5 4 Strong Woking-style leverage flag; slight overlap with gross_borrowing_to_core_spend.
13 lgsco_uphold_rate Ombudsman/LGSCO 18 5 4 5 4 Headline per-council fault rate — strong outcome, volatile at small investigation counts.
14 dsg_deficit_share_of_grant SEND/DSG deficits 18 4 5 5 4 Size-normalised deficit — the cleanest cross-council performance comparator.
15 bnb_share_of_ta Temporary accommodation 18 4 5 5 4 The classic placement-mix efficiency measure: costliest TA type as a share of caseload, easy to read.
16 tsm_non_emergency_repairs_on_time TSM 18 4 5 5 4 Repairs timeliness against target — a core operational-efficiency measure.
17 tsm_emergency_repairs_on_time TSM 18 4 5 5 4 Emergency repairs timeliness; core efficiency signal alongside RP02 part 1.
18 tsm_repairs_satisfaction TSM 18 4 5 5 4 Repairs-specific satisfaction — the second TP representative, closest to efficiency.
19 residual_waste_per_household_kg WasteDataFlow 18 4 5 5 4 Defra-published ex-NI191 indicator, clean and directly comparable; minor Table_3 gaps.
20 ct_collection_rate Council tax collection 18 5 5 5 3 Headline in-year rate — published, obvious, the core efficiency measure; slightly duplicated by its own gap/trend derivatives.
21 efs_years_count EFS & s114 18 5 5 5 3 Strong chronic-distress signal but a direct derivative of efs_awarded_in_year.
22 debt_growth_rate Capital & debt 18 5 4 4 5 Pace of debt build-up is self-normalizing and a proven distress precursor.
23 tsm_overall_satisfaction TSM 18 4 5 4 5 Headline perception measure — kept as the representative of the TP family.
24 lgsco_upheld_per_100k Ombudsman/LGSCO 18 5 5 4 4 Population-normalised fault volume, cross-council comparable and already computed by LGSCO.
25 mrp_adequacy Capital & debt 17 5 2 5 5 Under-provisioning is a sharp, unique distress signal, but MRP/CFR is deep jargon.
26 cost_per_tonne_gbp WasteDataFlow 17 3 4 5 5 Headline GBP/tonne efficiency measure; capped by 2024-25-only cross-dataset spend join.
27 s114_issued EFS & s114 17 2 5 5 5 Hardest distress signal there is, but rests on the weak hand-compiled unofficial list.
28 tax-collection-cost-per-pound-collected Revenue Outturn 17 4 4 5 4 Cost per pound collected on a universal function is a sharp efficiency measure; needs the dataset 03 join and its rate as a guard.
29 waste-cost-per-tonne Revenue Outturn 17 4 4 5 4 Cleanest like-for-like unit cost in the dataset — universal service, real output denominator; needs the dataset 12 tonnage join.
30 collection_rate_gap_class Council tax collection 17 4 4 5 4 Peer-adjusted rate is the best like-for-like efficiency score here, though Class is a coarse comparator.
31 nightly_paid_share_of_ta Temporary accommodation 17 4 4 5 4 Spot-market exposure share, a direct high-cost procurement signal.
32 tsm_stage1_complaints_on_time TSM 17 4 4 5 4 Complaints handled within Ombudsman timescales — a direct process-efficiency read.
33 formal_distress_flag EFS & s114 17 3 5 5 4 Broadest binary distress outcome; s114 half inherits unofficial-list false negatives.
34 arrears_intensity Council tax collection 17 5 4 5 3 Self-normalised arrears burden with a clean built-in denominator; overlaps other arrears ratios in-set.
35 lgsco_upheld Ombudsman/LGSCO 17 5 4 5 3 Core per-council fault count — strong outcome, though duplicated by upheld_n and the rate family.
36 asc_share_care_home_vs_community Adult social care 17 4 4 4 5 Unique strategy-mix signal (care home vs community) tied to cheaper community-weighted models.
37 hra_repairs_spend_per_home TSM 17 4 4 4 5 Only cost-side variable here; pairs spend with TSM outcomes for value-for-money.
38 reserves-cover Revenue Outturn 17 5 4 4 4 Months-of-resilience ratio, the classic s114 precursor — available, intuitive, and strongly separates well- from badly-run councils.
39 nndr_collection_rate Council tax collection 17 5 4 4 4 Independent second tax stream with a distinct payer base; strong cross-check on systemic collection weakness.
40 q4_collection_share Council tax collection 17 5 4 4 4 Back-loaded cash profile is a distinctive process-quality signal not captured by any level or stock measure.
41 dsg_la_contribution SEND/DSG deficits 17 5 4 4 4 Direct evidence the deficit has spilled into the council's own funds.
42 lgsco_satisfactory_remedy_rate Ombudsman/LGSCO 17 5 4 4 4 Distinct self-correction outcome — fault fixed before the ombudsman — volatile on small upheld counts.
43 household_waste_per_person_kg WasteDataFlow 17 4 5 4 4 Clean per-head arisings measure; more a minimisation than cost-efficiency signal.
44 ta_out_of_area_rate Temporary accommodation 17 5 5 4 3 MHCLG's own size-adjusted out-of-area measure, cleanest of the three out-of-area variants.
45 ta_spend_per_household_in_ta Temporary accommodation 16 2 4 5 5 The unit-cost headline for the dataset; strongest efficiency signal but needs a cross-dataset RO4 join.
46 asc_inhouse_vs_external_residential_cost_gap Adult social care 16 2 4 5 5 Prime in-sourcing efficiency gap, unique across all datasets, but suppression leaves incomplete coverage.
47 dsg_in_year_movement SEND/DSG deficits 16 4 3 5 4 Current-year deterioration vs recovery — the sharpest management signal, though Safety Valve cash can flatter it.
48 lgsco_compliance_failures Ombudsman/LGSCO 16 4 3 5 4 Hard non-compliance outcome, though constructed/estimated from the rate and small counts.
49 ta-cost-per-household-in-ta Revenue Outturn 16 3 4 5 4 True unit cost that cleanly separates expensive provision from demand, but needs the dataset 06 join and only exists from 2019-20.
50 efs_share_of_revenue_spend EFS & s114 16 3 4 5 4 Best size-adjusted severity measure; weakened by free-text name join to outturn data.
51 ho_maladministration_rate Ombudsman/LGSCO 16 3 4 5 4 The HO headline failure rate — strong outcome; volatile near the 5-findings inclusion threshold.
52 ho_severe_maladministration Ombudsman/LGSCO 16 3 4 5 4 Rare, high-signal gravest-tier count — the sharpest HO severity marker.
53 collection_cost_per_household_gbp WasteDataFlow 16 3 4 5 4 Strong per-household cost signal but constructed from a 2024-25-only RO5 join.
54 efs_cumulative_amount EFS & s114 16 4 4 5 3 Accumulated scale of support; a sum of efs_amount_in_year with revision risk.
55 tsm_perception_gap_repairs TSM 16 4 3 4 5 Reported timeliness minus perceived satisfaction — a distinctive expectations-gap construct.
56 dsg_deficit_vs_revenue SEND/DSG deficits 16 3 4 4 5 Unique cross-dataset materiality measure; availability docked for the fragile join to dataset 01.
57 real-spend-trend Revenue Outturn 16 4 4 4 4 Cost discipline over time is a genuine efficiency read that raw levels miss; reorganisations break some series.
58 spend_per_pupil SEND/DSG deficits 16 4 4 4 4 DfE-computed per-pupil normalisation enabling like-for-like spend comparisons.
59 own_stock_share_of_ta Temporary accommodation 16 4 4 4 4 Cheap-supply proxy with real efficiency content, though structurally biased by stock transfer.
60 ta_5yr_plus Temporary accommodation 16 4 4 4 4 Distinct stuck-caseload duration signal; six-column sum compounds rounding.
61 ta_bnb_mean_stay_days Temporary accommodation 16 4 4 4 4 Direct duration-in-worst-TA measure; TA7 quirks and mean skew temper it.
62 asc_income_recovery_rate Adult social care 16 4 4 4 4 Distinct charging-efficiency ratio, though confounded by deprivation and care mix.
63 tsm_repairs_per_home TSM 16 4 4 4 4 Demand-normalised repairs volume; mixes stock condition with reporting ease.
64 corporate-overhead-ratio Revenue Outturn 16 5 4 4 3 Intuitive 'spend on running the organisation' share with real efficiency content, though recharging artefacts and overlap with central-services-ratio cap it.
65 asset_sale_reliance Capital & debt 16 5 4 4 3 Good ratio but overlaps receipts_coverage_of_capex; jargon around applied receipts.
66 fees-and-charges-recovery Revenue Outturn 16 5 4 3 4 Clear 'users pay vs taxpayers pay' story from this CSV alone; fee caps and policy choices dilute the efficiency reading.
67 social-care-squeeze Revenue Outturn 16 5 4 3 4 Crowding-out share is vivid and computable from this CSV, but it is a pressure indicator more than an inefficiency measure.
68 asc_unit_cost_by_setting_age_psr Adult social care 15 3 2 5 5 Strong case-mix-adjusted unit costs, but 61 cuts with heavy suppression are hard to navigate.
69 duplicate_payment_candidates Transparency spend 15 2 3 5 5 Direct avoidable-cost screen unique to transaction-level spend data; Sefton over-counts.
70 arrears_recovery_rate Council tax collection 15 3 3 5 4 Direct debt-chasing effort measure, but note a zeros in col 20 understate it for some councils.
71 recycling_cost_per_recycled_tonne_gbp WasteDataFlow 15 3 3 5 4 Sharp recycling-efficiency measure; negative values and the RO5 join make it harder to read.
72 ho_maladministration_findings Ombudsman/LGSCO 15 3 4 5 3 Core adverse-finding count — strong outcome, but volume-driven and threshold-limited.
73 writeoff_rate Council tax collection 15 4 3 4 4 Propensity to abandon debt is a distinct behaviour signal, but prudent write-off muddies interpretation.
74 tsm_complaint_escalation_rate TSM 15 4 3 4 4 First-response quality proxy; volatile at low complaint counts.
75 tsm_repairs_backlog_ratio TSM 15 4 3 4 4 Completed-vs-raised balance signals whether the repairs service is keeping up.
76 ho_chfos Ombudsman/LGSCO 15 3 4 4 4 Distinct process-failure order signal, but 5+-findings threshold and lossy name-join limit coverage.
77 landfill_share_pct WasteDataFlow 15 3 4 4 4 Clear diversion outcome but only populated for authorities with disposal responsibility.
78 central-services-ratio Revenue Outturn 15 5 3 4 3 Recharge-robust overhead comparator computable from this CSV alone, but substantially overlaps corporate-overhead-ratio.
79 reserves-drawdown-rate Revenue Outturn 15 5 3 4 3 Living-off-savings signal with real discriminating power, but heavily overlaps reserves-cover and needs the earmarked caveat.
80 arrears_growth Council tax collection 15 4 4 4 3 Direction of the arrears pile is a good dynamic signal, but confounded by write-off policy and year joins.
81 collection_rate_trend Council tax collection 15 4 4 4 3 Improvement vs deterioration adds a dynamic axis, though it is a thin derivative of the two rates.
82 dsg_deficit_trend SEND/DSG deficits 15 4 4 4 3 Trajectory separates recovering from deteriorating councils, but is a repackaging of dsg_deficit_cum.
83 tsm_stage2_complaints_on_time TSM 15 4 4 4 3 Stage-two timeliness; strong signal but correlated with stage-one on-time.
84 internal_borrowing_gap Capital & debt 15 5 2 3 5 Unique refinancing-risk angle, but internal borrowing is hard to explain simply.
85 staff-cost-share-per-service Revenue Outturn 15 5 3 3 4 Delivery-model share that can flag bad outsourcing contracts, though cost share alone does not say which model is cheaper.
86 dsg_grant_adj SEND/DSG deficits 15 5 3 3 4 Mostly a Safety-Valve bailout marker — a caveat variable more than a performance signal.
87 tsm_stage1_complaints_per_1000 TSM 15 4 4 3 4 Complaint volume is ambiguous (dissatisfaction vs accessible process) but distinct.
88 tsm_satisfaction_yoy_change TSM 15 4 4 3 4 Direction of travel in TP01; adds a trend dimension the levels lack.
89 hn_independent SEND/DSG deficits 15 5 4 3 3 Key cost driver, but the ratio version (hn_independent_share) carries most of its signal.
90 ta_la_ha_stock_total Temporary accommodation 15 5 4 3 3 Cheap-supply mix component; the share version is the analytical variable.
91 spend-per-resident Revenue Outturn 15 4 5 3 3 The one-sentence headline comparator, but needs external population and mixes need, funding and efficiency; per-resident normalisations recur across datasets.
92 tsm_gas_safety TSM 15 4 5 3 3 Statutory gas compliance; near-100% ceiling compresses signal, but shortfalls are meaningful.
93 lgsco_complaints_received_total Ombudsman/LGSCO 15 5 5 2 3 Universally available and simple, but raw escalation demand is not a fault measure.
94 tsm_homes_lcra TSM 15 4 5 1 5 Stock size — a denominator/control, not an efficiency outcome, but uniquely useful.
95 ho_maladministration_per_1000_homes Ombudsman/LGSCO 14 2 3 5 4 Best size-comparable HO failure rate, but depends on the lossy TSM name-join and 1,000+-home coverage.
96 ho_compensation_per_determination Ombudsman/LGSCO 14 3 4 4 3 Good size-adjusted severity outcome, still case-mix driven and threshold-limited.
97 ho_compensation_total Ombudsman/LGSCO 14 3 4 4 3 Clear financial-severity outcome, but scales with volume and only exists above the 5-findings threshold.
98 ho_findings_damp_mould Ombudsman/LGSCO 14 3 4 4 3 High-salience Awaab-adjacent failure count; small numbers and threshold-limited coverage.
99 ho_orders_made Ombudsman/LGSCO 14 3 4 4 3 Mandatory-remedy volume — a solid severity outcome, though volume-confounded.
100 pwlb_dependence Capital & debt 14 4 3 3 4 Distinct liquidity-stress angle; quarterly-only and needs manual denominator.

Composition

What was cut and why

Transparency spend keeps only its best variable (duplicate-payment candidates) because the dataset is currently a three-council sample; its other measures re-enter the shortlist if a full scrape happens.