Dataset 11 — Transparency Code spend over £500

What this is (and why there is no single file)

Under the Local Government Transparency Code 2015 (mandatory for English local authorities), each council must publish, at least quarterly (in practice usually monthly), details of every item of expenditure over £500. There is no central government file: every council publishes its own files, on its own website, in its own schema (column names, sign conventions, VAT treatment, redaction rules all vary). Aggregators exist (e.g. spendnetwork.com, data.gov.uk mirrors) but are third-party and often stale — e.g. the data.gov.uk mirror of Leeds City Council's dataset (CKAN package 80446967-46ef-4283-bf65-1014ecfc4fbc) stops at May 2021 even though Leeds still publishes monthly on Data Mill North.

Per the corpus rule, this folder therefore holds a representative sample of three councils (one county, one metropolitan borough, one district pair with a joint-services arrangement), plus documentation of what a full collection would take.

Files downloaded (data/), 2026-08-13

Adur District Council & Worthing Borough Council (districts, shared services)

• Landing page: https://www.adur-worthing.gov.uk/about-the-councils/open-data/payments-to-suppliers/2026-payments-to-suppliers
• adur-june-2026.csv — https://www.adur-worthing.gov.uk/media/Media,174581,smxx.csv
• worthing-june-2026.csv — https://www.adur-worthing.gov.uk/media/Media,174582,smxx.csv
• adur-worthing-joint-june-2026.csv — https://www.adur-worthing.gov.uk/media/Media,174583,smxx.csv
• Edition: June 2026 (latest published at download time). Columns: Department, DepartmentSubsection, Expense_Type, Procurement_Class_Type, Posted_Date, Transaction_ID, Amount, Capital\Revenue, Supplier, AP_Ac_Postcode, Entity.
• Caveat: some Amount fields contain a mis-encoded £ sign (Windows-1252 in a file consumed as UTF-8); strip non-numeric characters before parsing.

Gloucestershire County Council (county)

• Landing page: https://www.gloucestershire.gov.uk/council-and-democracy/performance-and-spending/spend-over-500/
• gloucestershire-june-2026.csv — https://www.gloucestershire.gov.uk/media/hbapmuig/payments-made-during-june-2026-csv.csv
• gloucestershire-may-2026.csv — https://www.gloucestershire.gov.uk/media/llkdxypa/payments-made-during-may-2026-csv.csv
• Editions: May and June 2026. Published within 20 working days of month-end.
• Caveat: payment amounts are negative (credit-side sign convention); includes supplier industry/account-group classification columns others lack.

Sefton Council (metropolitan borough)

• Landing page: https://www.sefton.gov.uk/your-council/transparency/transparency/council-spend-other-key-documents/council-spend-over-500/
• sefton-april-2026.csv — https://www.sefton.gov.uk/media/uxdlq5b0/sefton-council-supplier-spend-april-2026.csv
• sefton-march-2026.csv — https://www.sefton.gov.uk/media/lq3jjvbs/sefton-council-supplier-spend-march-2026.csv
• Editions: March and April 2026 (April was the latest published at download time — Sefton lags 3–4 months).
• Caveat: first row is a title line, headers on row 2. Excludes staff payments, housing benefits, financing/investments, and redacted social-care/tax items.

Re-download

cd research/datasets/11-transparency-code-spend/data
curl -sL -o adur-june-2026.csv "https://www.adur-worthing.gov.uk/media/Media,174581,smxx.csv"
curl -sL -o worthing-june-2026.csv "https://www.adur-worthing.gov.uk/media/Media,174582,smxx.csv"
curl -sL -o adur-worthing-joint-june-2026.csv "https://www.adur-worthing.gov.uk/media/Media,174583,smxx.csv"
curl -sL -o gloucestershire-june-2026.csv "https://www.gloucestershire.gov.uk/media/hbapmuig/payments-made-during-june-2026-csv.csv"
curl -sL -o gloucestershire-may-2026.csv "https://www.gloucestershire.gov.uk/media/llkdxypa/payments-made-during-may-2026-csv.csv"
curl -sL -o sefton-april-2026.csv "https://www.sefton.gov.uk/media/uxdlq5b0/sefton-council-supplier-spend-april-2026.csv"
curl -sL -o sefton-march-2026.csv "https://www.sefton.gov.uk/media/lq3jjvbs/sefton-council-supplier-spend-march-2026.csv"

Note the URLs contain opaque media IDs; new months get new IDs, so re-scrape the landing pages each month rather than guessing URLs.

Granularity, coverage, cadence

• Granularity: per transaction (supplier-level), one file per council per month.
• Coverage: England only (the Transparency Code applies to English local authorities; Scotland/Wales/NI have separate, non-mandatory regimes).
• Cadence: monthly per council, typically published 1–4 months in arrears. "Next edition" is council-specific: Adur & Worthing and Gloucestershire were current to June 2026 in mid-August 2026; Sefton to April 2026.

Licence

• The Transparency Code requires publication under the Open Government Licence, and Leeds' mirror on data.gov.uk is explicitly tagged "UK Open Government Licence (OGL)".
• Gloucestershire and Sefton's spend pages carry no explicit licence statement — OGL is the Code's default expectation but is unconfirmed on those specific pages. Adur & Worthing's open-data section likewise did not show an explicit statement on the fetched page.

Blockers encountered

• dorsetcouncil.gov.uk returned HTTP 403 to non-browser fetches.
• datamillnorth.org (Leeds) returned 403 / a ~5KB challenge page to curl and WebFetch; recent Leeds files would need browser access (the stale data.gov.uk CKAN mirror only reaches May 2021).

What a full collection would take

~317 English councils × 12 months/yr, each with a bespoke page structure, schema, sign convention and redaction policy. Practical routes: (a) a scraper per council (or per CMS family — many share Jadu/Umbraco patterns), with browser automation for Cloudflare-fronted sites; (b) a commercial/aggregated source such as Spend Network (third-party, out of scope for this corpus's official-sources rule); (c) restrict analysis to a fixed panel of councils chosen from CIPFA nearest-neighbour groups and collect those manually each quarter. Any cross-council analysis must first normalise sign conventions, VAT treatment and redaction differences.