1. Overview

This dataset provides longitudinal financial data — total income, total expenditure, and detailed breakdowns — for registered charities across the United Kingdom. It was produced by the UK Third and Civil Society Sector Database project, which collects, processes, and links public administrative data on civil society organisations across the United Kingdom.

The data is drawn from the annual returns and financial reports filed by charities with their respective regulators: the Charity Commission for England and Wales (CCEW), the Office of the Scottish Charity Regulator (OSCR), and the Charity Commission for Northern Ireland (CCNI). These regulatory filings are publicly available and provide a consistent, official record of charity finances over time.

The dataset consolidates these three sources into a single standardised format, covering over 305,000 charities and spanning financial years from 2000 to 2025. Each record represents one charity’s financial return for a single financial year, enabling longitudinal analysis of the UK charity sector’s financial health at scale.

2. What are Charity Financial Records?

Registered charities in the United Kingdom are required by law to file annual financial information with the charity regulator in their jurisdiction. The nature and detail of these filings varies by regulator and by charity size, but all returns include at minimum the charity’s total income and total expenditure for a given financial year.

England and Wales (CCEW)

Charities registered with the Charity Commission for England and Wales submit annual returns that include headline financial data (Part A) and, for charities with income exceeding £500,000, a detailed financial breakdown (Part B). Part B provides granular information on sources of income (donations, trading, investments, etc.) and categories of expenditure (raising funds, charitable activities, governance, etc.). The source data is available from the CCEW full register download.

Scotland (OSCR)

Charities registered with the Office of the Scottish Charity Regulator submit annual monitoring returns. OSCR publishes a rolling five-year extract of the Scottish Charity Register that includes total income and total expenditure, as well as a subset of detailed income and expenditure categories (donations and legacies, charitable activities, other trading activities, investments, other income; raising funds spending, charitable activities spending, and other spending). The source data is available from the OSCR register download.

Northern Ireland (CCNI)

Charities registered with the Charity Commission for Northern Ireland submit annual reports that include headline financial figures. Some CCNI records include detailed income and expenditure categories similar to the CCEW Part B data. CCNI publishes a current snapshot of the register and historical snapshots are archived periodically. The source data is available from the CCNI charity search.

3. Dataset Contents

The dataset contains 3,998,722 records across 23 fields. Each row represents one charity’s financial return for a single financial year. The fields are organised into four groups: identifiers, aggregate financials, detailed breakdowns, and metadata.

Identifiers and Time Period

Aggregate Financials

Income Breakdown

Expenditure Breakdown

Metadata

4. Coverage & Completeness

Core Field Coverage

The core fields — uid, fy, fye, inc, and exp — are present in virtually all records. These fields are required for inclusion in the dataset; records missing a valid charity number, financial year end date, or income value are excluded during processing.

Jurisdiction Breakdown

The dataset draws from all three UK charity regulators. England and Wales accounts for the large majority of records, reflecting the greater number of charities registered with CCEW.

Year-by-Year Coverage

The dataset covers financial years from 2000 to 2025, with peak coverage between 2015 and 2022 (170,000–181,000 records per year). Coverage in recent years declines due to reporting lag — charities have up to 10 months after their financial year end to file their annual return, so the most recent years will always be incomplete at any given extraction date.

Detailed Financial Data Availability

The 15 detailed income and expenditure fields are populated for approximately 5–6% of all records. Detailed breakdowns are available from:

• CCEW Part B filers — charities with annual income exceeding £500,000, which are required to submit detailed financial breakdowns as part of their annual return. This provides the fullest set of detailed fields (all 15 columns).
• OSCR — the public extract includes a subset of detailed categories under different column names: 5 income categories (donations and legacies, charitable activities, other trading activities, investments, other) and 3 expenditure categories (raising funds, charitable activities, other). The remaining detailed fields (legacies, endowments, investment management, grants, governance, support costs, depreciation) are not available from OSCR.
• Some CCNI records — certain CCNI filings include detailed income and expenditure categories.

Although the detailed fields cover a small proportion of all records, the charities that do report detailed breakdowns tend to be the largest in the sector. The roughly 234,000 records with detailed data represent the most financially significant charities in England, Wales, Scotland, and Northern Ireland.

5. What Can You Learn?

This dataset enables a wide range of analyses of the UK registered charity sector. Below are some illustrative use cases, several of which have already been demonstrated using this data in published research.

Sector-Level Trends

Track aggregate income and expenditure across the entire UK charity sector over 25 years. Analyse how the sector responded to economic shocks such as the 2008 financial crisis, the period of austerity, and the COVID-19 pandemic. Identify long-term growth or contraction trends in charity finances. When combined with CPI deflators, the data supports both nominal and real-terms (inflation-adjusted) time-series analysis.

Source: Charity Financial Records dataset. All values in 2023 prices, adjusted using ONS CPI (series D7BT). 2023–2024 figures are provisional due to reporting lag; the sharp rise in 2024 median income reflects incomplete filing by smaller charities.

Financial Health Analysis

Identify charities with declining income over consecutive years, or those where expenditure consistently exceeds income. For example, deriving a “loss-making” indicator (where expenditure exceeds income in a given year) enables analysis of financial distress rates across regions and over time. Annual relative growth rates can be calculated to track the year-on-year trajectory of individual charities or cohorts.

Jurisdictional Comparisons

Compare financial patterns across England and Wales, Scotland, and Northern Ireland. Examine whether charities in different jurisdictions show different income levels, growth rates, or financial resilience during economic downturns.

Subsector Analysis

When linked to the TCSS Organisation Register, analyse finances by charity type, activity area, or geographic region. For example, the data can be joined with the UK Charity Activity Tags (UKCAT) ICNPTSO classification to compare median income and expenditure trends across charity sectors such as social services, health, education, culture and recreation, and environment.

Median annual relative income growth by ICNPTSO sector, nominal and real (2023 prices). Source: UK Third and Civil Society Sector Database.

Donation Dependency

For charities with detailed income breakdowns, the proportion of income from donations and legacies can be tracked over time and compared across jurisdictions. This enables analysis of how reliant different parts of the charity sector are on voluntary income, and whether that reliance has changed in response to policy or economic shifts.

Geographic Analysis

When linked to the Organisation Register’s geographic data (postcodes, local authorities, regions), financial data can be mapped at the local authority level. This supports analysis of regional variation in charity finances, the relationship between local deprivation and charity financial health, and per-capita measures of charitable activity.

Income Composition

For the larger charities with detailed breakdowns, examine how income sources are distributed — for example, the relative importance of donations, trading income, charitable activities, and investment returns. Track whether charities have diversified their income sources over time or become more dependent on a single source.

6. Limitations & Caveats

Reporting Lag

Charities file their annual returns after their financial year ends, with deadlines varying by jurisdiction (typically 10 months for CCEW). As a result, the most recent financial years in the dataset will always be incomplete at any given extraction date. The sharp decline in records for 2024 and 2025 reflects this reporting lag, not a decline in the number of active charities.

Detailed Breakdowns

The 15 detailed income and expenditure fields are only available for a subset of records (approximately 5–6%). CCEW Part B provides the fullest detail (all 15 fields), while OSCR provides a subset of 8 categories and some CCNI records include partial detail. The remaining detailed fields (e.g., legacies, endowments, governance, support costs) are only available from CCEW. Analyses using these fields cannot be considered representative of the full sector.

OSCR Rolling Window

OSCR publishes a rolling five-year extract of the Scottish Charity Register. Historical OSCR data beyond the most recent five-year window is only available if it was captured in a previous extraction. The dataset’s OSCR coverage for earlier years depends on which baseline snapshots were available during data collection.

Variable Financial Year Periods

Charities do not all operate on the same financial year. While many use an April–March cycle, others use calendar years (January–December) or other reporting periods. The fy field represents the calendar year in which the financial period ends, enabling comparison across charities, but users should be aware that the actual reporting periods may not align precisely.

Spine Filtering

The dataset only includes charities that appear in the TCSS Organisation Register (the “Spine”). Some recently registered or very small charities may not yet have been added to the Spine and will therefore be absent from this dataset. The exclusion log records all filtered records and the reason for their exclusion.

Currency and Inflation

All financial values are reported in nominal terms (current prices) as filed by the charity. No inflation adjustment has been applied. Users conducting time-series analysis should apply their own deflators if real-terms comparison is needed.

What is NOT in the Data

The dataset does not include: balance sheet items (assets, liabilities, reserves), staff numbers or employment data, trustee or director details, geographic location, or activity classifications. Many of these gaps can be addressed by linking to the wider data resources of the UK Third and Civil Society Sector Database, which provides organisation registers, CIC 36 forms, and procurement data for the same organisations.

7. Citation & Licence

Suggested Citation

McDonnell, D. et al. (2026). Charity Financial Records. UK Third and Civil Society Sector Database. Available at: https://uk-third-sector-database.github.io/data/. Licensed under CC BY 4.0.

If you would like to learn more about this dataset and how it can be applied to your project or research programme, please contact research@brawdata.com.

8. Changelog

A1. Pipeline Architecture

The charity financial records pipeline processes publicly available regulatory data from three UK charity regulators and consolidates it into a single standardised dataset. The pipeline is implemented in Python and is orchestrated through a command-line interface (run_pipeline.py) that supports running individual jurisdiction processors or the full workflow.

The pipeline operates in four phases:

1. Jurisdiction Processing — Each regulator’s source data is processed by a dedicated module (process_ccew.py, process_oscr.py, process_ccni.py). These modules parse the raw files, validate required fields, standardise dates and financial values, construct UIDs, and deduplicate within-source records. Each module produces a jurisdiction-specific extract CSV.
2. Combination — The three jurisdiction extracts are concatenated into a single dataset. Records are filtered against the TCSS Organisation Register (“Spine”) to retain only charities that appear in the cross-register lookup. This ensures consistency with other datasets in the TCSS ecosystem.
3. Baseline Append — New records are compared against an existing baseline dataset using a composite key of uid + fy. Only genuinely new records are appended, preventing duplicate accumulation across pipeline runs.
4. Output Validation — The final dataset is validated against the defined output schema (23 columns), sorted by uid and fy, and written to CSV.

All excluded records are logged to an exclusion log (charity-exclusion-log.csv) with a timestamp, source, reason code, and descriptive detail, providing a full audit trail of data processing decisions.

A2. Source Data

The pipeline draws on publicly available data from three regulatory sources. Each source has distinct file formats, date conventions, and field structures that are handled by dedicated processing modules.

CCEW (England & Wales)

OSCR (Scotland)

CCNI (Northern Ireland)

A3. Deduplication & Validation

Data quality is maintained through a multi-stage validation and deduplication process. Each stage logs excluded records to the exclusion log with a reason code.

Required Field Validation

Every record must have:

• A valid charity registration number (used to construct the uid)
• A parseable financial year end date (used to derive fy and fye)
• A non-null income value (inc)

Records failing any of these checks are excluded with reason code missing_required or invalid_date.

Within-Source Deduplication

Some source files contain duplicate records for the same charity and financial year. The deduplication strategy varies by source:

• CCEW Part A: When multiple records exist for the same (uid, fy) pair, the record with the most recent ar_received_date is retained, as it reflects the latest submission.
• OSCR and CCNI: The first occurrence for each (uid, fy) pair is retained.

Excluded duplicates are logged with reason code duplicate_source.

Spine Filtering

After combining the three jurisdiction extracts, records are filtered against the TCSS Organisation Register (Spine). Only charities with a UID present in the Spine are retained. This ensures the dataset is consistent with other TCSS datasets and that all records can be linked using the common uid field. Excluded records are logged with reason code not_in_spine.

Baseline Deduplication

When appending new records to the existing baseline dataset, a composite key of uid + fy is used to identify records that already exist. Only genuinely new records are appended, preventing duplicate accumulation across pipeline runs. Duplicates are logged with reason code duplicate_baseline.

Exclusion Reason Codes

A4. Reproducibility

The pipeline code is available in the project’s repository, with the charity financial records pipeline located in the code/charities/ directory.

Scripts

Dependencies

The pipeline requires Python 3.11 or later, with dependencies managed using uv. The only external dependency is:

• pandas — for data manipulation and CSV I/O

Running the Pipeline

The full pipeline can be run with a single command:

python run_pipeline.py all

Individual stages can also be run independently:

python run_pipeline.py ccew      # Process CCEW data only
python run_pipeline.py oscr      # Process OSCR data only
python run_pipeline.py ccni      # Process CCNI data only
python run_pipeline.py combine   # Combine and append
python run_pipeline.py coverage  # Generate coverage reports

Input Data

Running the pipeline requires the source data files to be downloaded from each regulator’s website and placed in the expected directory structure under data/input/financial-history/raw/. The baseline dataset and Spine lookup file must also be present. All input paths are defined in config.py.