Measuring Inclusive Education with UNESCO's SDG 4 Data

28 August 2024

How do you measure whether inclusive education works? I've been trying to answer that with data from UNESCO's Sustainable Development Goal 4 (SDG 4) database, which the UNESCO Institute for Statistics publishes. This post is the first in a short series on that work.

The data

We looked at 138 countries between 2013 and 2023 and focused on three areas:

completion rates at each level of education,
accessibility of school infrastructure,
gender parity indices.

The database is global, but the number of usable data points per metric is small:

# Distribution of data points across metrics
metrics_distribution = {
    'Infrastructure metrics': '300+ observations',
    'Completion rates': '10-12 observations per metric',
    'Gender parity indices': '13 observations per metric'
}

Correlations between students with and without disabilities

We compared completion rates for students with and without disabilities:

Primary: r = 0.234 (p = 0.464, n = 12)
Lower secondary: r = 0.281 (p = 0.376, n = 12)
Upper secondary: r = 0.081 (p = 0.823, n = 10)

The correlations are weak, and with 10 to 12 countries per level none of them is statistically significant. So a country doing well for students without disabilities isn't necessarily doing well for students with disabilities. That's a real question for how well current inclusive education policies work.

Working with the data

Small samples. Reporting is inconsistent, so many metrics have only a handful of observations. We used bootstrap resampling to get confidence intervals:

def bootstrap_correlation(x, y, n_iterations=10000):
    correlations = []
    for _ in range(n_iterations):
        idx = np.random.randint(0, len(x), len(x))
        r = stats.pearsonr(x[idx], y[idx])[0]
        correlations.append(r)

    ci = np.percentile(correlations, [2.5, 97.5])
    return np.mean(correlations), ci

Uneven data quality. Countries collect and report data differently. We weighted the analysis by data reliability scores.

Missing years. Countries don't all report in the same years, which makes trends hard to compare. We used sliding time windows:

def analyze_time_window(data, window_size=3):
    """
    Analyze data within sliding time windows to handle temporal inconsistency
    """
    windows = []
    for year in range(data['Year'].min(), data['Year'].max() - window_size + 1):
        window_data = data[(data['Year'] >= year) & 
                          (data['Year'] < year + window_size)]
        windows.append({
            'period': f'{year}-{year+window_size}',
            'mean': window_data['Value'].mean(),
            'std': window_data['Value'].std(),
            'n_countries': window_data['Country'].nunique()
        })
    return pd.DataFrame(windows)

What we found

Infrastructure. Schools with adapted infrastructure have consistently higher completion rates. The effect is strongest in primary education and varies a lot by region.

Gender. Gender parity is improving over time. Where gender and disability intersect, the patterns are complex. Regional differences in gender parity are larger in secondary education.

Completion gaps. The gap between students with and without disabilities grows at higher levels of education. Some regions have consistently smaller gaps, which suggests they are doing something right. Economic factors are strongly correlated with the size of the gap.

What this means for policy

Start early. The gap grows as students move up, so support should start in primary school.
Invest in infrastructure. Adapted infrastructure is clearly linked to higher completion rates.
Learn from the regions that do well. Regions with smaller gaps are worth studying.