Predicting Education Completion Rates

10 September 2024

This is the second post in my series on UNESCO's SDG 4 data (the first one covers the data itself). Here I describe how we built models to predict completion rates, and what we had to change to make standard methods work on education data.

The task

We predict completion rates at three levels of education, taking disability status into account. The features fall into three groups:

# Key features in our prediction task
feature_categories = {
    'Infrastructure': [
        'adapted_infrastructure_percentage',
        'accessibility_score',
        'learning_materials_availability'
    ],
    'Economic': [
        'education_funding_per_student',
        'gdp_per_capita',
        'unemployment_rate'
    ],
    'Social': [
        'teacher_training_level',
        'parent_engagement_score',
        'community_support_index'
    ]
}

Feature engineering and missing data

We added time-based features (years since the country's first record), interaction terms such as infrastructure × funding, and regional averages so each country can be compared with its region. Missing values are filled differently by type: regional averages for infrastructure metrics, interpolation over time for economic indicators.

def prepare_features(df):
    # Create time-based features
    df['years_of_inclusion'] = df.groupby('Country')['Year'].transform(
        lambda x: x - x.min())

    # Generate interaction terms
    df['infrastructure_funding'] = (
        df['adapted_infrastructure_percentage'] * 
        df['education_funding_per_student']
    )

    # Create regional aggregates
    df['region_completion_mean'] = df.groupby('Region')['completion_rate'].transform('mean')
    df['country_vs_region'] = df['completion_rate'] - df['region_completion_mean']

    return df

# Handle missing values using domain-specific knowledge
def impute_missing_values(df):
    # Use regional averages for infrastructure metrics
    for col in infrastructure_cols:
        df[col].fillna(df.groupby('Region')[col].transform('mean'), inplace=True)

    # Use temporal interpolation for economic indicators
    for col in economic_cols:
        df[col] = df.groupby('Country')[col].interpolate(method='time')

    return df

Models

We compared random forest, lasso regression and XGBoost:

from sklearn.ensemble import RandomForestRegressor
from sklearn.linear_model import LassoCV
from xgboost import XGBRegressor

def train_evaluate_models(X_train, y_train, X_test, y_test):
    models = {
        'random_forest': RandomForestRegressor(
            n_estimators=100,
            max_depth=None,
            min_samples_leaf=5
        ),
        'lasso': LassoCV(
            cv=5,
            random_state=42
        ),
        'xgboost': XGBRegressor(
            learning_rate=0.05,
            n_estimators=100,
            max_depth=6
        )
    }

    results = {}
    for name, model in models.items():
        model.fit(X_train, y_train)
        predictions = model.predict(X_test)
        results[name] = {
            'rmse': mean_squared_error(y_test, predictions, squared=False),
            'r2': r2_score(y_test, predictions),
            'mae': mean_absolute_error(y_test, predictions)
        }

    return results

Results

Accuracy goes down as the education level goes up:

performance_metrics = {
    'primary': {
        'RMSE': 8.2,
        'R²': 0.76,
        'MAE': 6.5
    },
    'lower_secondary': {
        'RMSE': 9.8,
        'R²': 0.71,
        'MAE': 7.8
    },
    'upper_secondary': {
        'RMSE': 11.3,
        'R²': 0.68,
        'MAE': 9.1
    }
}

The most important features are different at each level.

Primary education:

teacher training level (0.28)
adapted infrastructure (0.25)
parent engagement (0.18)

Secondary education:

economic indicators (0.31)
infrastructure accessibility (0.24)
community support (0.17)

Imbalance and bias

The data is unbalanced. SMOTE normally works on classes, so we bin the continuous target, oversample, and convert back:

def handle_imbalance(X, y):
    # SMOTE for continuous target variable
    from sklearn.preprocessing import KBinsDiscretizer
    from imblearn.over_sampling import SMOTE

    # Discretize the continuous target for balancing
    kbd = KBinsDiscretizer(n_bins=5, encode='ordinal', strategy='quantile')
    y_binned = kbd.fit_transform(y.reshape(-1, 1))

    # Apply SMOTE
    smote = SMOTE(random_state=42)
    X_balanced, y_binned_balanced = smote.fit_resample(X, y_binned)

    # Convert back to continuous
    y_balanced = kbd.inverse_transform(y_binned_balanced)

    return X_balanced, y_balanced

Interpretability

If a model is going to inform policy, people have to be able to see why it predicts what it does. We used SHAP values:

import shap

def explain_predictions(model, X):
    explainer = shap.TreeExplainer(model)
    shap_values = explainer.shap_values(X)

    # Plot feature importance
    shap.summary_plot(shap_values, X)

    # Generate per-instance explanations
    return shap_values

Uses

Early warning: identifying students at risk of not completing, targeting interventions, allocating resources.
Policy planning: prioritizing infrastructure investment and designing teacher training programs.
Monitoring: tracking whether interventions work and measuring the effect of policy.

Next improvements

More data: socioeconomic indicators, health and wellbeing metrics, availability of educational resources.
Better models: deep learning for time patterns, hierarchical models for regional effects, and causal analysis.