S3U

Geography / Grid maps and scale / Graduate / gge01

Validate Spatial Predictions for Their Intended Use

Design cross-validation that matches geographic deployment.

All worksheets
5 questions0m 0s

Prerequisites

Regression, train-test separation, spatial autocorrelation, and prediction error.

Learn the skill

Define whether the model will interpolate nearby or transfer to distant regions. Nearby train and test observations can share information through spatial dependence. Choose folds, buffers, and preprocessing rules that reflect the deployment question.

Worked example

A random point split can test prediction near sampled sites while a regional holdout tests geographic transfer. Neither number is universally correct without specifying the target use.

Model and Assumptions

An invented mapping project will predict in unsampled regions. Random folds yield RMSE 2; whole-region holdouts yield RMSE 7. The team selected hyperparameters after repeatedly checking those same region holdouts. Some predictors were imputed using all regions before splitting.

1. Which split more closely resembles the intended transfer task?
2. Why is RMSE 7 not a clean final estimate after repeated tuning on those holdouts?
3. Where should imputation parameters be learned within a fold?
4. What design supports tuning and a final transfer estimate?
5. What additional limit remains even after careful spatial validation?

Further inquiry

Specify a nested fold scheme, including buffers, preprocessing, and hyperparameter selection. Define the target geography and explain how you would report uncertainty across regions without treating neighboring pixels as independent replicates.

Review criteria

  • Keep outer outcomes isolated from every tuning step.
  • Justify spatial units using dependence and deployment context.
  • Explain remaining extrapolation limits.