Match real forecasts to compatible observations, inspect scalar, wind and probability errors, and compare reference skill on common cases.
Choose one variable, unit, height, location footprint and valid interval per run. Record the forecast model/version, issue time, lead time and a reference forecast that would have been available then. Define any weather-regime grouping before reviewing performance so the verification question does not shift to favor one model.
Join by station or extraction location, valid time, level and accumulation interval. Keep issue time separate from valid time and use explicit UTC. Retain missing values and quality flags. When several forecast cycles verify at the same time, they must use the same observation; these repeated cases are not independent samples.
Calculate forecast-minus-observation bias, MAE and RMSE together with sample count. Review group and exact-lead summaries rather than comparing unlike lead distributions. Pearson correlation is undefined for a constant series or fewer than two pairs. Reference skill must compare model and baseline on the identical common subset.
Match wind direction references and averaging periods. Compute the shortest angular difference across north instead of subtracting 350° from 10° as a 340° miss. Exclude cases where either wind is calm and retain missing counts. A zero circular resultant has no unique mean error.
State the threshold, footprint and time interval before coding observed events as 0 or 1. Use the original probabilities for Brier score and a documented reference for Brier skill. Reliability bins show observed event frequency and count; they must not replace exact probabilities with bin centers in the score.
Save input cases, missing-data rules, model/reference provenance and grouping choices. Compare additional periods or stations when the question requires them, and assess uncertainty with an appropriate statistical method outside these descriptive scores. Do not invent universal model MAE targets or claim a small sample proves one model superior.
Validate unique station/issue/valid identities and expose missing forecast/observation pairs. · Calculate scalar errors and common-sample MSE skill with a visible zero-reference-error case. · Score non-calm wind FROM directions with angular MAE/RMSE and a circular mean. · Calculate Brier score, common-reference skill and reliability-bin counts from actual probabilities. · Save separate verification cases and export the original inputs and score breakdown.
Review the observation source and measurement basis before scoring.
Use compatible supplied reference normals when the chosen baseline requires them.
Loading your checklist…
MET documents continuous error scores, correlation, MSE skill, Brier score and reference skill definitions.
Verification depends on actual observations with compatible sampling and quality context.
Skill relative to a perfect reference has a zero denominator and must remain undefined.
Multiple leads or forecast cycles can share one observed event; a large row count is not automatically a large independent sample.