Skip to content

Forecast Verification: Scalar, Wind and Probability

Verify matched scalar, wind-direction and probability forecasts with error scores, common-reference skill, lead groups and reliability bins.

Use this result well

Inputs that matter
Variable, level, unit and interval, Forecast cases, Sources and assumptions, Wind level, speed unit and averaging period, and 3 more
Output to expect
Matched scalar forecast scores, Wind-direction forecast errors, Brier score and reliability bins
  • Check the units and required inputs before comparing results.
  • Keep the assumptions with a copied result so you can reproduce the calculation later.
Was this tool helpful?

Related Tools

Tools you might need next

Weather Observations and Paired Sensor Checks

Record timestamped weather observations, trace rain and exact pressure changes, or compare co-timed sensors with bias, MAE and RMSE summaries.

Recommended next stepMeteorology
Daily Climate Records and Matched Normal Comparisons

Summarize daily climate coverage, rain and degree days, then compare observations with supplied normals on the same matched calendar dates.

Recommended next stepMeteorology
UV Index Estimator

Estimate UV index from solar zenith angle, sun factor, and ozone transmission. Free uv index forecast tool for index, forecast, and more.

Meteorology

Reference & details

How it works

Matched scalar forecast scores

Verify actual forecast/observation pairs with bias, MAE, RMSE, correlation and sample counts. Compare reference skill only on their common sample, with separate group and exact lead-time summaries.

Error = forecast − observation; bias = mean(error); MAE = mean(|error|); RMSE = √mean(error²); MSE skill = 1 − MSEforecast/MSEreference on identical cases.

Wind-direction forecast errors

Compare wind FROM directions on the same reference using the shortest signed angular difference. Exclude cases where either wind is exactly calm; keep missing and calm counts separate.

Direction error = wrap(forecast − observed) to [−180°, 180°); circular mean = atan2(mean sin(error), mean cos(error)). MAE/RMSE use shortest errors.

Brier score and reliability bins

Score actual probabilities against a binary observed event. Brier scores use the original probabilities; ten displayed reliability bins summarize counts, mean probability and observed frequency without changing the scoring inputs.

Brier = mean((probability − event)²); Brier skill = 1 − Brierforecast/Brierreference on common cases.

Updated: September 2026

Example Scenarios

Inspect the example and its input basis, then substitute your own documented measurements or matched cases.

Variable, level, unit and interval: 2 m air temperature, °C, instantaneousForecast cases: Station A | 2026-09-01T00:00:00Z | 2026-09-01T06:00:00Z | 22 | 20 | 18 | dry Station A | 2026-09-02T00:00:00Z | 2026-09-02T06:00:00Z | 18 | 19 | 17 | dry Station A | 2026-09-03T00:00:00Z | 2026-09-03T06:00:00Z | M | 21 | 19 | dry

→ 1.581139 RMSE (2 matched cases)

Inspect the example and its input basis, then substitute your own documented measurements or matched cases.

Wind level, speed unit and averaging period: 10 m, m/s, true FROM direction, 10-minute meanWind forecast cases: Station A | 2026-09-01T00:00:00Z | 2026-09-01T06:00:00Z | 5 | 10 | 5 | 350 | all Station A | 2026-09-02T00:00:00Z | 2026-09-02T06:00:00Z | 0 | M | 2 | 20 | all

→ 20° direction MAE (1 non-calm pairs)

Inspect the example and its input basis, then substitute your own documented measurements or matched cases.

Event definition, place and interval: Station daily precipitation at least 1 mm, 00–24 UTCProbability cases: Station A | 2026-09-01T00:00:00Z | 2026-09-02T00:00:00Z | 0.8 | 1 | 0.5 | all Station A | 2026-09-02T00:00:00Z | 2026-09-03T00:00:00Z | 0.2 | 0 | 0.5 | all

→ 0.04 Brier score (2 cases)

Common Mistakes to Avoid

Mixing observation and model inputs

Use the exact units, timestamp, level and measurement or model basis stated by the selected mode. A similar quantity from another instrument or product is not automatically interchangeable.

Treating an illustrative case as a measurement

Replace the example with your own sourced inputs. Retain missing values and method limits, and keep raw source data alongside a saved calculation.

FAQ

Verify actual forecast/observation pairs with bias, MAE, RMSE, correlation and sample counts. Compare reference skill only on their common sample, with separate group and exact lead-time summaries. Compare wind FROM directions on the same reference using the shortest signed angular difference. Exclude cases where either wind is exactly calm; keep missing and calm counts separate. Score actual probabilities against a binary observed event. Brier scores use the original probabilities; ten displayed reliability bins summarize counts, mean probability and observed frequency without changing the scoring inputs.

Retain the station or profile identity, observation or issue/valid time, measurement units, quality flags, source and stated method. Missing values, trace precipitation and known zeros have different meanings.

Save weather case keeps an explicit local record. Copy MD includes current inputs and notes; Download CSV includes the current result breakdown. Keep portable copies separately and retain raw source data.

About Forecast Verification: Scalar, Wind and Probability

Verify actual forecast/observation pairs with bias, MAE, RMSE, correlation and sample counts. Compare reference skill only on their common sample, with separate group and exact lead-time summaries. Compare wind FROM directions on the same reference using the shortest signed angular difference. Exclude cases where either wind is exactly calm; keep missing and calm counts separate. Score actual probabilities against a binary observed event. Brier scores use the original probabilities; ten displayed reliability bins summarize counts, mean probability and observed frequency without changing the scoring inputs.