Articles | Volume 18, issue 20
https://doi.org/10.5194/gmd-18-7417-2025
© Author(s) 2025. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/gmd-18-7417-2025
© Author(s) 2025. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Methods for assessment of models
| 
20 Oct 2025
Methods for assessment of models |
| 20 Oct 2025
| 20 Oct 2025
Smoothing and spatial verification of global fields
Gregor Skok
CORRESPONDING AUTHOR
University of Ljubljana, Faculty of Mathematics and Physics, Jadranska Cesta 19, 1000 Ljubljana, Slovenia
Katarina Kosovelj
University of Ljubljana, Faculty of Mathematics and Physics, Jadranska Cesta 19, 1000 Ljubljana, Slovenia
Related authors
Marco Stefanelli, Žiga Zaplotnik, and Gregor Skok
EGUsphere, https://doi.org/10.48550/arXiv.2512.18289, https://doi.org/10.48550/arXiv.2512.18289, 2026
This preprint is open for discussion and under review for Geoscientific Model Development (GMD).
Short summary
Short summary
Weather radars provide storm intensity and location, but weather forecasting systems do not readily use them. We trained a neural network on 5 years of reflectivity radar and model output data to map model fields into radar reflectivity space, allowing forecasts to be corrected with radar data. In a major flood case, this cut errors in storm position and strength. Broadly speaking, the methodology provides a simplified solution for assimilating observations with no direct model-equivalent field.
Sabin Roman, Gregor Skok, Ljupčo Todorovski, and Sašo Džeroski
EGUsphere, https://doi.org/10.48550/arXiv.2508.11307, https://doi.org/10.48550/arXiv.2508.11307, 2026
This preprint is open for discussion and under review for Geoscientific Model Development (GMD).
Short summary
Short summary
This study aimed to improve how the Universal Thermal Climate Index, a key measure of human thermal comfort, is calculated. Existing methods use a simplified polynomial approximation that is straightforward to apply but can introduce errors. We developed a new version using sparse regression with orthogonal polynomials, which keeps computational efficiency while improving accuracy and stability. The results enable more reliable assessments of outdoor thermal comfort and climate analyses.
Marco Stefanelli, Žiga Zaplotnik, and Gregor Skok
EGUsphere, https://doi.org/10.48550/arXiv.2512.18289, https://doi.org/10.48550/arXiv.2512.18289, 2026
This preprint is open for discussion and under review for Geoscientific Model Development (GMD).
Short summary
Short summary
Weather radars provide storm intensity and location, but weather forecasting systems do not readily use them. We trained a neural network on 5 years of reflectivity radar and model output data to map model fields into radar reflectivity space, allowing forecasts to be corrected with radar data. In a major flood case, this cut errors in storm position and strength. Broadly speaking, the methodology provides a simplified solution for assimilating observations with no direct model-equivalent field.
Sabin Roman, Gregor Skok, Ljupčo Todorovski, and Sašo Džeroski
EGUsphere, https://doi.org/10.48550/arXiv.2508.11307, https://doi.org/10.48550/arXiv.2508.11307, 2026
This preprint is open for discussion and under review for Geoscientific Model Development (GMD).
Short summary
Short summary
This study aimed to improve how the Universal Thermal Climate Index, a key measure of human thermal comfort, is calculated. Existing methods use a simplified polynomial approximation that is straightforward to apply but can introduce errors. We developed a new version using sparse regression with orthogonal polynomials, which keeps computational efficiency while improving accuracy and stability. The results enable more reliable assessments of outdoor thermal comfort and climate analyses.
Cited articles
Baddeley, A.: An error metric for binary images, in: Robust Computer Vision: Quality of Vision Algorithms, Wichmann verlag, Karlsruhe, 1992. a
Bentley, J.: Multidimensional binary search trees used for associative searching, Commun. ACM, 18, 509–517, https://doi.org/10.1145/361002.361007, 1975. a
Bentley, J.: Multidimensional Binary Search Trees in Database Applications, IEEE T. Software. Eng., SE-5, 333–340, https://doi.org/10.1109/TSE.1979.234200, 1979. a, b
Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., and Tian, Q.: Accurate medium-range global weather forecasting with 3D neural networks, Nature, 619, https://doi.org/10.1038/s41586-023-06185-3, 2023. a, b
Bouallègue, Z. B., Theis, S. E., and Gebhardt, C.: Enhancing COSMO-DE ensemble forecasts by inexpensive techniques, Meteorol. Z., 22, 49–59, https://doi.org/10.1127/0941-2948/2013/0374, 2013. a
Brown, B. G., Gilleland, E., and Ebert, E. E.: Forecasts of Spatial Fields, in: Forecast Verification, John Wiley and Sons, Ltd, Chichester, UK, 95–117, https://doi.org/10.1002/9781119960003.ch6, 2012. a
Brown, R. A.: Building a Balanced k-d Tree in O(knlog n) Time, J. Comput. Graph. Tech., 4, 50–68, http://jcgt.org/published/0004/01/03/ (last access: 13 October 2025), 2015. a
Buschow, S. and Friederichs, P.: SAD: Verifying the scale, anisotropy and direction of precipitation forecasts, Q. J. Roy. Meteor. Soc., 147, 1150–1169, https://doi.org/10.1002/qj.3964, 2021. a
Casati, B.: New Developments of the Intensity-Scale Technique within the Spatial Verification Methods Intercomparison Project, Weather Forecast., 25, 113–143, https://doi.org/10.1175/2009WAF2222257.1, 2010. a
Casati, B., Ross, G., and Stephenson, D. B.: A new intensity‐scale approach for the verification of spatial precipitation forecasts, Meteorol. Appl., 11, 141–154, https://doi.org/10.1017/S1350482704001239, 2004. a
Casati, B., Lussana, C., and Crespi, A.: Scale‐separation diagnostics and the Symmetric Bounded Efficiency for the inter‐comparison of precipitation reanalyses, Int. J. Climatol., 43, 2287–2304, https://doi.org/10.1002/joc.7975, 2023. a
Davis, C., Brown, B., and Bullock, R.: Object-Based Verification of Precipitation Forecasts. Part I: Methodology and Application to Mesoscale Rain Areas, Mon. Weather Rev., 134, 1772–1784, https://doi.org/10.1175/MWR3145.1, 2006a. a
Davis, C., Brown, B., and Bullock, R.: Object-Based Verification of Precipitation Forecasts. Part II: Application to Convective Rain Systems, Mon. Weather Rev., 134, 1785–1795, https://doi.org/10.1175/MWR3146.1, 2006b. a
Davis, C. A., Brown, B. G., Bullock, R., and Halley-Gotway, J.: The Method for Object-Based Diagnostic Evaluation (MODE) Applied to Numerical Forecasts from the 2005 NSSL/SPC Spring Program, Weather Forecast., 24, 1252–1267, https://doi.org/10.1175/2009WAF2222241.1, 2009. a
Dey, S. R. A., Leoncini, G., Roberts, N. M., Plant, R. S., and Migliorini, S.: A Spatial View of Ensemble Spread in Convection Permitting Ensembles, Mon. Weather Rev., 142, 4091–4107, https://doi.org/10.1175/MWR-D-14-00172.1, 2014. a
Dey, S. R. A., Plant, R. S., Roberts, N. M., and Migliorini, S.: Assessing spatial precipitation uncertainties in a convective‐scale ensemble, Q. J. Roy. Meteor. Soc., 142, 2935–2948, https://doi.org/10.1002/qj.2893, 2016. a
Dorninger, M., Gilleland, E., Casati, B., Mittermaier, M. P., Ebert, E. E., Brown, B. G., and Wilson, L. J.: The setup of the MesoVICT project, B. Am. Meteorol. Soc., 99, 1887–1906, https://doi.org/10.1175/BAMS-D-17-0164.1, 2018. a
Duc, L., Saito, K., and Seko, H.: Spatial-temporal fractions verification for high-resolution ensemble forecasts, Tellus A, 65, 18171, https://doi.org/10.3402/tellusa.v65i0.18171, 2013. a, b
Ebert, E. and McBride, J.: Verification of precipitation in weather systems: determination of systematic errors, J. Hydrol., 239, 179–202, https://doi.org/10.1016/S0022-1694(00)00343-7, 2000. a
ECMWF: IFS Documentation CY48R1 – Part III: Dynamics and Numerical Procedures, https://doi.org/10.21957/26F0AD3473, 2023a. a
ECMWF: IFS Documentation CY48R1 – Part IV: Physical Processes, https://doi.org/10.21957/02054F0FBF, 2023b. a
Faggian, N., Roux, B., Steinle, P., and Ebert, B.: Fast calculation of the fractions skill score, MAUSAM, 66, 457–466, https://doi.org/10.54302/mausam.v66i3.555, 2015. a
Friedman, J. H., Bentley, J. L., and Finkel, R. A.: An Algorithm for Finding Best Matches in Logarithmic Expected Time, ACM T. Math. Software, 3, 209–226, https://doi.org/10.1145/355744.355745, 1977. a, b
Gainford, A., Gray, S. L., Frame, T. H. A., Porson, A. N., and Milan, M.: Improvements in the spread–skill relationship of precipitation in a convective‐scale ensemble through blending, Q. J. Roy. Meteor. Soc., 150, 3146–3166, https://doi.org/10.1002/qj.4754, 2024. a, b
Gilleland, E.: A new characterization within the spatial verification framework for false alarms, misses, and overall patterns, Weather Forecast., 32, 187–198, https://doi.org/10.1175/WAF-D-16-0134.1, 2017. a
Gilleland, E., Ahijevych, D., Brown, B. G., Casati, B., and Ebert, E. E.: Intercomparison of Spatial Forecast Verification Methods, Weather Forecast., 24, 1416–1430, https://doi.org/10.1175/2009WAF2222269.1, 2009. a
Keil, C. and Craig, G. C.: A Displacement-Based Error Measure Applied in a Regional Ensemble Forecasting System, Mon. Weather Rev., 135, 3248–3259, https://doi.org/10.1175/MWR3457.1, 2007. a
Keil, C. and Craig, G. C.: A Displacement and Amplitude Score Employing an Optical Flow Technique, Weather Forecast., 24, 1297–1308, https://doi.org/10.1175/2009WAF2222247.1, 2009. a
Konca-Kedzierska, K., Wibig, J., and Gruszczyńska, M.: Comparison and combination of interpolation methods for daily precipitation in Poland: evaluation using the correlation coefficient and correspondence ratio, Meteorol. Hydrol. Wate., https://doi.org/10.26491/mhwm/171699, 2023. a
Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger, P., Fortunato, M., Alet, F., Ravuri, S., Ewalds, T., Eaton-Rosen, Z., Hu, W., Merose, A., Hoyer, S., Holland, G., Vinyals, O., Stott, J., Pritzel, A., Mohamed, S., and Battaglia, P.: Learning skillful medium-range global weather forecasting, Science, 382, 1416–1421, https://doi.org/10.1126/science.adi2336, 2023. a, b
Lang, S., Alexe, M., Chantry, M., Dramsch, J., Pinault, F., Raoult, B., Clare, M. C. A., Lessig, C., Maier-Gerber, M., Magnusson, L., Bouallègue, Z. B., Nemesio, A. P., Dueben, P. D., Brown, A., Pappenberger, F., and Rabier, F.: AIFS – ECMWF's data-driven forecasting system, arXiv [preprint], https://doi.org/10.48550/arXiv.2406.01465, 2024. a, b
Ma, S., Chen, C., He, H., Wu, D., and Zhang, C.: Assessing the Skill of Convection-Allowing Ensemble Forecasts of Precipitation by Optimization of Spatial-Temporal Neighborhoods, Atmosphere-Basel, 9, 43, https://doi.org/10.3390/atmos9020043, 2018. a, b
Malardel, S., Wedi, N., Deconinck, W., Diamantakis, M., Kuehnlein, C., Mozdzynski, G., Hamrud, M., and Smolarkiewicz, P.: A new grid for the IFS, ECMWF Newsletter, 23–28, https://doi.org/10.21957/ZWDU9U5I, 2016. a
Markou, M. and Kassomenos, P.: Cluster analysis of five years of back trajectories arriving in Athens, Greece, Atmos. Res., 98, 438–457, https://doi.org/10.1016/j.atmosres.2010.08.006, 2010. a
Marzban, C., Sandgathe, S., Lyons, H., and Lederer, N.: Three Spatial Verification Techniques: Cluster Analysis, Variogram, and Optical Flow, Weather Forecast., 24, 1457–1471, https://doi.org/10.1175/2009WAF2222261.1, 2009. a
Mittermaier, M., North, R., Semple, A., and Bullock, R.: Feature-Based Diagnostic Evaluation of Global NWP Forecasts, Mon. Weather Rev., 144, 3871–3893, https://doi.org/10.1175/mwr-d-15-0167.1, 2016. a
Mittermaier, M. P.: Using an intensity‐scale technique to assess the added benefit of high‐resolution model precipitation forecasts, Atmos. Sci. Lett., 7, 36–42, https://doi.org/10.1002/asl.127, 2006. a
Mittermaier, M. P.: Is there any skill in daily global precipitation forecasts over the Maritime Continent?, Q. J. Roy. Meteor. Soc., https://doi.org/10.1002/qj.4877, 2025. a, b, c
Necker, T., Wolfgruber, L., Kugler, L., Weissmann, M., Dorninger, M., and Serafin, S.: The fractions skill score for ensemble forecast verification, Q. J. Roy. Meteor. Soc., 150, 4457–4477, https://doi.org/10.1002/qj.4824, 2024. a
Roberts, N.: Assessing the spatial and temporal variation in the skill of precipitation forecasts from an NWP model, Meteorol. Appl., 15, 163–169, https://doi.org/10.1002/met.57, 2008. a, b
Schwartz, C. S., Kain, J. S., Weiss, S. J., Xue, M., Bright, D. R., Kong, F., Thomas, K. W., Levit, J. J., Coniglio, M. C., and Wandishin, M. S.: Toward Improved Convection-Allowing Ensembles: Model Physics Sensitivities and Optimizing Probabilistic Guidance with Small Ensemble Membership, Weather Forecast., 25, 263–280, https://doi.org/10.1175/2009WAF2222267.1, 2010. a
Skok, G.: Analysis of Fraction Skill Score properties for a displaced rainy grid point in a rectangular domain, Atmos. Res., 169, 556–565, https://doi.org/10.1016/j.atmosres.2015.04.012, 2016. a
Skok, G.: A New Spatial Distance Metric for Verification of Precipitation, Appl. Sci.-Basel, 12, 4048, https://doi.org/10.3390/app12084048, 2022. a, b, c, d
Skok, G.: Precipitation attribution distance, Atmos. Res., 295, https://doi.org/10.1016/j.atmosres.2023.106998, 2023. a, b
Skok, G.: Snapshot of the Smoothing on Sphere package, Zenodo [code], https://doi.org/10.5281/zenodo.15100264, 2025. a
Skok, G. and Hladnik, V.: Verification of Gridded Wind Forecasts in Complex Alpine Terrain: A New Wind Verification Methodology Based on the Neighborhood Approach, Mon. Weather Rev., 146, 63–75, https://doi.org/10.1175/MWR-D-16-0471.1, 2018. a
Skok, G. and Lledó, L.: Spatial verification of global precipitation forecasts, Q. J. Roy. Meteor. Soc., https://doi.org/10.1002/qj.5006, 2025. a, b, c, d
Skok, G. and Roberts, N.: Analysis of Fractions Skill Score properties for random precipitation fields and ECMWF forecasts, Q. J. Roy. Meteor. Soc., 142, 2599–2610, https://doi.org/10.1002/qj.2849, 2016. a
Skok, G. and Roberts, N.: Estimating the displacement in precipitation forecasts using the Fractions Skill Score, Q. J. Roy. Meteor. Soc., 144, 414–425, https://doi.org/10.1002/qj.3212, 2018. a
Smith, S. W.: The Scientist and Engineer's Guide to Digital Signal Processing, California Technical Publishing, ISBN 0-9660176-7-6, 1999. a
Wernli, H., Paulat, M., Hagen, M., and Frei, C.: SAL – A novel quality measure for the verification of quantitative precipitation forecasts, Mon. Weather Rev., 136, https://doi.org/10.1175/2008MWR2415.1, 2008. a
Wernli, H., Hofmann, C., and Zimmer, M.: Spatial forecast verification methods intercomparison project: Application of the SAL technique, Weather Forecast., 24, https://doi.org/10.1175/2009WAF2222271.1, 2009. a
Weyn, J. A., Durran, D. R., and Caruana, R.: Improving Data-Driven Global Weather Prediction Using Deep Convolutional Neural Networks on a Cubed Sphere, J. Adv. Model. Earth. Sy., 12, https://doi.org/10.1029/2020MS002109, 2020. a
Wilks, D. S.: Statistical Methods in the Atmospheric Sciences, Elsevier, ISBN 9780128158234, https://doi.org/10.1016/C2017-0-03921-6, 2019. a
Woodhams, B. J., Birch, C. E., Marsham, J. H., Bain, C. L., Roberts, N. M., and Boyd, D. F. A.: What Is the Added Value of a Convection-Permitting Model for Forecasting Extreme Rainfall over Tropical East Africa?, Mon. Weather Rev., 146, 2757–2780, https://doi.org/10.1175/MWR-D-17-0396.1, 2018. a, b
Zacharov, P. and Rezacova, D.: Using the fractions skill score to assess the relationship between an ensemble QPF spread and skill, Atmos. Res., 94, 684–693, https://doi.org/10.1016/j.atmosres.2009.03.004, 2009. a
Short summary
Forecast verification is essential for improving weather prediction models but faces challenges with traditionally used metrics. New spatial verification metrics like the Fraction Skill Score (FSS) perform better but are difficult to use in a global domain due to large computational cost. We introduce two new global smoothing methodologies that can be used with smoothing-based metrics in a global domain. We demonstrate their effectiveness with an analysis of global precipitation forecasts.
Forecast verification is essential for improving weather prediction models but faces challenges...
