GMD Geoscientific Model Development GMD Geosci. Model Dev. 1991-9603 Copernicus Publications Göttingen, Germany 10.5194/gmd-7-3135-2014 MeteoIO 2.4.2: a preprocessing library for meteorological data Bavay M. https://orcid.org/0000-0002-5039-1578 1 Egger T. 2 WSL Institute for Snow and Avalanche Research SLF, Flüelastrasse 11, 7260 Davos Dorf, Switzerland Egger Consulting, Postgasse 2, 1010 Vienna, Austria 19 12 2014 7 6 3135 3151 Copyright: © 2014 M. Bavay 2014 This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this licence, visit https://creativecommons.org/licenses/by/3.0/ This article is available from https://gmd.copernicus.org/articles/7/3135/2014/gmd-7-3135-2014.html The full text article is available as a PDF file from https://gmd.copernicus.org/articles/7/3135/2014/gmd-7-3135-2014.pdf

Using numerical models which require large meteorological data sets is sometimes difficult and problems can often be traced back to the Input/Output functionality. Complex models are usually developed by the environmental sciences community with a focus on the core modelling issues. As a consequence, the I/O routines that are costly to properly implement are often error-prone, lacking flexibility and robustness. With the increasing use of such models in operational applications, this situation ceases to be simply uncomfortable and becomes a major issue. <br><br> The MeteoIO library has been designed for the specific needs of numerical models that require meteorological data. The whole task of data preprocessing has been delegated to this library, namely retrieving, filtering and resampling the data if necessary as well as providing spatial interpolations and parameterizations. The focus has been to design an Application Programming Interface (API) that (i) provides a uniform interface to meteorological data in the models, (ii) hides the complexity of the processing taking place, and (iii) guarantees a robust behaviour in the case of format errors, erroneous or missing data. Moreover, in an operational context, this error handling should avoid unnecessary interruptions in the simulation process. <br><br> A strong emphasis has been put on simplicity and modularity in order to make it extremely easy to support new data formats or protocols and to allow contributors with diverse backgrounds to participate. This library is also regularly evaluated for computing performance and further optimized where necessary. Finally, it is released under an Open Source license and is available at <a href="http://models.slf.ch/p/meteoio"target="_blank">http://models.slf.ch/p/meteoio</a>. <br><br> This paper gives an overview of the MeteoIO library from the point of view of conceptual design, architecture, features and computational performance. A scientific evaluation of the produced results is not given here since the scientific algorithms that are used have already been published elsewhere.

 References Ballou, D. P. and Pazer, H. L.: Modeling data and process quality in multi-input, multi-output information systems, Manage. Sci., 31, 150–162, 1985. Barrenetxea, G., Ingelrest, F., Schaefer, G., Vetterli, M., Couach, O., and Parlange, M.: Sensorscope: Out-of-the-box environmental monitoring, In Information Processing in Sensor Networks, 2008, IPSN'08, International Conference, 332–343, IEEE, 2008. Beck, K. and Andres, C.: Extreme Programming Explained: Embrace Change, Addison-Wesley Professional, 2nd Edn., 2004. Brutsaert, W.: On a derivable formula for long-wave radiation from clear skies, Water Resour. Res., 11, 742–744, 1975. Butterworth, S.: On the theory of filters amplifiers, Experimental Wireless & the Wireless Engineer, 7, 536–541, 1930. Clark, G. and Allen, C.: The estimation of atmospheric radiation for clear and cloudy skies, in: Proc. 2nd National Passive Solar Conference (AS/ISES), 675–678, 1978. Corripio, J. G.: Vectorial algebra algorithms for calculating terrain parameters from DEMs and solar radiation modelling in mountainous terrain, Int. J. Geogr. Inf. Sci., 17, 1–23, 2003. Consortium for Small-scale Modeling: available at: <a href="http://www.cosmo-model.org/">http://www.cosmo-model.org/</a> (last access: 30 May 2014), 2013. Crawford, T. M. and Duchon, C. E.: An improved parameterization for estimating effective atmospheric emissivity for use in calculating daytime downwelling longwave radiation, J. Appl. Meteorol., 38, 474–480, 1999. Cressie, N.: Statistics for spatial data, Terra Nova, 4, 613–617, 1992. Daqing, Y., Esko, E., Asko, T., Ari, A., Barry, G., Thilo, G., Boris, S., Henning, M., and Janja, M.: Wind-induced precipitation undercatch of the Hellmann gauges, Nord. Hydrol., 30, 57–80, 1999. Dilley, A. and O'brien, D.: Estimating downward clear sky long-wave irradiance at the surface from screen temperature and precipitable water, Q. J. Roy. Meteor. Soc., 124, 1391–1401, 1998. Duff, I. S., Erisman, A. M., and Reid, J. K.: Direct Methods for Sparse Matrices, Clarendon Press, Oxford, 1986. Dunn, M. and Hickey, R.: The effect of slope algorithms on slope estimates within a GIS, Cartography, 27, 9–15, 1998. Durand, Y., Brun, E., Merindol, L., Guyomarc'h, G., Lesaffre, B., Martin, E.: A meteorological estimation of relevant parameters for snow models, Ann. Glaciol., 18, 65–71, 1993. Endrizzi, S., Gruber, S., Dall'Amico, M., and Rigon, R.: GEOtop 2.0: simulating the combined energy and water balance at and below the land surface accounting for soil freezing, snow cover and terrain effects, Geosci. Model Dev., 7, 2831–2857, <a href="http://dx.doi.org/10.5194/gmd-7-2831-2014">https://doi.org/10.5194/gmd-7-2831-2014</a>, 2014. Fleming, M. D. and Hoffer, R. M.: Machine processing of landsat MSS data and DMA topographic data for forest cover type mapping, Tech. Rep. LARS Technical Report 062879, Laboratory for Applications of Remote Sensing, Purdue University, 1979. Fliegel, H. F. and van Flandern, T. C.: Letters to the editor: a machine algorithm for processing calendar dates, Commun. ACM, 11, 657, <a href="http://dx.doi.org/10.1145/364096.364097">https://doi.org/10.1145/364096.364097</a>, 1968. Førland, E. J., Allerup, P., Dahlstrøm, B., Elomaa, E., Jonsson, T., Madsen, H., Per, J., Rissanen, P., Vedin, H., and Vejen, F.: Manual for Operational Correction of Nordic Precipitation Data, DNMI-Reports 24/96 KLIMA, 1996. gin: Common Information Platform for Natural Hazards, available at: <a href="http://www.gin-info.admin.ch/">http://www.gin-info.admin.ch/</a> (last access: 20 October 2014), 2014. Goodison, B., Louie, P., and Yang, D.: The WMO solid precipitation measurement intercomparison, World Meteorological Organization-Publications-WMO TD, 65–70, 1997. Goovaerts, P.: Geostatistics for Natural Resources Evaluation, Oxford University Press, 1997. Goring, D. G. and Nikora, V. I.: Despiking acoustic Doppler velocimeter data, J. Hydraul. Eng., 128, 117–126, 2002. Hager, J. W., Behensky, J. F., and Drew, B. W.: The Universal Grids: Universal Transverse Mercator (UTM) and Universal Polar Stereographic (UPS), Edition 1, Tech. rep., Defense Mapping Agency, 1989. Hamon, W. R.: Computing actual precipitation, in: Distribution of precipitation in mountaineous areas, Geilo symposium 1, 159–174, World Meteorological Organization, 1972. Hastings, C., Hayward, J. T., and Wong, J. P.: Approximations for digital computers, Vol. 170, Princeton University Press, Princeton, NJ, 1955. Hodgson, M. E.: Comparison of angles from surface slope/aspect algorithms, Cartogr. Geogr. Inform., 25, 173–185, 1998. Horn, B. K.: Hill shading and the reflectance map, Proc. IEEE, 69, 14–47, 1981. Huwald, H., Higgins, C. W., Boldi, M.-O., Bou-Zeid, E., Lehning, M., and Parlange, M. B.: Albedo effect on radiative errors in air temperature measurements, Water Resour. Res., 45, W08431, <a href="http://dx.doi.org/10.1029/2008WR007600">https://doi.org/10.1029/2008WR007600</a>, 2009. Idso, S.: A set of equations for full spectrum and 8-to 14-μm and 10.5-to 12.5-μm thermal radiation from cloudless skies, Water resources research, 17, 295–304, 1981. Inishell: A flexible configuration interface for numerical models: available at: <a href="https://models.slf.ch/p/inishell/">https://models.slf.ch/p/inishell/</a> (last access: 30 October 2014), 2014. Iqbal, M.: An Introduction to Solar Radiation, Academic Press, 1983. Kasten, F. and Czeplak, G.: Solar and terrestrial radiation dependent on the amount and type of cloud, Sol. Energy, 24, 177–189, 1980. Konzelmann, T., van de Wal, R. S., Greuell, W., Bintanja, R., Henneken, E. A., and Abe-Ouchi, A.: Parameterization of global and longwave incoming radiation for the Greenland Ice Sheet, Global Planet. Change, 9, 143–164, 1994. Kotsiantis, S., Kanellopoulos, D., and Pintelas, P.: Data preprocessing for supervised leaning, Int. J. Comput. Sci., 1, 2006. Kruijff, E., Mendez, E., Veas, E., Grünewald, T., Simoni, S., Luyet, V., Salminen, O., Nurminen, A., Lehtinen, V.: HYDROSYS: on-site monitoring and management of environmental processes using handheld devices, in: GeoHydroinformatics: Integrating GIS and Water Engineering, 2010. Lancaster, O. E.: Machine method for the extraction of cube root, J. Am. Stat. Assoc., 37, 112–115, 1942. Liston, G. E. and Elder, K.: A meteorological distribution system for high-resolution terrestrial modeling (MicroMet), J. Hydrometeorol., 7, 217–234, 2006. Lomont, C.: Fast inverse square root, Tech. rep., Purdue University, 2003. Marjamäki, D.: Cppcheck: a tool for static C/C++ code analysis, available at: <a href="http://cppcheck.sourceforge.net/">http://cppcheck.sourceforge.net/</a> (last access: 30 May 2014), 2013. Martin, K. and Hoffman, B.: An open source approach to developing software in a small organization, software, IEEE, 24, 46–53, 2007. Meeus, J. H.: Astronomical Algorithms, 2nd Edn., Willmann-Bell, Incorporated, 1998. Meyers, S.: Effective C++: 50 Specific Ways to Improve Your Programs and Designs, 2nd Edn., Addison-Wesley Publishing Company, 1992. Michel, S., Salehi, A., Luo, L., Dawes, N., Aberer, K., Barrenetxea, G., Bavay, M., Kansal, A., Kumar, K. A., Nath, S., Parlange, M., Tansley, S., van Ingen, C., Zhao, F., and Zhou, Y. L.: Environmental Monitoring 2.0, in: Data Engineering, 2009, ICDE'09. IEEE 25th International Conference on, 1507–1510, IEEE, 2009. Montgomery, P. L.: Speeding the Pollard and elliptic curve methods of factorization, Math. Comput., 48, 243–264, 1987. Musser, D. R., Derge, G. J., and Saini, A.: STL Tutorial and Reference Guide: C++ Programming with the Standard Template Library, Addison-Wesley, 2nd Edn., 2001. Nakamura, R. and Mahrt, L.: Air temperature measurement errors in naturally ventilated radiation shields, J. Atmos. Ocean. Tech., 22, 1046–1058, 2005. Nethercote, N. and Seward, J.: How to Shadow Every Byte of Memory Used by a Program, in: Proceedings of the Third International ACM SIGPLAN/SIGOPS Conference on Virtual Execution Environments (VEE 2007), 2007. Omstedt, A.: A coupled one-dimensional sea ice–ocean model applied to a semi-enclosed basin, Tellus A, 42, 568–582, 1990. Prata, A.: A new long-wave formula for estimating downward clear-sky radiation at the surface, Q. J. Roy. Meteor. Soc., 122, 1127–1151, 1996. Press, W. H., Flannery, B. P., Teukolsky, S. A., and Vetterling, W. T.: LU decomposition and its applications, in: Numerical Recipes in FORTRAN: The Art of Scientific Computing, Cambridge University Press, 2nd Edn., 34–42, 1992. PROJ.4: Cartographic Projections Library, available at: <a href="https://trac.osgeo.org/proj/">https://trac.osgeo.org/proj/</a> (last access: 30 May 2014), 2013. Rouson, D., Xia, J., and Xu, X.: Scientific software design: the object-oriented way, Cambridge University Press, 2011. Ryan, B.: A mathematical model for diagnosis and prediction of surface winds in mountainous terrain, J. Appl. Meteorol., 16, 571–584, 1977. Scapin, D. L. and Bastien, J. C.: Ergonomic criteria for evaluating the ergonomic quality of interactive systems, Behav. Inform. Technol., 16, 220–231, 1997. Seward, J., Nethercote, N., and Hughes, T.: valgrind, available at: <a href="http://valgrind.org/">http://valgrind.org/</a> (last access: 30 May 2014), 2013. Spence, R. and Bavay, M.: Precipitation Redistribution for Snow Models, in: 11th Swiss Geoscience Meeting, Lausanne, 2013. Tang, R., Etzion, Y., and Meir, I.: Estimates of clear night sky emissivity in the Negev Highlands, Israel, Energ. Convers. Manage., 45, 1831–1843, 2004. Thomas, L. H.: Elliptic Problems in Linear Differential Equations over a Network, Tech. rep., Watson Sci. Comput. Lab, Columbia University, 1949. Unsworth, M. H. and Monteith, J.: Long-wave radiation at the ground I. Angular distribution of incoming radiation, Q. J. Roy. Meteor. Soc., 101, 13–24, 1975. van Heesch, D.: Doxygen: Source code documentation generator tool, available at: <a href="http://www.doxygen.org/">http://www.doxygen.org/</a> (last access: 30 May 2014), 2008. Vincenty, T.: Closed formulas for the direct and reverse geodetic problems, J. Geodesy., 51, 241–242, 1977. Viviroli, D., Zappa, M., Gurtz, J., Weingartner, R.: An introduction to the hydrological modelling system PREVAH and its pre-and post-processing-tools, Environ. Model. Softw., 24, 10, 1209–1222, 2009. Wagner, A.: Literature study on the correction of precipitation measurements, Bavarian State Institute of Forestry, FutMin C1-Met-29 (BY), 2009. Wand, Y. and Wang, R. Y.: Anchoring data quality dimensions in ontological foundations, Commun. ACM, 39, 86–95, 1996. Wheeler, D. A.: Flawfinder, available at: <a href="http://www.dwheeler.com/flawfinder/">http://www.dwheeler.com/flawfinder/</a> (last access: 30 May 2014), 2013. Winstral, A., Elder, K., Davis, R.: Spatial snow modeling of wind-redistributed snow using terrain-based parameters, J. Hydrometeorol., 3, 524–538, 2002. Yokoyama, K., Ohno, H., Kominami, Y., Inoue, S., and Kawakata, T.: Performance of Japanese precipitation gauges in winter, Seppyo, 65, 303–316, 2003. Yu, E. S. and Mylopoulos, J.: Understanding "why" in software process modelling, analysis, and design, in: Proceedings of the 16th international conference on Software engineering, 159–168, IEEE Computer Society Press, 1994.