References

GMD

Geoscientific Model Development

GMD

Geosci. Model Dev.

1991-9603

Copernicus Publications

Göttingen, Germany

10.5194/gmd-5-869-2012

Land surface Verification Toolkit (LVT) – a generalized framework for land surface model evaluation

Kumar

S. V.

¹ ² Peters-Lidard

C. D.

² Santanello

² Harrison

² ³ Liu

² ³ Shaw

¹ ² ⁴

Science Applications International Corporation, Beltsville, MD, USA

Hydrological Sciences Laboratory, NASA Goddard Space Flight Center, Greenbelt, MD, USA

Earth System Science Interdisciplinary Center, College Park, MD, USA

Air Force Weather Agency, Offutt, NE, USA

26 06 2012

5 3 869 886

2012

This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this licence, visit https://creativecommons.org/licenses/by/3.0/

This article is available from https://gmd.copernicus.org/articles/5/869/2012/gmd-5-869-2012.html

The full text article is available as a PDF file from https://gmd.copernicus.org/articles/5/869/2012/gmd-5-869-2012.pdf

Model evaluation and verification are key in improving the usage and applicability of simulation models for real-world applications. In this article, the development and capabilities of a formal system for land surface model evaluation called the Land surface Verification Toolkit (LVT) is described. LVT is designed to provide an integrated environment for systematic land model evaluation and facilitates a range of verification approaches and analysis capabilities. LVT operates across multiple temporal and spatial scales and employs a large suite of in-situ, remotely sensed and other model and reanalysis datasets in their native formats. In addition to the traditional accuracy-based measures, LVT also includes uncertainty and ensemble diagnostics, information theory measures, spatial similarity metrics and scale decomposition techniques that provide novel ways for performing diagnostic model evaluations. Though LVT was originally designed to support the land surface modeling and data assimilation framework known as the Land Information System (LIS), it supports hydrological data products from non-LIS environments as well. In addition, the analysis of diagnostics from various computational subsystems of LIS including data assimilation, optimization and uncertainty estimation are supported within LVT. Together, LIS and LVT provide a robust end-to-end environment for enabling the concepts of model data fusion for hydrological applications. The evolving capabilities of LVT framework are expected to facilitate rapid model evaluation efforts and aid the definition and refinement of formal evaluation procedures for the land surface modeling community.

References 1

Abramowitz, G., Leuning, R., Clark, M., and Pitman, A.: Evaluating the performance of land surface models, J. Climate, 21, 5468–5481, 2008.

Barlage, M., Chen, F., Tewari, M., Ikeda, K., Gochis, D., Dudhia, J., Rasmussen, R., Livneh, B., Ek, M., and Mitchell, M.: Noah Land Surface Model modifications to improve snowpack prediction in the Colorado Rocky Mountains, J. Geophys. Res., 115, D22101, <a href="http://dx.doi.org/10.1029/2009JD013470">https://doi.org/10.1029/2009JD013470</a>, 2010.

Barrett, A.: National Operational Hydrologic Remote Sensing Center Snow Data Assimilation System (\uppercase{SNODAS}) products at \uppercase{NSIDC}, Tech. rep., National Snow and Ice Data Center, Boulder, CO, digital Media, 2003.

Bloschl, G.: Scaling issues in snow hydrology, Hydrol. Process., 13, 2149–2175, 1999.

Blöschl, G. and Sivapalan, M.: Scale issues in hydrological modeling, Hydrol. Process., 9, 251–290, 1995.

Blyth, E., Gash, J., Lloyd, A., Pryor, M., Weedon, G., and Shuttleworth, J.: Evaluating the \uppercase{JULES} model energy fluxes using the \uppercase{FLUXNET} data, J. Hydrometeor., 11, 509–519, 2010.

Blyth, E., Clark, D. B., Ellis, R., Huntingford, C., Los, S., Pryor, M., Best, M., and Sitch, S.: A comprehensive set of benchmark tests for a land surface model of simultaneous fluxes of water and carbon at both the global and seasonal scale, Geosci. Model Dev., 4, 255–269, <a href="http://dx.doi.org/10.5194/gmd-4-255-2011">https://doi.org/10.5194/gmd-4-255-2011</a>, 2011.

Brown, R. and Brasnett, B.: Canadian Meteorological Center (\uppercase{CMC}) daily snow analysis data, Tech. rep., National Snow and Ice Data Center, Boulder, CO, digital Media, 2010.

Brown, B., Gotway, J., Bullock, R., Gilleland, E., Fowler, T., Ahijevych, D., and Jensen, T.: The Model Evaluation Tools (\uppercase{MET}): Community tools for forecast evaluation, in: 25th Conf. on International Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, Amer. Metero. Soc., Phoenix, AZ, 2009.

Brown, J., Demargne, J., Seo, D.-J., and Liu, Y.: The \uppercase{E}nsemble \uppercase{V}erification \uppercase{S}ystem (\uppercase{EVS}): a software tool for verifying ensemble foreasts of hydrometeorological and hydrologic variables at discrete locations, Environ. Model. Software, 25, 854–872, 2010.

Casati, B., Ross, G., and Stephenson, D. B.: A new intensity- scale approach for the verification of spatial precipitation forecasts, Meteor. Appl., 11, 141–154, 2004.

Chen, F., Mitchell, K., Schaake, J., Xue, Y., Pan, H., Koren, V., Duan, Y., Ek, M., and Betts, A.: Modeling of land-surface evaporation by four schemes and comparison with \uppercase{FIFE} observations, J. Geophys. Res., 101, 7251–7268, 1996.

Dai, Y., Zeng, X., Dickinson, R., Baker, I., Bonan, G., Bosilovich, M., Denning, S., Dirmeyer, P., Houser, P., Niu, G., Oleson, K., Schlosser, A., and Yang, Z.-L.: The common land model (\textsc{CLM}), B. Am. Meteorol. Soc., 84, 1013–1023, <a href="http://dx.doi.org/10.1175/BAMS-84-8-1013">https://doi.org/10.1175/BAMS-84-8-1013</a>, 2003.

Decyk, V. K., Norton, C. D., and Szymanski, B. K.: How to express \uppercase{C++} concepts in \uppercase{F}ortran 90, Scientific Programming, 6, 363–390, 1997.

Derber, J., Parrish, D., and Lord, S.: The new global operational analysis system at the \textsc{N}ational \textsc{M}eteorological \textsc{C}enter, Weather Forecast., 6, 538–547, 1991.

de Rosnay, P., Boone, A., Beljaars, A., and Polcher, J.: \uppercase{AMMA} Land surface intercomparison projects, \uppercase{GEWEX N}ews, 16, 10–11, 2006.

Dirmeyer, P., Gao, X., Zhao, M., Guo, Z., Oki, T., and Hanasaki, N.: \uppercase{GSWP-2}: Multimodel analysis and implications for our perception of the land surface, B. Am. Meteorol. Soc., 87, 1381–1397, <a href="http://dx.doi.org/10.1175/BAMS-87-10-1381">https://doi.org/10.1175/BAMS-87-10-1381</a>, 2006.

Ek, M. B., Mitchell, K. E., Lin, Y., Rogers, E., Grunmann, P., Koren, V., Gayno, G., and Tarpley, J. D.: Implementation of Noah land surface model advances in the National Centers for Environmental Prediction operational mesoscale Eta model, J. Geophys. Res., 108, 8851, <a href="http://dx.doi.org/10.1029/2002JD003296">https://doi.org/10.1029/2002JD003296</a>, 2003.

Entekhabi, D., Asrar, G., Betts, A., Beven, K., Bras, R., Duffy, C., Dunne, T., Koster, R., Lettenmaier, D., McLaughlin, D., Shuttleworth, W., van Genuchten, M., Wei, M.-Y., and Wood, E.: An agenda for land surface hydrology research and a call for the second international hydrological decade, B. Am. Meteorol. Soc., 80, 2043–2058, 1999.

Entekhabi, D., Reichle, R., Koster, R., and Crow, W.: Performance Metrics for Soil Moisture Retrievals and Application Requirements, J. Hydrometeor., 11, 832–840, <a href="http://dx.doi.org/10.1175/2010JHM1223.1">https://doi.org/10.1175/2010JHM1223.1</a>, 2010.

Erickson, T. A., Williams, M. W., and Winstral, A.: Persistence of topographic controls on the spatial distribution of snow in rugged mountain terrain, Colorado, United States, Water Resour. Res., 41, W04014, <a href="http://dx.doi.org/10.1029/2003WR002973">https://doi.org/10.1029/2003WR002973</a>, 2005.

Fennessey, M. and Shukla, J.: Impact of initial soil wetness on seasonal atmospheric prediction, J. Climate, 12, 3167–3180, 1999.

Foster, J., Hall, D., Eylander, J., Riggs, G., Nghiem, S., Tedesco, M., Kim, E., Montesano, P., Kelly, R., Casey, K. A., and Choudhury, B.: A blended global snow product using visible, passive microwave and scatterometer satellite data, Int. J. Remote Sens., 32, 5, <a href="http://dx.doi.org/10.1080/01431160903548013">https://doi.org/10.1080/01431160903548013</a>, 2011.

Friend, A. and Kiang, N.: Land surface model development for the \uppercase{GISS GCM}: Effects of improved canopy physiology on simulated climate, J. Climate, 18, 2883–2902, 2005.

Gelb, A.: Applied Optimal Estimation, MIT Press, Cambridge, MA, 1974.

Grayson, R. and Bloschl, G.: Spatial patterns in catchment hydrology: Observations and modeling, Cambridge University Press, Cambridge, 2000.

Gulden, L., Rosero, E., Yang, Z.-L., Wagener, T., and Niu, G.-Y.: Model performance, model robustness, and model fitness scores: A new method for identifying good land surface models, Geophys. Res. Lett., 35, L11404, <a href="http://dx.doi.org/10.1029/2008GL033721">https://doi.org/10.1029/2008GL033721</a>, 2008.

Gupta, V., Rodriguez-Iturbe, I., and Wood, E.: Scale problems in Hydrology, Reidel, Dordrecht, 1986.

Gupta, H., Wagener, T., and Liu, Y.: Reconciling theory with observations: elements of a diagnostic approach to model evaluation, Hydrol. Process., 22, 3802–3813, <a href="http://dx.doi.org/10.1002/hyp.6989">https://doi.org/10.1002/hyp.6989</a>, 2008.

Gupta, H., Kling, H., Yilmaz, K., and Martinez, G.: Decomposition of the mean squared error and \uppercase{NSE} performance criteria: Implications for improving hydrological modeling, J. Hydrol., 377, 80–91, <a href="http://dx.doi.org/10.1016/j.jhydrol.2009.08.003">https://doi.org/10.1016/j.jhydrol.2009.08.003</a>, 2009.

Hall, D., Riggs, G., and Salomonson, V.: \textsc{MODIS/T}erra snow cover daily L3 Global 500 m Grid V005, Tech. rep., National Snow and Ice Data Center, Colorado, USA, digital media, 2006.

Harrison, K., Kumar, S., Peters-Lidard, C., and Santanello, J.: Quantifying soil moisture modeling uncertainty with remote sensing observations using Bayesian inference techniques, Water Resour. Res., under review, 2012.

Heim, R. J.: A review of twentieth century drought indices used in the United States, B. Am. Meteorol. Soc., 83, 1149–1165, 2002.

Henderson-Sellers, A., Pitman, A., Love, P., Irannejad, P., and Chen, T.: The project for Intercomparison of land surface parameterization schemes (\uppercase{PILPS}): Phases 2 and 3, B. Am. Meteorol. Soc., 76, 489–503, 1995.

Higgins, R., Janowiak, J., and Yao, Y.-P.: A gridded hourly precipitation database for the United States (1963–1993)., Tech. rep., NCEP Climate Prediction Center Atlas 1, 46 pp., 1996.

Hill, C., DeLuca, C., Balaji, V., Suarez, M., and da Silva, A.: The Architecture of the Earth System Modeling Framework, Comput. Sci. Eng., 6, 18–28, 2004.

Jiminez, C., Prigent, C., Mueller, B., Seneviratne, S., McCabe, M., Wood, E., Rossow, W., Balsamo, G., Betts, A., Dirmeyer, P., Fisher, J., Jung, M., Kanamitsu, M., Reichle, R., Reichstein, M., Rodell, M., Sheffield, J., Tu, K., and Wang, K.: Global intercomparison of 12 land surface heat flux estimates, J. Geophys. Res., 116, D02102, <a href="http://dx.doi.org/10.1029/2010JD014545">https://doi.org/10.1029/2010JD014545</a>, 2011.

Jung, M., Reichstein, M., and Bondeau, A.: Towards global empirical upscaling of FLUXNET eddy covariance observations: validation of a model tree ensemble approach using a biosphere model, Biogeosciences, 6, 2001–2013, <a href="http://dx.doi.org/10.5194/bg-6-2001-2009">https://doi.org/10.5194/bg-6-2001-2009</a>, 2009.

Kato, H., Rodell, M., Beyrich, F., Cleugh, H., van Gorsel, E., Liu, H., and Meyers, T.: Sensitivity of land surface simulations to model physics, land characteristics, and forcings at four \uppercase{CEOP} sites, J. Meteor. Soc. Japan, 85A, 187–204, 2007.

Koren, V., Schaake, J., Mitchell, K., Duan, Q.-Y., and Chen, F.: A parameterization of snowpack and frozen ground intended for NCEP weather and climate models, J. Geophys. Res., 104, 19569–19585, 1999.

Koster, R., Suarez, M., Liu, P., Jambor, U., Berg, A., Kistler, M., Reichle, R., Rodell, M., and Famiglietti, J.: Realistic initialization of land surface states: Impacts on subseasonal forecast skill, J. Hydrometeor., 5, 1049–1063, 2004.

Koster, R., Guo, Z., Dirmeyer, P., Yang, R., Mitchell, K., and Puma, M.: On the nature of soil moisture in land surface models, J. Climate, 22, 4322–4335, <a href="http://dx.doi.org/10.1175/2009JCLI2832.1">https://doi.org/10.1175/2009JCLI2832.1</a>, 2009.

Kumar, S., Peters-Lidard, C., Tian, T., Houser, P., Geiger, J., Olden, S., Lighty, L., Eastman, J., Doty, B., Dirmeyer, P., Adams, J., Mitchell, K., Wood, E., and Sheffield, J.: Land information system: An interoperable framework for high resolution land surface modeling, Environ. Model. Software, 21, 1402–1415, 2006.

Kumar, S., Peters-Lidard, C., Eastman, J. L., and Tao, W.-K.: An integrated high resolution hydrometeorological modeling testbed using LIS and WRF, Environ. Model. Software, 23, 169–181, 2007.

Kumar, S., Peters-Lidard, C., Tian, Y., Reichle, R. H., Alonge, C., Geiger, J., Eylander, J., and Houser, P.: An integrated hydrologic modeling and data assimilation framework enabled by the Land Information System (LIS), IEEE Computer, 41, 52–59, <a href="http://dx.doi.org/10.1109/MC.2008.511">https://doi.org/10.1109/MC.2008.511</a>, 2008{a}.

Kumar, S., Reichle, R., Peters-Lidard, C., Koster, R., Zhan, X., Crow, W., Eylander, J., and Houser, P.: A land surface data assimilation framework using the Land Information System: Description and Applications, Adv. Water Resour., 31, 1419–1432, <a href="http://dx.doi.org/10.1016/j.advwatres.2008.01.013">https://doi.org/10.1016/j.advwatres.2008.01.013</a>, 2008{b}.

Kumar, S. V., Reichle, R. H., Harrison, K. W., Peters-Lidard, C. D., Yatheendradas, S., and Santanello, J. A.: A comparison of methods for a priori bias correction in soil moisture data assimilation, Water Resour. Res., 48, W03515, <a href="http://dx.doi.org/10.1029/2010WR010261">https://doi.org/10.1029/2010WR010261</a>, 2012.

Lange, H.: Are ecosystems dynamical systems?, International Journal of computing anticipatory systems, 3, 169–186, 1999.

Lawrence, D., Oleson, K., Flanner, M., Thornton, P., Swenson, S., Lawrence, P., Zeng, X., Yang, Z.-L., Levis, S., Sakaguchi, K., Bonan, G., and Slater, A.: Parameterization improvements and functional and structural advances in version 4 of the Community Land Model, J. Adv. Model. Earth Sys., 3, M03001, <a href="http://dx.doi.org/10.1029/JAMES.2011.3.1">https://doi.org/10.1029/JAMES.2011.3.1</a>, 2011.

Liu, Q., Reichle, R., Bindlish, R., Cosh, M., Crow, W., de Jeu, R., De Lannoy, G., Huffman, G., and Jackson, T.: The contributions of precipitation and soil moisture observations to the skill of soil moisture estimates in land data assimilation system, J. Hydrometeor., 12, 5, <a href="http://dx.doi.org/10.1175/JHM-D-10.05000.1">https://doi.org/10.1175/JHM-D-10.05000.1</a>, 2011{a}.

Liu, Y., Brown, J., Demargne, J., and Seo, D.-J.: A wavelet-based approach to assessing timing errors in hydrologic predictions, J. Hydrol., 397, 210–224, 2011{b}.

Lohmann, D., Mitchell, K., Houser, P., Wood, E., Schaake, J., Robock, A., Cosgrove, B., Sheffield, J., Duan, Q., Luo, L., Higgins, W., Pinker, R., and Tarpley, J.: Streamflow and water balance intercomparisons of four land surface models in the North American Land Data Assimilation System project, J. Geophys. Res., 109, D07S91, <a href="http://dx.doi.org/10.1029/2003JD003517">https://doi.org/10.1029/2003JD003517</a>, 2004.

Mitchell, K., Lohmann, D., Houser, P., Wood, E., Schaake, J., Robock, A., Cosgrove, B., Sheffield, J., Duan, Q., Luo, L., Higgins, R., Pinker, R., Tarpley, J., Lettenmaier, D., Marshall, C., Entin, J., Pan, M., Shi, W., Koren, V., Meng, J., Ramsay, B., and Bailey, A.: The multi-institution North American Land Data Assimilation System (NLDAS): Utilizing multiple GCIP products and partners in a continental distributed hydrological modeling system, J. Geophys. Res., 109, D07S90, <a href="http://dx.doi.org/10.1029/2003JD003823">https://doi.org/10.1029/2003JD003823</a>, 2004.

Mo, K., Long, L., Xia, Y., Yang, S., Schemm, J., and Ek, M.: Drought indices based on the climate forecast system reanalysis and ensemble \uppercase{NLDAS}, J. Hydrometeor., 12, 181–205, 2011.

Moore, B., Bertone, S., Mitchell, K., Rice, P., and Neill, R.: A worldwide near-real time diagnostic agrometeorological model, in: 20th AMS Conf. Ag and Forest Meteorology, 7–11, 1990.

Mu, Q., Heinsch, F., Zhao, M., and Running, S.: Development of a Global evapotranspiration algorithm based on \uppercase{MODIS} and global meteorology data, Remote Sens. Environ., 111, 519–536, 2007.

Murphy, A. and Winkler, R.: Diagnostic verification of probability forecasts, Int. J. Forecast., 7, 435–455, 1992.

Niu, G.-Y., Yang, Z.-L., Mitchell, K., Chen, F., Ek, M., Barlage, M., Kumar, A., Manning, K., Niyogi, D., Rosero, E., Tewari, M., and Xia, Y.-L.: The community Noah land surface model with multiparameterization options (Noah-\uppercase{MP}): 1. Model description and evaluation with local-scale measurements, J. Geophys. Res., 116, D12109, <a href="http://dx.doi.org/10.1029/2010JD015139">https://doi.org/10.1029/2010JD015139</a>, 2011.

Njoku, E., Jackson, T., Lakshmi, V., Chan, T., and Nghiem, S.: Soil moisture retrieval from \uppercase{AMSR-E}, IEEE Trans. Geosci. Remote Sens., 41, 215–229, 2003.

NRC: Assessment of Hydrologic and Hydrometeorological Operations and Services, Tech. rep., National Academy Press, 1996.

Owe, M., de Jeu, R., and Holmes, T.: Multi-sensor historical climatology of satellite-derived global land surface moisture, J. Geophys. Res., 13, F01002, <a href="http://dx.doi.org/10.1029/2007JF000769">https://doi.org/10.1029/2007JF000769</a>, 2008.

Pachepsky, Y., Guber, A., Jacques, D., Simunek, J., van Genuchten, M., Nicholson, T., and Cady, R.: Information content and complexity of simulated soil water fluxes, Geoderma, 253–266, 2006.

Pan, F., Pachepsky, Y., Andrey, G., and Hill, R.: Information and Complexity Measures Applied to Observed and Simulated Soil Moisture Time Series, Hydrol. Sci. J., 56, 1027–1039, 2011.

Pan, M., Ming, J., Sheffield, J., Wood, E., Mitchell, K., Houser, P., Schaake, J., Robock, A., Lohmann, D., Cosgrove, B., Duan, Q., Luo, L., Higgins, R., Pinker, R., and Tarpley, J.: Snow process modeling in the North American Land Data Assimilation System (\uppercase{NLDAS}): 2. Evaluation of model simulated snow water equivalent, J. Geophys. Res., 108, 8850, <a href="http://dx.doi.org/10.1029/2003JD003994">https://doi.org/10.1029/2003JD003994</a>, 2003.

Peters-Lidard, C. D., Houser, P. R., Tian, Y., Kumar, S. V., Geiger, J., Olden, S., Lighty, L., Eastman, J. L., Doty, B., Dirmeyer, P., Adams, J., Mitchell, K., Wood, E., and Sheffield, J.: High-performance Earth System modeling with \uppercase{NASA/GSFC}'s Land Information System, Innovations in Systems and Software Engineering, 3, 157–165, 2007.

Peters-Lidard, C., Mocko, D., Garcia, M., Santanello, J., Tischler, M., and Moran, M. S.: Role of precipitation uncertainty in the estimation of hydrologic soil properties using remotely sensed soil moisture in a semiarid environment, Water Resour. Res., 44, W05S18, <a href="http://dx.doi.org/10.1029/2007WR005884">https://doi.org/10.1029/2007WR005884</a>, 2008.

Peters-Lidard, C., Kumar, S., Mocko, D., and Tian, Y.: Estimating evapotranspiration with Land Data Assimilation Systems, Hydrol. Process., 25, 3979–3992, <a href="http://dx.doi.org/10.1002/hyp.8387">https://doi.org/10.1002/hyp.8387</a>, 2011.

Pitman, A. and Henderson-Sellers, A.: Recent progress and results from the Project for the Intercomparison of Land surface Parameterization Schemes, J. Hydrol., 212–213, 128–135, 1998.

Pulliainen, J.: Mapping of snow water equivalent and snow depth in boreal and subarctic zones by assimilating space-borne microwave radiometer data and ground-based observations, Remote Sens. Environ., 101, 257–269, 2006.

Randerson, J., Hoffman, F., Thornton, P., Mahowald, N., Lindsay, K., Lee, Y.-H., Nevison, C., Doney, S., Bonan, G., Stockli, R., Covey, C., Running, S., and Fung, I.: Systematic assessment of terrestrial biogeochemistry in coupled climate-carbon models, Global Change Biol., 15, 2462–2484, <a href="http://dx.doi.org/10.1111/j.1365-2486.2009.01912.x">https://doi.org/10.1111/j.1365-2486.2009.01912.x</a>, 2009.

Raupach, M., Rayner, P., Barrett, D., DeFries, R., Heimann, M., Ojima, D., Quegan, S., and Schmullius, C.: Model-data synthesis in terrestrial carbon observation: methods, data requirements and data uncertainty specifications, Global Change Biol., 11, 378–397, 2005.

Reichle, R. and Crow, W.: Comparison of adaptive filtering techniques for land surface data assimilation, Water Resour. Res., 44, W08423, <a href="http://dx.doi.org/10.1029/2008WR006883">https://doi.org/10.1029/2008WR006883</a>, 2008.

Reichle, R. and Koster, R.: Land data assimilation with the Ensemble Kalman Filter: Assessing model error parameters using innovations, in: Developments in Water Science – Computational Methods in Water Resources, edited by: Hassanizadeh, S., Schotting, R., Gray, W., and Pinder, G., 47, Elsevier, New York, 1387–1394, 2002.

Reichle, R., Koster, R., Liu, P., Mahanama, S., Njoku, E., and Owe, M.: Comparison and assimilation of global soil moisture retrievals from the Advanced Microwave Scanning Radiometer for the Earth Observing System (\uppercase{AMSR-E}) and the Scanning Multichannel Microwave Radiometer (\uppercase{SMMR}), J. Geophys. Res.-Atmos., 112, D09108, <a href="http://dx.doi.org/10.1029/2006JD008033">https://doi.org/10.1029/2006JD008033</a>, 2007.

Reichle, R., Crow, W., and Keppenne, C.: An adaptive ensemble Kalman Filter for soil moisture data assimilation, Water Resour. Res., 44, W03423, <a href="http://dx.doi.org/10.1029/2007WR006357">https://doi.org/10.1029/2007WR006357</a>, 2008.

Robock, A., Luo, L., Wood, E., Wen, F., Mitchell, K., Houser, P., Schaake, J., Lohmann, D., Cosgrove, B., Sheffield, J., Duan, Q., Higgins, R., Pinker, R., Tarpley, J., Basara, J., and Crawford, K.: Evaluation of the North American Land Data Assimilation System over the southern Great Plains during the warm season, J. Geophys. Res., 108, 8846, <a href="http://dx.doi.org/10.1029/2002JD003245">https://doi.org/10.1029/2002JD003245</a>, 2003.

Rodell, M., Famiglietti, J., Chen, J., Seneviratne, S., Viterbo, P., Holl, S., and Wilson, C.: Basin scale estimates of evapotranspiration using \uppercase{GRACE} and other observations, Geophys. Res. Lett., 31, L20504, <a href="http://dx.doi.org/10.1029/2004GL020873">https://doi.org/10.1029/2004GL020873</a>, 2004{a}.

Rodell, M., Houser, P. R., Jambor, U., Gottschalck, J., Mitchell, K., Meng., C.-J., Arsenault, K., Cosgrove, B., Radakovich, J., Bosilovich, M., Entin, J. K., Walker, J. P., Lohmann, D., and Toll, D.: The \uppercase{G}lobal \uppercase{L}and \uppercase{D}ata \uppercase{A}ssimilation \uppercase{S}ystem, B. Am. Meteorol. Soc., 85, 381–394, 2004{b}.

Rosenzweig, C., Tubiello, F., Goldberg, R., Mills, E., and Bloomfield, J.: Increased crop damage in the \uppercase{US} from excess precipitation under climate change, Global Environ. Chang., 12, 197–202, 2002.

Rossow, W. and Schiffer, R.: \uppercase{ISCCP} cloud data products, B. Am. Meteorol. Soc., 72, 2–20, 1991.

Santanello, J., Peters-Lidard, C., Garcia, M., Mocko, D., Tischler, M., Moran, M., and Thoma, D.: Using remotely-sensed estimates of soil moisture to infer soil texture and hydraulic properties across a semi-arid watershed, Remote Sens. Environ., 110, 79–97, 2007.

Santanello, J., Peters-Lidard, C., Kumar, S., Alonge, C., and Tao, W.-K.: A Modeling and Observational Framework for Diagnosing local land-atmosphere coupling on diurnal time scales, J. Hydrometeor., 10, 577–599, <a href="http://dx.doi.org/10.1175/2009JHM1066.1">https://doi.org/10.1175/2009JHM1066.1</a>, 2009.

Santanello, J., Peters-Lidard, C., and Kumar, S.: Diagnosing the Sensitivity of Local Land-Atmosphere Coupling via the Soil Moisture-Boundary Layer Interaction, J. Hydrometeor., 12, 766–786, <a href="http://dx.doi.org/10.1175/JHM-D-10-05014.1">https://doi.org/10.1175/JHM-D-10-05014.1</a>, 2011.

Seyfried, M. and Wilcox, B.: Scale and the nature of spatial variability: Field examples having implications for hydrologic modeling, Water Resour. Res., 31, 173–184, 1995.

Sheffield, J., Pan, M., Wood, E., Mitchell, K., Houser, P., Schaake, J., Robock, A., Lohmann, D., Cosgrove, B., Duan, Q., Luo, L., Higgins, R., Pinker, R., Tarpley, J., and Ramsay, B.: Snow process modeling in the North American Land Data Assimilation System (\uppercase{NLDAS}): 1. Evaluation of model-simulated snow cover extent, J. Geophys. Res., 108, 8849, <a href="http://dx.doi.org/10.1029/2002JD003274">https://doi.org/10.1029/2002JD003274</a>, 2003.

Sivapalan, M. and Kalma, J.: Scale problems in hydrology: Contributions of the Robertson Workshop, Hydrol. Process., 9, 243–250, 1995.

Stanski, H., Wilson, L., and Burrows, W.: Survey of common verification methods in meteorology, Tech. report 8,\uppercase{WMO/TD} 35, WMO World Weather Watch, 114 pp., 1989.

Trujillo, E., Ramirez, J., and Elder, K.: Scaling properties and spatial organization of snow depth fields in sub-alpine forest and alpine tundra, Hydrol. Process., 23, 1575–1590, <a href="http://dx.doi.org/10.1002/hyp.7270">https://doi.org/10.1002/hyp.7270</a>, 2009.

van den Hurk, B., Best, M., Dirmeyer, P., Pitman, A., Polcher, J., and Santanello, J.: Over a decade of \uppercase{GLASS} has accelerated land surface model development, B. Am. Meteorol. Soc., 92, 1593–1600, <a href="http://dx.doi.org/10.1175/BAMS-D-11-00007.1">https://doi.org/10.1175/BAMS-D-11-00007.1</a>, 2011.

Wackerbauer, R., Witt, A., Atmanspacher, H., Kurths, J., and Scheingraber, H.: Comparative classification of complexity measures, Solitons Fractals, 4, 133–173, 1994.

Wealands, S., Grayson, R., and Walker, J.: Quantitative comparison of spatial fields for hydrological model assessment – some promising approaches, Adv. Water Resour., 28, 15–32, <a href="http://dx.doi.org/10.1016/j.advwatres.2004.10.001">https://doi.org/10.1016/j.advwatres.2004.10.001</a>, 2005.

Williams, M., Richardson, A. D., Reichstein, M., Stoy, P. C., Peylin, P., Verbeeck, H., Carvalhais, N., Jung, M., Hollinger, D. Y., Kattge, J., Leuning, R., Luo, Y., Tomelleri, E., Trudinger, C. M., and Wang, Y.-P.: Improving land surface models with FLUXNET data, Biogeosciences, 6, 1341–1359, <a href="http://dx.doi.org/10.5194/bg-6-1341-2009">https://doi.org/10.5194/bg-6-1341-2009</a>, 2009.

Wood, E., Sivapalan, M., Beven, K., and Band, L.: Effects of spatial variability and scale with implications to hydrologic modeling, J. Hydrol., 102, 29–47, 1988.

Wood, E., Sivapalan, M., and Beven, K.: Similarity and scale in catchment storm response, Rev. Geophys., 28, 1–18, <a href="http://dx.doi.org/10.1029/RG028i001p00001">https://doi.org/10.1029/RG028i001p00001</a>, 1990.

Xia, Y., Mitchell, K., Ek, M., Cosgrove, B., Sheffield, J., Luo, L., Alonge, C., Wei, H., Meng, J., Livneh, B., Duan, Y., and Lohmann, D.: Continental-scale water and energy flux analysis and validation for North-American Land Data Assimilation System Project Phase-2, Part 2: Validation of model-simulated streamflow, J. Geophys. Res., 117, D03109, <a href="http://dx.doi.org/10.1029/2011JD016051">https://doi.org/10.1029/2011JD016051</a>, 2012{a}.

Xia, Y., Mitchell, K., Ek, M., Sheffield, J., Wood, E., Cosgrove, B., Luo, L., Alonge, C., Wei, H., Meng, J., Livneh, B., Lettenmaier, D., Koren, V., Duan, Y. K. M., and Fan, Y.: Continental-scale water and energy flux analysis and validation for the North American Land Data Assimilation System Project Phase 2 (NLDAS-2), Part 1: Comparison Analysis and Application of Model Products, J. Geophys. Res., 117, D03110, <a href="http://dx.doi.org/10.1029/2011JD016048">https://doi.org/10.1029/2011JD016048</a>, 2012{b}.

Xie, P. and Arkin, P.: Global precipitation: A 17-year monthly analysis based on gauge observations, satellite estimates, and numerical model outputs, B. Am. Meteorol. Soc., 78, 2539–2558, 1997.