Articles | Volume 16, issue 8
https://doi.org/10.5194/gmd-16-2215-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/gmd-16-2215-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Model description paper
| 
24 Apr 2023
Model description paper |
| 24 Apr 2023
| 24 Apr 2023
Structural k-means (S k-means) and clustering uncertainty evaluation framework (CUEF) for mining climate data
Quang-Van Doan
CORRESPONDING AUTHOR
Center for Computational Sciences, University of Tsukuba, Tsukuba, Japan
Toshiyuki Amagasa
Center for Computational Sciences, University of Tsukuba, Tsukuba, Japan
Thanh-Ha Pham
University of Science, Vietnam National University, Hanoi, Vietnam
Takuto Sato
Center for Computational Sciences, University of Tsukuba, Tsukuba, Japan
Fei Chen
Research Applications Laboratory, National Center for Atmospheric
Research, Boulder, USA
Hiroyuki Kusaka
Center for Computational Sciences, University of Tsukuba, Tsukuba, Japan
Related authors
Lidia Lazarova Vitanova, Darin Pavlov, Tereza K. Trendafilova, Ryota Karube, Koki Nakamura, Agata Zabuska, Quang-Van Doan, Dessislava Petrova-Antonova, and Sylvia Ilieva
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., XII-4-W2-2026, 211–218, https://doi.org/10.5194/isprs-annals-XII-4-W2-2026-211-2026, https://doi.org/10.5194/isprs-annals-XII-4-W2-2026-211-2026, 2026
Koki Nakamura, Lidia Lazarova Vitanova, Quang-Van Doan, and Dessislava Petrova-Antonova
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., L-4-W2-2026, 139–146, https://doi.org/10.5194/isprs-archives-L-4-W2-2026-139-2026, https://doi.org/10.5194/isprs-archives-L-4-W2-2026-139-2026, 2026
Lidia Lazarova Vitanova, Kunihiko Fujiwara, Ryota Karube, Evgeny Shirinyan, Yasuhiko Azegami, Quang-Van Doan, Hiroyuki Kusaka, Dessislava Petrova-Antonova, and Sylvia Ilieva
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., L-4-W2-2026, 245–252, https://doi.org/10.5194/isprs-archives-L-4-W2-2026-245-2026, https://doi.org/10.5194/isprs-archives-L-4-W2-2026-245-2026, 2026
Lidia Lazarova Vitanova, Darin Pavlov, Tereza K. Trendafilova, Ryota Karube, Koki Nakamura, Agata Zabuska, Quang-Van Doan, Dessislava Petrova-Antonova, and Sylvia Ilieva
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., XII-4-W2-2026, 211–218, https://doi.org/10.5194/isprs-annals-XII-4-W2-2026-211-2026, https://doi.org/10.5194/isprs-annals-XII-4-W2-2026-211-2026, 2026
Ryota Karube, Lidia Lazarova Vitanova, Evgeny Shirinyan, Takuto Sato, Dessislava Petrova-Antonova, and Hiroyuki Kusaka
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., L-4-W3-2026, 67–74, https://doi.org/10.5194/isprs-archives-L-4-W3-2026-67-2026, https://doi.org/10.5194/isprs-archives-L-4-W3-2026-67-2026, 2026
Koki Nakamura, Lidia Lazarova Vitanova, Quang-Van Doan, and Dessislava Petrova-Antonova
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., L-4-W2-2026, 139–146, https://doi.org/10.5194/isprs-archives-L-4-W2-2026-139-2026, https://doi.org/10.5194/isprs-archives-L-4-W2-2026-139-2026, 2026
Lidia Lazarova Vitanova, Kunihiko Fujiwara, Ryota Karube, Evgeny Shirinyan, Yasuhiko Azegami, Quang-Van Doan, Hiroyuki Kusaka, Dessislava Petrova-Antonova, and Sylvia Ilieva
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., L-4-W2-2026, 245–252, https://doi.org/10.5194/isprs-archives-L-4-W2-2026-245-2026, https://doi.org/10.5194/isprs-archives-L-4-W2-2026-245-2026, 2026
Dario Di Santo, Cenlin He, Fei Chen, and Lorenzo Giovannini
Geosci. Model Dev., 18, 433–459, https://doi.org/10.5194/gmd-18-433-2025, https://doi.org/10.5194/gmd-18-433-2025, 2025
Short summary
Short summary
This paper presents the Machine Learning-based Automated Multi-method Parameter Sensitivity and Importance analysis Tool (ML-AMPSIT), a computationally efficient tool that uses machine learning algorithms for sensitivity analysis in atmospheric models. It is tested with the Weather Research and Forecasting (WRF) model coupled with the Noah-Multiparameterization (Noah-MP) land surface model to investigate sea breeze circulation sensitivity to vegetation-related parameters.
Yuya Takane, Yukihiro Kikegawa, Ko Nakajima, and Hiroyuki Kusaka
Geosci. Model Dev., 17, 8639–8664, https://doi.org/10.5194/gmd-17-8639-2024, https://doi.org/10.5194/gmd-17-8639-2024, 2024
Short summary
Short summary
A new parameterisation for dynamic anthropogenic heat and electricity consumption is described. The model reproduced the temporal variation in and spatial distributions of electricity consumption and temperature well in summer and winter. The partial air conditioning was the most critical factor, significantly affecting the value of anthropogenic heat emission.
Cenlin He, Prasanth Valayamkunnath, Michael Barlage, Fei Chen, David Gochis, Ryan Cabell, Tim Schneider, Roy Rasmussen, Guo-Yue Niu, Zong-Liang Yang, Dev Niyogi, and Michael Ek
Geosci. Model Dev., 16, 5131–5151, https://doi.org/10.5194/gmd-16-5131-2023, https://doi.org/10.5194/gmd-16-5131-2023, 2023
Short summary
Short summary
Noah-MP is one of the most widely used open-source community land surface models in the world, designed for applications ranging from uncoupled land surface and ecohydrological process studies to coupled numerical weather prediction and decadal climate simulations. To facilitate model developments and applications, we modernize Noah-MP by adopting modern Fortran code and data structures and standards, which substantially enhance model modularity, interoperability, and applicability.
Zhe Zhang, Yanping Li, Fei Chen, Phillip Harder, Warren Helgason, James Famiglietti, Prasanth Valayamkunnath, Cenlin He, and Zhenhua Li
Geosci. Model Dev., 16, 3809–3825, https://doi.org/10.5194/gmd-16-3809-2023, https://doi.org/10.5194/gmd-16-3809-2023, 2023
Short summary
Short summary
Crop models incorporated in Earth system models are essential to accurately simulate crop growth processes on Earth's surface and agricultural production. In this study, we aim to model the spring wheat in the Northern Great Plains, focusing on three aspects: (1) develop the wheat model at a point scale, (2) apply dynamic planting and harvest schedules, and (3) adopt a revised heat stress function. The results show substantial improvements and have great importance for agricultural production.
Xinlei He, Yanping Li, Shaomin Liu, Tongren Xu, Fei Chen, Zhenhua Li, Zhe Zhang, Rui Liu, Lisheng Song, Ziwei Xu, Zhixing Peng, and Chen Zheng
Hydrol. Earth Syst. Sci., 27, 1583–1606, https://doi.org/10.5194/hess-27-1583-2023, https://doi.org/10.5194/hess-27-1583-2023, 2023
Short summary
Short summary
This study highlights the role of integrating vegetation and multi-source soil moisture observations in regional climate models via a hybrid data assimilation and machine learning method. In particular, we show that this approach can improve land surface fluxes, near-surface atmospheric conditions, and land–atmosphere interactions by implementing detailed land characterization information in basins with complex underlying surfaces.
Cited articles
Arthur, D. and Vassilvitskii, S.: k-means
Barua, D. K.: Beaufort Wind Scale, in: Encyclopedia of Coastal Science,
edited by: Finkl, C. W. and Makowski, C., Springer International Publishing,
Cham, 315–317, https://doi.org/10.1007/978-3-319-93806-6_45,
2019.
Bradley, P. S. and Fayyad, U. M.: Refining Initial Points for K-Means
Clustering, in: Proc. 15th International Conf. on Machine Learning, Morgan Kaufmann, San Francisco, CA, 91–99, 1998.
Camus, P., Menéndez, M., Méndez, F. J., Izaguirre, C., Espejo, A.,
Cánovas, V., Pérez, J., Rueda, A., Losada, I. J., and Medina, R.: A
weather-type statistical downscaling framework for ocean wave climate, J.
Geophys. Res.-Oceans, 119, 7389–7405, https://doi.org/10.1002/2014JC010141,
2014.
Chan, E. Y., Ching, W. K., Ng, M. K., and Huang, J. Z.: An optimization
algorithm for clustering using weighted dissimilarity measures,
Pattern Recogn., 37, 943–952, https://doi.org/10.1016/j.patcog.2003.11.003, 2004.
Choi, K.-S., Cha, Y.-M., and Kim, T.-R.: Cluster analysis of tropical
cyclone tracks around Korea and its climatological properties, Nat. Hazards,
64, 1–18, https://doi.org/10.1007/s11069-012-0192-7, 2012.
Cordeiro de Amorim, R. and Mirkin, B.: Minkowski metric, feature weighting
and anomalous cluster initializing in K-Means clustering, Pattern Recogn.,
45, 1061–1075, https://doi.org/10.1016/j.patcog.2011.08.012, 2012.
de Amorim, R. C.: A Survey on Feature Weighting Based K-Means Algorithms, J.
Classif., 33, 210–242, https://doi.org/10.1007/s00357-016-9208-4, 2016.
Dee, D. P., Uppala, S. M., Simmons, A. J., Berrisford, P., Poli, P., Kobayashi, S., Andrae, U.,
Balmaseda, M. A., Balsamo, G., Bauer, P., Bechtold, P., Beljaars, A. C. M., van de Berg, L., Bidlot, J.,
Bormann, N., Delsol, C., Dragani, R., Fuentes, M., Geer, A. J., Haimberger, L., Healy, S. B.,
Hersbach, H., Hólm, E. V., Isaksen, L., Kållberg, P., Köhler, M., Matricardi, M., McNally, A. P.,
Monge-Sanz, B. M., Morcrette, J.-J., Park, B.-K., Peubey, C., de Rosnay, P., Tavolato, C., Thépaut,
J.-N., and Vitart, F.: The ERA-Interim reanalysis: configuration and performance of the
data assimilation system, Q. J. Roy. Meteor. Soc., 137, 553–597, https://doi.org/10.1002/qj.828, 2011.
Doan, Q.-V.: Structural k-means algorithm (v1.0), Zenodo [code], https://doi.org/10.5281/zenodo.6976609, 2022.
Doan, Q.-V., Kusaka, H., Sato, T., and Chen, F.: S-SOM v1.0: a structural self-organizing map algorithm for weather typing, Geosci. Model Dev., 14, 2097–2111, https://doi.org/10.5194/gmd-14-2097-2021, 2021.
Eltibi, M. F. and Ashour, W. M.: Initializing k-means clustering algorithm
using statistical information, Int. J. Comput. Appl., 29, 51–55,
https://doi.org/10.5120/3573-4930,
2011.
Esteban, P., Jones, P. D., Martín-Vide, J., and Mases, M.: Atmospheric
circulation patterns related to heavy snowfall days in Andorra, Pyrenees,
Int. J. Climatol., 25, 319–329, https://doi.org/10.1002/joc.1103, 2005.
Fahim, A. M., Salem, A. M., Torkey, F. A., and Ramadan, M. A.: An efficient
enhanced k-means clustering algorithm, J. Zhejiang Univ.-Sc. A, 7,
1626–1633, https://doi.org/10.1631/jzus.2006.A1626, 2006.
Forgy, E. W.: Cluster analysis of multivariate data: efficiency versus
interpretability of classifications, Biometrics, 21, 768–769, 1965.
Gibson, P. B., Perkins-Kirkpatrick, S. E., Uotila, P., Pepler, A. S., and
Alexander, L. V.: On the use of self-organizing maps for studying climate
extremes, J. Geophys. Res.-Atmos., 122, 3891–3903,
https://doi.org/10.1002/2016JD026256, 2017.
Gutiérrez, J. M., San-Martín, D., Brands, S., Manzanas, R., and
Herrera, S.: Reassessing Statistical Downscaling Techniques for Their Robust
Application under Climate Change Conditions, J. Climate, 26, 171–188,
https://doi.org/10.1175/JCLI-D-11-00687.1, 2013.
Han, F. and Szunyogh, I.: A Technique for the Verification of Precipitation
Forecasts and Its Application to a Problem of Predictability, Mon. Weather
Rev., 146, 1303–1318, https://doi.org/10.1175/MWR-D-17-0040.1, 2018.
Hassani, M. and Seidl, T.: Using internal evaluation measures to validate
the quality of diverse stream clustering algorithms, Vietnam J. Comput.
Sci., 4, 171–183, https://doi.org/10.1007/s40595-016-0086-9, 2017.
Holten, D.: Hierarchical Edge Bundles: Visualization of Adjacency Relations
in Hierarchical Data, IEEE T. Vis. Comput. Gr., 12, 741–748,
https://doi.org/10.1109/TVCG.2006.147, 2006.
Houssos, E. E., Lolis, C. J., and Bartzokas, A.: Atmospheric circulation patterns associated with extreme precipitation amounts in Greece, Adv. Geosci., 17, 5–11, https://doi.org/10.5194/adgeo-17-5-2008, 2008.
Huang, J. Z., Ng, M. K., Rong, H., and Li, Z.: Automated variable weighting
in k-means type clustering, IEEE T. Pattern Anal., 27,
657–668, https://doi.org/10.1109/TPAMI.2005.95, 2005.
Jancey, R. C.: Multidimensional group analysis, Aust. J. Bot., 14, 127–130,
1966.
Kannan, S. and Ghosh, S.: Prediction of daily rainfall state in a river
basin using statistical downscaling from GCM output,
Stoch. Env. Res.
Risk A., 25, 457–474, https://doi.org/10.1007/s00477-010-0415-y, 2011.
Kantardzic, M.: Data mining: concepts, models, methods, and algorithms, John
Wiley & Sons, https://www.wiley.com/ense/Data+Mining:+Concepts,+Models,+Methods,+and+Algorithms,+3rd+Edition-p-9781119516040 (last access: 20 February 2023), 2011.
Katsavounidis, I., Jay Kuo, C.-C., and Zhang, Z.: A new initialization
technique for generalized Lloyd iteration, IEEE Signal Proc. Lett., 1,
144–146, https://doi.org/10.1109/97.329844, 1994.
Khan, S. S. and Ahmad, A.: Cluster center initialization algorithm for
K-means clustering, Pattern Recogn. Lett., 25, 1293–1302,
https://doi.org/10.1016/j.patrec.2004.04.007, 2004.
Kim, H.-K. and Seo, K.-H.: Cluster Analysis of Tropical Cyclone Tracks over
the Western North Pacific Using a Self-Organizing Map, J. Climate, 29,
3731–3751, https://doi.org/10.1175/JCLI-D-15-0380.1, 2016.
Kim, H.-S., Kim, J.-H., Ho, C.-H., and Chu, P.-S.: Pattern Classification of
Typhoon Tracks Using the Fuzzy c-Means Clustering Method, J. Climate, 24,
488–508, https://doi.org/10.1175/2010JCLI3751.1, 2011.
Lai, J. Z. C. and Huang, T.-J.: Fast global k-means clustering using cluster
membership and inequality, Pattern Recogn., 43, 1954–1963,
https://doi.org/10.1016/j.patcog.2009.11.021, 2010.
Le Roux, R., Katurji, M., Zawar-Reza, P., Quénol, H., and Sturman, A.:
Comparison of statistical and dynamical downscaling results from the WRF
model, Environ. Model. Softw., 100, 67–73,
https://doi.org/10.1016/j.envsoft.2017.11.002, 2018.
Lloyd, S. P.: Least square quantization in PCM. Bell Telephone Laboratories
Paper, Lloyd, SP: Least squares
quantization in PCM, IEEE Trans Inf. Theor19571982, 18, 11, 1957.
MacQueen, J.: Some methods for classification and analysis of multivariate
observations, in: Proceedings of the fifth Berkeley symposium on
mathematical statistics and probability, University of California Press, 281–297, 1967.
Mo, R., Ye, C., and Whitfield, P. H.: Application potential of four
nontraditional similarity metrics in hydrometeorology, J. Hydrometeorol.,
15, 1862–1880, 2014.
Overpeck, J. T., Meehl, G. A., Bony, S., and Easterling, D. R.: Climate Data
Challenges in the 21st Century, Science, 331, 700–702,
https://doi.org/10.1126/science.1197869, 2011.
Pelleg, D. and Moore, A. W.: X-means: Extending K-means with Efficient Estimation
of the Number of Clusters, in: Proceedings of the Seventeenth International Conference on
Machine Learning (ICML '00), Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 727–734, https://www.cs.cmu.edu/~dpelleg/download/xmeans.pdf (last access: 6 July 2022), 2000.
Perez, J., Mexicano, A., Santaolaya, R., Hidalgo, M., Moreno, A., and Pazos,
R.: Improvement to the K-Means algorithm through a heuristics based on a bee
honeycomb structure, in: 2012 Fourth World Congress on Nature and
Biologically Inspired Computing (NaBIC), 2012 Fourth World Congress on
Nature and Biologically Inspired Computing (NaBIC), Mexico City, Mexico, 5–9 November 2012, 175–180,
https://doi.org/10.1109/NaBIC.2012.6402258, 2012.
Pérez-Ortega, J., Almanza-Ortega, N. N., Vega-Villalobos, A.,
Pazos-Rangel, R., Zavala-Díaz, C., and Martínez-Rebollar, A.: The
K-means algorithm evolution, in: Introduction to Data Science and Machine
Learning, IntechOpen, https://doi.org/10.5772/intechopen.85447, 2019.
Pomee, M. S. and Hertig, E.: Precipitation projections over the Indus River
Basin of Pakistan for the 21st century using a statistical downscaling
framework, Int. J. Climatol., 42, 289–314,
https://doi.org/10.1002/joc.7244, 2022.
Romano, S., Vinh, N. X., Bailey, J., and Verspoor, K.: Adjusting for chance
clustering comparison measures, J. Mach. Learn. Res., 17, 4635–4666, 2016.
Rousseeuw, P. J.: Silhouettes: a graphical aid to the interpretation and
validation of cluster analysis, J. Comput. Appl. Math., 20, 53–65, 1987.
Selim, S. Z. and Ismail, M. A.: K-Means-Type Algorithms: A Generalized
Convergence Theorem and Characterization of Local Optimality, IEEE T.
Pattern Anal., PAMI-6, 81–87,
https://doi.org/10.1109/TPAMI.1984.4767478, 1984.
Smith, E. T., Lee, C. C., Barnes, B. B., Adams, R. E., Pirhalla, D. E.,
Ransibrahmanakul, V., Hu, C., and Sheridan, S. C.: A Synoptic Climatological
Analysis of the Atmospheric Drivers of Water Clarity Variability in the
Great Lakes, J. Appl. Meteorol. Clim., 59, 915–935,
https://doi.org/10.1175/JAMC-D-19-0156.1, 2020.
Spekat, A., Kreienkamp, F., and Enke, W.: An impact-oriented classification
method for atmospheric patterns, Phys. Chem. Earth, 35, 352–359,
https://doi.org/10.1016/j.pce.2010.03.042, 2010.
Su, T. and Dy, J. G.: In search of deterministic methods for initializing
K-means and Gaussian mixture clustering, Intell. Data Anal., 11, 319–338,
https://doi.org/10.3233/IDA-2007-11402, 2007.
Sydow, A.: Tou, JT/Gonzalez, RC, Pattern Recognition Principles,
London-Amsterdam-Dom Mills, Ontario-Sydney-Tokyo, Addison-Wesley Publishing
Company, Z. Angew. Math. Mech., 57, 353–354,
1977.
Vinh, N. X. and Epps, J.: A novel approach for automatic number of clusters
detection in microarray data based on consensus clustering, in: 2009 Ninth
IEEE International Conference on Bioinformatics and BioEngineering, Taichung, Taiwan, 22–24 June 2009, 84–91, https://doi.org/10.1109/BIBE.2009.19,
2009.
Vinh, N. X., Epps, J., and Bailey, J.: Information theoretic measures for
clusterings comparison: Variants, properties, normalization and correction
for chance, J. Mach. Learn. Res., 11, 2837–2854, 2010.
Wang, Z. and Bovik, A. C.: Mean squared error: Love it or leave it? A new
look at signal fidelity measures, IEEE Signal Proc. Mag., 26, 98–117,
2009.
Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P.: Image quality
assessment: from error visibility to structural similarity,
IEEE T. Image Process., 13, 600–612, 2004.
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H.,
McLachlan, G. J., Ng, A., Liu, B., Yu, P. S., Zhou, Z.-H., Steinbach, M.,
Hand, D. J., and Steinberg, D.: Top 10 algorithms in data mining,
Knowl. Inf. Syst., 14, 1–37, https://doi.org/10.1007/s10115-007-0114-2, 2008.
Zeng, S., Vaughan, M., Liu, Z., Trepte, C., Kar, J., Omar, A., Winker, D., Lucker, P., Hu, Y., Getzewich, B., and Avery, M.: Application of high-dimensional fuzzy k-means cluster analysis to CALIOP/CALIPSO version 4.1 cloud–aerosol discrimination, Atmos. Meas. Tech., 12, 2261–2285, https://doi.org/10.5194/amt-12-2261-2019, 2019.
Short summary
This study proposes (i) the structural k-means (S k-means) algorithm for clustering spatiotemporally structured climate data and (ii) the clustering uncertainty evaluation framework (CUEF) based on the mutual-information concept.
This study proposes (i) the structural k-means (S k-means) algorithm for clustering...
