the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 02 Jun 2022
CLUMondo v2.0: Improved model by adaptive determination of conversion orders for simulating land system changes with many-to-many demand-supply relationships
Abstract. Land resources are fundamentally important to human society, and their transition from one macroscopic state to another is a vital driving force of environment and climate change locally and globally. Thus, many efforts have been devoted to the simulations of land changes. Among all spatially explicit simulation models, CLUMondo is the only one that simulates land changes by incorporating the multifunctionality of a land system and allows the establishment of many-to-many demand-supply relationships. Its central mechanism is complex and has not been fully revealed or clearly explained, thus preventing further improvement. In this study, we first investigated the source code of CLUMondo, providing for the first time the complete, detailed mechanism of this model. More importantly, we found that the featured function of CLUMondo – balancing demands and supplies in a many-to-many mode – relies on a parameter called conversion order. Still, the setting of this parameter should be improved because it is a manual process according to the characteristics of each study area and based on expert knowledge, which is not feasible for users without an understanding of the whole, detailed mechanism. Therefore, the second contribution of this study is the development of an automatic method for adaptively determining conversion orders. We revised the source code of CLUMondo to incorporate the proposed automated method, resulting CLUMondo Version 2.0. Comparative experiments demonstrated the proposed automated method’s validity, high effectiveness, and universal applicability. They showed that the new version of CLUMondo is more effective and easier to use than the existing version. A case study showed that the simulation performance has improved as high as 103.36 %. This study facilitates future improvement on CLUMondo and deep coupling with other earth system models, clearly describing its mechanism. It also helps to exploit the full potential of CLUMondo with a new version.
- Preprint
(3281 KB) - Metadata XML
- BibTeX
- EndNote
Status: closed
-
RC1: 'Comment on gmd-2022-123', Peter H. Verburg, 03 Jun 2022
As one of the developpers of the CLUMondo model it is nice to see others studying our model in a lot of detail. As the model is published as open source with detailed online documentation and manuals we have seen many groups making small or large improvements. As the model is published under the creative commons license it is good practice to make such a deviant version of the model distinguishable from the original developper version of the model. The authors of this paper have not done so, calling it version2.0 is not really something distinguishable from the main versions of this software.
The authors have done a good job in analyzing the source code and the ways they explain the model in a more technical manner as compared to the original paper is mostly correct. The claim that there is no good explanation is not fully correct: the main mechanisms are well documented in the published papers (there are a lot more of these than the authors refer to), their supplementary materials and the user manual available online.
The authors justify the 'new' version by making a very small 'improvement'to the source code. This does not change the functioning of the model. Where a hierarchy of land systems in the original version of the model is manually set by the user they have programmed an automatic approach to determine the hierarchy. They claim different results. This is not correct. The hierarchy can be manually set exactly the same as with their automatic approach (which is basically comparing values and setting the order according to their size). The model should then give exactly the same result. So, the claims are not justified.
Moreover, the accuracy of the simulation is determined by standard kappa values. There is a large literature (Pontius et al.) that has shown that kappa values are not appropriate for validating land use models. Authors have completely ignored this while this is very common knowledge in the literature. Moreover, better performance in one case would not mean 'universal improvement' as the authors are indicating.
But, most important. The 'improvement' is not an 'improvement'. As the authors site the 'Asselen and Verburg 2013' paper for the reasons for setting this manually indicates there is a good reason for enabling the user to set these values manually. As the land system classification and ecosystem service data used are not always fully aligned in terms of resolution, approach etc there are sometimes some small or big biases in the average provisioning of services that a land system provides when this is determined by an overlay method (as applied by the authors). If these would lead to some weird conclusions: a pasture land providing more carbon storage than a peat forest, the user can manually modify this hierarchy. Such a situation can happen as maybe current pasture land is only a few pixels in very special conditions. The automatic method would promote the conversion to pasture land to fulfill a carbon demand. By setting the hierarchy manually this can be avoided. Also, the threshold used to distinguish small from big difference depends on the distribution of values, units etc. Therefore, we have in the past changed the automatic procedure we had in the beta version to a manual version that still allows the same settings, but provides the user with more flexibility. The proposal to make this automatic again is thus, given the history of the model, a step back! (and decreases the flexibility of the approach and user control on the functions).
The authors rightfully claim that an automatic method makes it easier for the user, and the user would need less expertise.
Land systems are complex. Modelling these appropriately requires a deep understanding of system function and model representation. 'making it easy' leads to applications without sufficient understanding and thus flawed results. If one wants to do things 'easy' we refer to the IDRISI land use modeller which is made for that purpose.
Finally, there are many small errors in the manuscript and the writing is far from clear, obscure sentences include: 'We present in this section a subtle and feasible method for the establishment, first qualitatively and then quantitatively. The
qualitative establishment involves two steps: generating land systems based on scale transformations and defining the
305 services of different land systems. The quantitative establishment is to quantify the land system-dependent supply (in 2010)
of and the aggregated demand (in 2020) for each land system service.'. There is no explanation what this 'scale transformations' are and how this is all incorporated. To me there remain more question marks on how the authors have implemented the model.Citation: https://doi.org/10.5194/gmd-2022-123-RC1 -
AC2: 'Reply on RC1', Changqing Song, 05 Sep 2022
1) As one of the developpers of the CLUMondo model it is nice to see others studying our model in a lot of detail. As the model is published as open source with detailed online documentation and manuals we have seen many groups making small or large improvements. As the model is published under the creative commons license it is good practice to make such a deviant version of the model distinguishable from the original developper version of the model. The authors of this paper have not done so, calling it version2.0 is not really something distinguishable from the main versions of this software.
Re: It is our great honor to receive comments from the developers! We believe a revision according to these comments will improve our manuscript’s quality! As for the model name, we understand the comments and agree with the reviewer that “CLUMondo v2.0” is inappropriate. Indeed, the model here should be distinguished from the main version of CLUMondo. Thus, we changed the name of our improved model from “CLUMondo v2.0” to “CLUMondo-BNU” in the revised manuscript (“BNU” stands for the university where all the authors come from).
**********************************
2) The authors have done a good job in analyzing the source code and the ways they explain the model in a more technical manner as compared to the original paper is mostly correct. The claim that there is no good explanation is not fully correct: the main mechanisms are well documented in the published papers (there are a lot more of these than the authors refer to), their supplementary materials and the user manual available online.
Re: We would like to thank the reviewer for his kind appreciation of our study on his source code. We spent a lot of time reading the source code and interpreting the mechanism more detailedly. We hope the detailed interpretation will facilitate future improvements by us and others on this fantastic model of CLUMondo. Because the reviewer pointed out that our “claim that there is no good explanation is not fully correct,” we removed all such claims from the revised manuscript.
**********************************
3) The authors justify the 'new' version by making a very small 'improvement' to the source code. This does not change the functioning of the model. Where a hierarchy of land systems in the original version of the model is manually set by the user they have programmed an automatic approach to determine the hierarchy. They claim different results. This is not correct. The hierarchy can be manually set exactly the same as with their automatic approach (which is basically comparing values and setting the order according to their size). The model should then give exactly the same result. So, the claims are not justified.
Re: Yes, the reviewer is right that we do not change the model’s functioning. Also, the reviewer is right that “the hierarchy can be manually set exactly the same as with their automatic approach.” The following claim is misleading: Our proposed method changed the result of CLUMondo if all parameters (including conversion orders) were the same.
We apologize for these misleading claims. In order to not mislead our readers, we have revised the last paragraph of the introduction as follows: “To facilitate the application of CLUMondo, we developed an automatic method for adaptively determining conversion orders. Evaluation results demonstrated that with this method, users could easily achieve a good simulation performance using CLUMondo. This method benefits not also non-expert but also expert users because its results can serve as a good starting point for fine-tuning conversion orders. We modified the source code of CLUMondo to integrate the proposed method as an option for users (who can still set conversion orders manually). To distinguish the modified CLUMondo from the official version, we referred to this modified one as CLUMondo-BNU (where the abbreviation “BNU” stands for the university of the authors of this paper) and also released it for public use.”
We emphasized that (a) our contribution is to help users in the setting or conversion orders and (b) our proposed method is only an option (and a reference) for users.
**********************************4) Moreover, the accuracy of the simulation is determined by standard kappa values. There is a large literature (Pontius et al.) that has shown that kappa values are not appropriate for validating land use models. Authors have completely ignored this while this is very common knowledge in the literature. Moreover, better performance in one case would not mean 'universal improvement' as the authors are indicating.
Re: Thank you for recommending the literature by Pontius et al. We found that they published a paper entitled “Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment,” although Kappa seems to be the most popular metric in evaluating land change simulation models including CLUMondo. In that paper, the authors suggest employing two new metrics: quantity disagreement and allocation disagreement. The authors referred to the sum of these two metrics as total disagreement, which turns out to be better than Kappa.
Therefore, we included the total disagreement in the revised version of our manuscript (please see the metric description in Section 4.4 and the updated results in Section 4.5). The total disagreement also demonstrated the effectiveness of the proposed method. In additon, we revised our discussion section to include the result of the total disagreement. We want to thank this comment because it made our evaluation results more convincing.
Regarding the term “universal improvement,” we agree with the reviewer that one case is insufficient to support it. Actually, this is also why we presented two cases in the original manuscript. The study area in one case is Henan, and that in the other case is Sichuan. As explained in Section 4.1, the two study areas have distinct characteristics. We agree with the reviewer that such two cases are still not sufficient. Since testing with every possible study area is impractical, we removed the term “universal improvement” from the revised manuscript.
**********************************5) But, most important. The 'improvement' is not an 'improvement'. As the authors site the 'Asselen and Verburg 2013' paper for the reasons for setting this manually indicates there is a good reason for enabling the user to set these values manually. As the land system classification and ecosystem service data used are not always fully aligned in terms of resolution, approach etc there are sometimes some small or big biases in the average provisioning of services that a land system provides when this is determined by an overlay method (as applied by the authors). If these would lead to some weird conclusions: a pasture land providing more carbon storage than a peat forest, the user can manually modify this hierarchy. Such a situation can happen as maybe current pasture land is only a few pixels in very special conditions. The automatic method would promote the conversion to pasture land to fulfill a carbon demand. By setting the hierarchy manually this can be avoided. Also, the threshold used to distinguish small from big difference depends on the distribution of values, units etc. Therefore, we have in the past changed the automatic procedure we had in the beta version to a manual version that still allows the same settings, but provides the user with more flexibility. The proposal to make this automatic again is thus, given the history of the model, a step back! (and decreases the flexibility of the approach and user control on the functions).
Re: This is an excellent comment as it tells us why the developers of CLUMondo leave the setting of conversion orders a manual process for users. The reviewer elaborated in this comment that “there is a good reason for enabling the user to set these values manually.” We completely agree with the reviewer and realized that we should not require users to apply our method. Thank you very much for this comment as a very timely suggestion!
Inspired by this comment, we modified our source code and expression in the manuscript. The proposed method is now an option for users (who can still set conversion orders manually).
**********************************6) The authors rightfully claim that an automatic method makes it easier for the user, and the user would need less expertise.
Re: Thank you for this positive comment. Making the model easier for our users is our purpose and beneficial, especially given that CLUMondo is much useful and increasingly popular.
**********************************
7) Land systems are complex. Modelling these appropriately requires a deep understanding of system function and model representation. 'making it easy' leads to applications without sufficient understanding and thus flawed results. If one wants to do things 'easy' we refer to the IDRISI land use modeller which is made for that purpose.
Re: Yes. What we want to do is not make land change simulation easy. If this is the purpose, the IDRISI land use modeler is indeed of use and seems to be a better choice. We want to make the powerful CLUMondo easier to use, especially for non-expert users.
We agree with the reviewer that “a deep understanding of system function and model representation.” Therefore, in the revised manuscript, we emphasized the necessity for manually determining the conversion orders, and our method is only an option and a reference. Please see the revised text at the end of Section 3.2.
**********************************8) Finally, there are many small errors in the manuscript and the writing is far from clear, obscure sentences include: 'We present in this section a subtle and feasible method for the establishment, first qualitatively and then quantitatively. The qualitative establishment involves two steps: generating land systems based on scale transformations and defining the services of different land systems. The quantitative establishment is to quantify the land system-dependent supply (in 2010) of and the aggregated demand (in 2020) for each land system service.'. There is no explanation what this 'scale transformations' are and how this is all incorporated. To me there remain more question marks on how the authors have implemented the model.
Re: We thank the reviewer for pointing out this lack of clarity. We have rewriten Section 4.2 “Establishment of multifunctional land systems” to explain the implementation. As for the term “scale transformation”, we revised the coresponding sentence to read as follow: “In this study, we generated the taxonomy land systems based on the scale transformation of the GlobeLand30 datasets, or more specifically, by transforming the spatial resolution of the GlobeLand30 datasets from 30 m to 1 km.” In addtion, we improve the clarity of expression throughout the whole manuscript; all changed text has been highlighted in red. We hope that these revisions have addressed the reviewer’s concern, but we are still open to more comments.
Citation: https://doi.org/10.5194/gmd-2022-123-AC2
-
AC2: 'Reply on RC1', Changqing Song, 05 Sep 2022
-
RC2: 'Comment on gmd-2022-123', Anonymous Referee #2, 07 Jun 2022
There are many models for simulating land changes, but CLUMondo is a featured one as it considers many-to-many demand-supply relationships. That means in CLUMondo, users can not only set the demand for different types of land use/cover but also the demand for multiple services provided by different land systems. In this view, the original version of CLUMondo developed by Peter H. Verburg et al is of great use, and improvement and/or modifications should be welcomed.
This study proposed an adaptive method for automatically setting one important parameter in CLUMondo, namely the so-called conversion orders. And this proposed method was coupled by the authors into the original version of CLUMondo, resulting in the so-called CLUMondo v2. Such a topic falls into the scope of “Development and technical papers” of this journal, which is “describe technical developments relating to model improvements such as the speed or accuracy of numerical integration schemes as well as new parameterizations for processes represented in modules.” In this manuscript, the proposed method is clearly described and all source code has been released.
More importantly, the experiment is comprehensive as it involved two study areas with quite different characteristics and tested the key feature of CLUMondo (note that in many studies, CLUMondo was treated simply as CLUE-s, without any many-to-many demand-supply relationships). Also, experimental results demonstrated the effectiveness of the proposed method. Therefore, I suggest accepting it for publication after some revisions.
Specific comments
It can be seen that the authors have fully understood the mechanism of CLUMondo and its source code. The descriptions of the mechanism are impressive and will facilitate future improvements.
I also read the open comment carefully. I agree with the comment that it is better not to refer to the new model as CLUMondo v2. The authors can propose a new name. In fact, although the original version of CLUMondo requires setting the so-called conversion orders manually. It is a way of coupling expert knowledge and should be kept for users as an option. The proposed method in this study serves as important assistance for users, but users should have the option to input their settings. Users can be guided by the results of the proposed method.
In the experiment, the simulation results were assessed by Kappa statistics. It is good as many simulation results by CLUMondo and other models had even not been assessed. Kappa statistics are still quite popular (e.g., Integrating the CLUMondo and InVEST models to assess the impact of the implementation of the Major Function Oriented Zone planning on carbon storage), but the authors can also consider other statistics to improve the reliability of the results or provide another view.
Other possible improvements of CLUMondo should be discussed.
It is also suggested for the authors to provide a manual on how to use their software/source code.
Citation: https://doi.org/10.5194/gmd-2022-123-RC2 -
AC1: 'Reply on RC2', Changqing Song, 05 Sep 2022
There are many models for simulating land changes, but CLUMondo is a featured one as it considers many-to-many demand-supply relationships. That means in CLUMondo, users can not only set the demand for different types of land use/cover but also the demand for multiple services provided by different land systems. In this view, the original version of CLUMondo developed by Peter H. Verburg et al is of great use, and improvement and/or modifications should be welcomed.
This study proposed an adaptive method for automatically setting one important parameter in CLUMondo, namely the so-called conversion orders. And this proposed method was coupled by the authors into the original version of CLUMondo, resulting in the so-called CLUMondo v2. Such a topic falls into the scope of “Development and technical papers” of this journal, which is “describe technical developments relating to model improvements such as the speed or accuracy of numerical integration schemes as well as new parameterizations for processes represented in modules.” In this manuscript, the proposed method is clearly described and all source code has been released.
More importantly, the experiment is comprehensive as it involved two study areas with quite different characteristics and tested the key feature of CLUMondo (note that in many studies, CLUMondo was treated simply as CLUE-s, without any many-to-many demand-supply relationships). Also, experimental results demonstrated the effectiveness of the proposed method. Therefore, I suggest accepting it for publication after some revisions.
Re: We would like to thank the reviewer for his/her kind appreciation of our work.
*************************************
Specific comments
It can be seen that the authors have fully understood the mechanism of CLUMondo and its source code. The descriptions of the mechanism are impressive and will facilitate future improvements.
Re: We would like to thank the reviewer for this comment.
*************************************
I also read the open comment carefully. I agree with the comment that it is better not to refer to the new model as CLUMondo v2. The authors can propose a new name. In fact, although the original version of CLUMondo requires setting the so-called conversion orders manually. It is a way of coupling expert knowledge and should be kept for users as an option. The proposed method in this study serves as important assistance for users, but users should have the option to input their settings. Users can be guided by the results of the proposed method.
Re: Similar suggestions were also made by the first reviewer. These suggestions have been adopted in this study. In the revised version of the manuscript, we distinguished our model from the official version of CLUMondo by changing its name from CLUMondo v2.0 to CLUMondo-BNU.
*************************************
In the experiment, the simulation results were assessed by Kappa statistics. It is good as many simulation results by CLUMondo and other models had even not been assessed. Kappa statistics are still quite popular (e.g., Integrating the CLUMondo and InVEST models to assess the impact of the implementation of the Major Function Oriented Zone planning on carbon storage), but the authors can also consider other statistics to improve the reliability of the results or provide another view.
Re: We would like to thank the reviewer for this insightful comment. We included the total disagreement in the revised version of our manuscript (please see the metric description in Section 4.4 and the updated results in Section 4.5). This metric was proposed in a paper entitled “Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment.” In that paper, the authors suggest employing two new metrics: quantity disagreement and allocation disagreement. The authors referred to the sum of these two metrics as total disagreement. This newly added metric also demonstrated the effectiveness of the proposed method.
*************************************Other possible improvements of CLUMondo should be discussed.
Re: According to this comment, we suggested improving CLUMondo models by considering the spatial heterogeneity of land system services. Actually, this is not the unique disadvantage of CLUMondo but all land change simulation models.
*************************************
It is also suggested for the authors to provide a manual on how to use their software/source code.Re: This suggestion is very useful! We have added a manual for users. We mentioned the manual in the Section “Code and data availability.”
Citation: https://doi.org/10.5194/gmd-2022-123-AC1
-
AC1: 'Reply on RC2', Changqing Song, 05 Sep 2022
Status: closed
-
RC1: 'Comment on gmd-2022-123', Peter H. Verburg, 03 Jun 2022
As one of the developpers of the CLUMondo model it is nice to see others studying our model in a lot of detail. As the model is published as open source with detailed online documentation and manuals we have seen many groups making small or large improvements. As the model is published under the creative commons license it is good practice to make such a deviant version of the model distinguishable from the original developper version of the model. The authors of this paper have not done so, calling it version2.0 is not really something distinguishable from the main versions of this software.
The authors have done a good job in analyzing the source code and the ways they explain the model in a more technical manner as compared to the original paper is mostly correct. The claim that there is no good explanation is not fully correct: the main mechanisms are well documented in the published papers (there are a lot more of these than the authors refer to), their supplementary materials and the user manual available online.
The authors justify the 'new' version by making a very small 'improvement'to the source code. This does not change the functioning of the model. Where a hierarchy of land systems in the original version of the model is manually set by the user they have programmed an automatic approach to determine the hierarchy. They claim different results. This is not correct. The hierarchy can be manually set exactly the same as with their automatic approach (which is basically comparing values and setting the order according to their size). The model should then give exactly the same result. So, the claims are not justified.
Moreover, the accuracy of the simulation is determined by standard kappa values. There is a large literature (Pontius et al.) that has shown that kappa values are not appropriate for validating land use models. Authors have completely ignored this while this is very common knowledge in the literature. Moreover, better performance in one case would not mean 'universal improvement' as the authors are indicating.
But, most important. The 'improvement' is not an 'improvement'. As the authors site the 'Asselen and Verburg 2013' paper for the reasons for setting this manually indicates there is a good reason for enabling the user to set these values manually. As the land system classification and ecosystem service data used are not always fully aligned in terms of resolution, approach etc there are sometimes some small or big biases in the average provisioning of services that a land system provides when this is determined by an overlay method (as applied by the authors). If these would lead to some weird conclusions: a pasture land providing more carbon storage than a peat forest, the user can manually modify this hierarchy. Such a situation can happen as maybe current pasture land is only a few pixels in very special conditions. The automatic method would promote the conversion to pasture land to fulfill a carbon demand. By setting the hierarchy manually this can be avoided. Also, the threshold used to distinguish small from big difference depends on the distribution of values, units etc. Therefore, we have in the past changed the automatic procedure we had in the beta version to a manual version that still allows the same settings, but provides the user with more flexibility. The proposal to make this automatic again is thus, given the history of the model, a step back! (and decreases the flexibility of the approach and user control on the functions).
The authors rightfully claim that an automatic method makes it easier for the user, and the user would need less expertise.
Land systems are complex. Modelling these appropriately requires a deep understanding of system function and model representation. 'making it easy' leads to applications without sufficient understanding and thus flawed results. If one wants to do things 'easy' we refer to the IDRISI land use modeller which is made for that purpose.
Finally, there are many small errors in the manuscript and the writing is far from clear, obscure sentences include: 'We present in this section a subtle and feasible method for the establishment, first qualitatively and then quantitatively. The
qualitative establishment involves two steps: generating land systems based on scale transformations and defining the
305 services of different land systems. The quantitative establishment is to quantify the land system-dependent supply (in 2010)
of and the aggregated demand (in 2020) for each land system service.'. There is no explanation what this 'scale transformations' are and how this is all incorporated. To me there remain more question marks on how the authors have implemented the model.Citation: https://doi.org/10.5194/gmd-2022-123-RC1 -
AC2: 'Reply on RC1', Changqing Song, 05 Sep 2022
1) As one of the developpers of the CLUMondo model it is nice to see others studying our model in a lot of detail. As the model is published as open source with detailed online documentation and manuals we have seen many groups making small or large improvements. As the model is published under the creative commons license it is good practice to make such a deviant version of the model distinguishable from the original developper version of the model. The authors of this paper have not done so, calling it version2.0 is not really something distinguishable from the main versions of this software.
Re: It is our great honor to receive comments from the developers! We believe a revision according to these comments will improve our manuscript’s quality! As for the model name, we understand the comments and agree with the reviewer that “CLUMondo v2.0” is inappropriate. Indeed, the model here should be distinguished from the main version of CLUMondo. Thus, we changed the name of our improved model from “CLUMondo v2.0” to “CLUMondo-BNU” in the revised manuscript (“BNU” stands for the university where all the authors come from).
**********************************
2) The authors have done a good job in analyzing the source code and the ways they explain the model in a more technical manner as compared to the original paper is mostly correct. The claim that there is no good explanation is not fully correct: the main mechanisms are well documented in the published papers (there are a lot more of these than the authors refer to), their supplementary materials and the user manual available online.
Re: We would like to thank the reviewer for his kind appreciation of our study on his source code. We spent a lot of time reading the source code and interpreting the mechanism more detailedly. We hope the detailed interpretation will facilitate future improvements by us and others on this fantastic model of CLUMondo. Because the reviewer pointed out that our “claim that there is no good explanation is not fully correct,” we removed all such claims from the revised manuscript.
**********************************
3) The authors justify the 'new' version by making a very small 'improvement' to the source code. This does not change the functioning of the model. Where a hierarchy of land systems in the original version of the model is manually set by the user they have programmed an automatic approach to determine the hierarchy. They claim different results. This is not correct. The hierarchy can be manually set exactly the same as with their automatic approach (which is basically comparing values and setting the order according to their size). The model should then give exactly the same result. So, the claims are not justified.
Re: Yes, the reviewer is right that we do not change the model’s functioning. Also, the reviewer is right that “the hierarchy can be manually set exactly the same as with their automatic approach.” The following claim is misleading: Our proposed method changed the result of CLUMondo if all parameters (including conversion orders) were the same.
We apologize for these misleading claims. In order to not mislead our readers, we have revised the last paragraph of the introduction as follows: “To facilitate the application of CLUMondo, we developed an automatic method for adaptively determining conversion orders. Evaluation results demonstrated that with this method, users could easily achieve a good simulation performance using CLUMondo. This method benefits not also non-expert but also expert users because its results can serve as a good starting point for fine-tuning conversion orders. We modified the source code of CLUMondo to integrate the proposed method as an option for users (who can still set conversion orders manually). To distinguish the modified CLUMondo from the official version, we referred to this modified one as CLUMondo-BNU (where the abbreviation “BNU” stands for the university of the authors of this paper) and also released it for public use.”
We emphasized that (a) our contribution is to help users in the setting or conversion orders and (b) our proposed method is only an option (and a reference) for users.
**********************************4) Moreover, the accuracy of the simulation is determined by standard kappa values. There is a large literature (Pontius et al.) that has shown that kappa values are not appropriate for validating land use models. Authors have completely ignored this while this is very common knowledge in the literature. Moreover, better performance in one case would not mean 'universal improvement' as the authors are indicating.
Re: Thank you for recommending the literature by Pontius et al. We found that they published a paper entitled “Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment,” although Kappa seems to be the most popular metric in evaluating land change simulation models including CLUMondo. In that paper, the authors suggest employing two new metrics: quantity disagreement and allocation disagreement. The authors referred to the sum of these two metrics as total disagreement, which turns out to be better than Kappa.
Therefore, we included the total disagreement in the revised version of our manuscript (please see the metric description in Section 4.4 and the updated results in Section 4.5). The total disagreement also demonstrated the effectiveness of the proposed method. In additon, we revised our discussion section to include the result of the total disagreement. We want to thank this comment because it made our evaluation results more convincing.
Regarding the term “universal improvement,” we agree with the reviewer that one case is insufficient to support it. Actually, this is also why we presented two cases in the original manuscript. The study area in one case is Henan, and that in the other case is Sichuan. As explained in Section 4.1, the two study areas have distinct characteristics. We agree with the reviewer that such two cases are still not sufficient. Since testing with every possible study area is impractical, we removed the term “universal improvement” from the revised manuscript.
**********************************5) But, most important. The 'improvement' is not an 'improvement'. As the authors site the 'Asselen and Verburg 2013' paper for the reasons for setting this manually indicates there is a good reason for enabling the user to set these values manually. As the land system classification and ecosystem service data used are not always fully aligned in terms of resolution, approach etc there are sometimes some small or big biases in the average provisioning of services that a land system provides when this is determined by an overlay method (as applied by the authors). If these would lead to some weird conclusions: a pasture land providing more carbon storage than a peat forest, the user can manually modify this hierarchy. Such a situation can happen as maybe current pasture land is only a few pixels in very special conditions. The automatic method would promote the conversion to pasture land to fulfill a carbon demand. By setting the hierarchy manually this can be avoided. Also, the threshold used to distinguish small from big difference depends on the distribution of values, units etc. Therefore, we have in the past changed the automatic procedure we had in the beta version to a manual version that still allows the same settings, but provides the user with more flexibility. The proposal to make this automatic again is thus, given the history of the model, a step back! (and decreases the flexibility of the approach and user control on the functions).
Re: This is an excellent comment as it tells us why the developers of CLUMondo leave the setting of conversion orders a manual process for users. The reviewer elaborated in this comment that “there is a good reason for enabling the user to set these values manually.” We completely agree with the reviewer and realized that we should not require users to apply our method. Thank you very much for this comment as a very timely suggestion!
Inspired by this comment, we modified our source code and expression in the manuscript. The proposed method is now an option for users (who can still set conversion orders manually).
**********************************6) The authors rightfully claim that an automatic method makes it easier for the user, and the user would need less expertise.
Re: Thank you for this positive comment. Making the model easier for our users is our purpose and beneficial, especially given that CLUMondo is much useful and increasingly popular.
**********************************
7) Land systems are complex. Modelling these appropriately requires a deep understanding of system function and model representation. 'making it easy' leads to applications without sufficient understanding and thus flawed results. If one wants to do things 'easy' we refer to the IDRISI land use modeller which is made for that purpose.
Re: Yes. What we want to do is not make land change simulation easy. If this is the purpose, the IDRISI land use modeler is indeed of use and seems to be a better choice. We want to make the powerful CLUMondo easier to use, especially for non-expert users.
We agree with the reviewer that “a deep understanding of system function and model representation.” Therefore, in the revised manuscript, we emphasized the necessity for manually determining the conversion orders, and our method is only an option and a reference. Please see the revised text at the end of Section 3.2.
**********************************8) Finally, there are many small errors in the manuscript and the writing is far from clear, obscure sentences include: 'We present in this section a subtle and feasible method for the establishment, first qualitatively and then quantitatively. The qualitative establishment involves two steps: generating land systems based on scale transformations and defining the services of different land systems. The quantitative establishment is to quantify the land system-dependent supply (in 2010) of and the aggregated demand (in 2020) for each land system service.'. There is no explanation what this 'scale transformations' are and how this is all incorporated. To me there remain more question marks on how the authors have implemented the model.
Re: We thank the reviewer for pointing out this lack of clarity. We have rewriten Section 4.2 “Establishment of multifunctional land systems” to explain the implementation. As for the term “scale transformation”, we revised the coresponding sentence to read as follow: “In this study, we generated the taxonomy land systems based on the scale transformation of the GlobeLand30 datasets, or more specifically, by transforming the spatial resolution of the GlobeLand30 datasets from 30 m to 1 km.” In addtion, we improve the clarity of expression throughout the whole manuscript; all changed text has been highlighted in red. We hope that these revisions have addressed the reviewer’s concern, but we are still open to more comments.
Citation: https://doi.org/10.5194/gmd-2022-123-AC2
-
AC2: 'Reply on RC1', Changqing Song, 05 Sep 2022
-
RC2: 'Comment on gmd-2022-123', Anonymous Referee #2, 07 Jun 2022
There are many models for simulating land changes, but CLUMondo is a featured one as it considers many-to-many demand-supply relationships. That means in CLUMondo, users can not only set the demand for different types of land use/cover but also the demand for multiple services provided by different land systems. In this view, the original version of CLUMondo developed by Peter H. Verburg et al is of great use, and improvement and/or modifications should be welcomed.
This study proposed an adaptive method for automatically setting one important parameter in CLUMondo, namely the so-called conversion orders. And this proposed method was coupled by the authors into the original version of CLUMondo, resulting in the so-called CLUMondo v2. Such a topic falls into the scope of “Development and technical papers” of this journal, which is “describe technical developments relating to model improvements such as the speed or accuracy of numerical integration schemes as well as new parameterizations for processes represented in modules.” In this manuscript, the proposed method is clearly described and all source code has been released.
More importantly, the experiment is comprehensive as it involved two study areas with quite different characteristics and tested the key feature of CLUMondo (note that in many studies, CLUMondo was treated simply as CLUE-s, without any many-to-many demand-supply relationships). Also, experimental results demonstrated the effectiveness of the proposed method. Therefore, I suggest accepting it for publication after some revisions.
Specific comments
It can be seen that the authors have fully understood the mechanism of CLUMondo and its source code. The descriptions of the mechanism are impressive and will facilitate future improvements.
I also read the open comment carefully. I agree with the comment that it is better not to refer to the new model as CLUMondo v2. The authors can propose a new name. In fact, although the original version of CLUMondo requires setting the so-called conversion orders manually. It is a way of coupling expert knowledge and should be kept for users as an option. The proposed method in this study serves as important assistance for users, but users should have the option to input their settings. Users can be guided by the results of the proposed method.
In the experiment, the simulation results were assessed by Kappa statistics. It is good as many simulation results by CLUMondo and other models had even not been assessed. Kappa statistics are still quite popular (e.g., Integrating the CLUMondo and InVEST models to assess the impact of the implementation of the Major Function Oriented Zone planning on carbon storage), but the authors can also consider other statistics to improve the reliability of the results or provide another view.
Other possible improvements of CLUMondo should be discussed.
It is also suggested for the authors to provide a manual on how to use their software/source code.
Citation: https://doi.org/10.5194/gmd-2022-123-RC2 -
AC1: 'Reply on RC2', Changqing Song, 05 Sep 2022
There are many models for simulating land changes, but CLUMondo is a featured one as it considers many-to-many demand-supply relationships. That means in CLUMondo, users can not only set the demand for different types of land use/cover but also the demand for multiple services provided by different land systems. In this view, the original version of CLUMondo developed by Peter H. Verburg et al is of great use, and improvement and/or modifications should be welcomed.
This study proposed an adaptive method for automatically setting one important parameter in CLUMondo, namely the so-called conversion orders. And this proposed method was coupled by the authors into the original version of CLUMondo, resulting in the so-called CLUMondo v2. Such a topic falls into the scope of “Development and technical papers” of this journal, which is “describe technical developments relating to model improvements such as the speed or accuracy of numerical integration schemes as well as new parameterizations for processes represented in modules.” In this manuscript, the proposed method is clearly described and all source code has been released.
More importantly, the experiment is comprehensive as it involved two study areas with quite different characteristics and tested the key feature of CLUMondo (note that in many studies, CLUMondo was treated simply as CLUE-s, without any many-to-many demand-supply relationships). Also, experimental results demonstrated the effectiveness of the proposed method. Therefore, I suggest accepting it for publication after some revisions.
Re: We would like to thank the reviewer for his/her kind appreciation of our work.
*************************************
Specific comments
It can be seen that the authors have fully understood the mechanism of CLUMondo and its source code. The descriptions of the mechanism are impressive and will facilitate future improvements.
Re: We would like to thank the reviewer for this comment.
*************************************
I also read the open comment carefully. I agree with the comment that it is better not to refer to the new model as CLUMondo v2. The authors can propose a new name. In fact, although the original version of CLUMondo requires setting the so-called conversion orders manually. It is a way of coupling expert knowledge and should be kept for users as an option. The proposed method in this study serves as important assistance for users, but users should have the option to input their settings. Users can be guided by the results of the proposed method.
Re: Similar suggestions were also made by the first reviewer. These suggestions have been adopted in this study. In the revised version of the manuscript, we distinguished our model from the official version of CLUMondo by changing its name from CLUMondo v2.0 to CLUMondo-BNU.
*************************************
In the experiment, the simulation results were assessed by Kappa statistics. It is good as many simulation results by CLUMondo and other models had even not been assessed. Kappa statistics are still quite popular (e.g., Integrating the CLUMondo and InVEST models to assess the impact of the implementation of the Major Function Oriented Zone planning on carbon storage), but the authors can also consider other statistics to improve the reliability of the results or provide another view.
Re: We would like to thank the reviewer for this insightful comment. We included the total disagreement in the revised version of our manuscript (please see the metric description in Section 4.4 and the updated results in Section 4.5). This metric was proposed in a paper entitled “Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment.” In that paper, the authors suggest employing two new metrics: quantity disagreement and allocation disagreement. The authors referred to the sum of these two metrics as total disagreement. This newly added metric also demonstrated the effectiveness of the proposed method.
*************************************Other possible improvements of CLUMondo should be discussed.
Re: According to this comment, we suggested improving CLUMondo models by considering the spatial heterogeneity of land system services. Actually, this is not the unique disadvantage of CLUMondo but all land change simulation models.
*************************************
It is also suggested for the authors to provide a manual on how to use their software/source code.Re: This suggestion is very useful! We have added a manual for users. We mentioned the manual in the Section “Code and data availability.”
Citation: https://doi.org/10.5194/gmd-2022-123-AC1
-
AC1: 'Reply on RC2', Changqing Song, 05 Sep 2022
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 1,032 | 311 | 52 | 1,395 | 37 | 40 |
- HTML: 1,032
- PDF: 311
- XML: 52
- Total: 1,395
- BibTeX: 37
- EndNote: 40
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1