A validation standard for area of habitat maps for terrestrial birds and mammals

Dahal, Prabhat Raj; Lumbierres, Maria; Butchart, Stuart H. M.; Donald, Paul F.; Rondinini, Carlo

doi:https://doi.org/10.5194/gmd-15-5093-2022

Articles | Volume 15, issue 13

https://doi.org/10.5194/gmd-15-5093-2022

© Author(s) 2022. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/gmd-15-5093-2022

© Author(s) 2022. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 15, issue 13

Methods for assessment of models

|

04 Jul 2022

Methods for assessment of models |

| 04 Jul 2022

A validation standard for area of habitat maps for terrestrial birds and mammals

Prabhat Raj Dahal, Maria Lumbierres, Stuart H. M. Butchart, Paul F. Donald, and Carlo Rondinini

Download

Final revised paper (published on 04 Jul 2022)
Preprint (discussion started on 31 Aug 2021)

Interactive discussion

Status: closed

RC1: 'Comment on gmd-2021-245', Anonymous Referee #1, 15 Oct 2021

The article is well written and clearly describes a two-stage approach for evaluating area of occupancy models with presence only species occurrence data. This is a significant contribution to global biodiversity studies. The authors acknowledge the limitations of the data available and take a pragmatic approach to improving models and their evaluation. It is a well thought through study and the issues of taxonomy, spatial uncertainly and errors related to cross-walking land cover to habitat classes are discussed. Finally the data and code are accessible in keeping with the open science principles.

Line 327 the work "with" is unnecessary in this sentence.

Figure 3 and 4 - No critical but could be helpful - would it be possible to change the colors to a color ramp?

Figure A4 This figure requires more explanation or different graphics to better make the point.

Figure A6 & A7 – says colors are in A5 – in that legend the colors represent taxa, in this with or without validation points A6 Mammals – A7 Birds?

Citation: https://doi.org/10.5194/gmd-2021-245-RC1
CC1: 'Comment on gmd-2021-245', Matthew Rubino, 28 Oct 2021

This manuscript addresses an often overlooked aspect of deductive modeling approaches - evaluation/validation. While many consider deductive approaches archaic, they are many times the only types of cursory mapping methods useful for large numbers of species across large geographies. Large data repositories of species occurrence records are a rich source of information, however, as the author's have illustrated, care must be taken when utilizing such resources as GBIF.

Line 138: 10 point localities as a minimum seems low. Is there any reference for this quantity as an accepted threshold?
Line 223: What does "one off pixel" mean?
Sentence spanning lines 226-228: What does this mean?
Regarding "coordinate uncertainty": Did the author's include coordinate precision in their estimates of coordinate uncertainty? In GBIF, coordinate uncertainty is a data source provided metric. However, if coordinates are provided for locations in decimal degrees, that decimal precision incorporates a quantity of uncertainty. For example, if latitiude and longitude of a given occurrence is provided with two decimal precision (30.72, -90.08) and trailing 0's are not included, the assumed precision will be two decimals. At the equator, two decimal precision of longitude spans a linear distance of approximately 1110 meters. That is more than 3 times the maximum coordinate uncertainty threshold the author's used to filter their occurrence in order to correspond with their land cover spatial resolution.

Citation: https://doi.org/10.5194/gmd-2021-245-CC1
CC2: 'Comment on gmd-2021-245', Everton Miranda, 01 Apr 2022

Comments for both authors and editors
The present study utilizes a public data of mammals and birds to derive a set of tools regarding Area of Habitat, high performance in both taxonomic sets. They used a new, two-step validation procedure for Area of Habitat, which includes first a model-based evaluation of model prevalence and second a validation using species point localities data. I feel like the authors could be more practical and propositional on their interpretations along the discussion. The authors infer from observations made on their models that their method of estimating Area of Habitat is high-performing - information that is of prime interest for conservation. These conclusions are corroborated by their associated data. This study is simple and elegant, and in my mind represents a perfect example of conservation-based range modelling. I think GMD is a good fit for this manuscript and I have no hesitation recommending it for publication. Congratulations.

Citation: https://doi.org/10.5194/gmd-2021-245-CC2
RC2: 'Comment on gmd-2021-245', Everton Miranda, 04 Apr 2022

Comments for both authors and editors

The present study utilizes a public dataset of mammals and birds to derive a tool to estimate the Area of Habitat, which is a model of high performance in both taxonomic sets. They used a new, two-step validation procedure for Area of Habitat, which includes first a model-based evaluation of model prevalence and second a validation using species point localities data. I feel like the authors could be more practical and propositional on their interpretations along the discussion. The authors infer from observations made on their models that their method of estimating Area of Habitat is high-performing - information that is of prime interest for conservation. These conclusions are corroborated by their associated data. This study is simple and elegant, and in my mind represents a perfect example of conservation-based range modelling. I think GMD is a good fit for this manuscript and I have no hesitation recommending it for publication. Congratulations.

Citation: https://doi.org/10.5194/gmd-2021-245-RC2
AC1: 'Comment on gmd-2021-245', Prabhat Raj Dahal, 18 Apr 2022

I would like to thank the reviewers for their valuable comments and also Matthew Rubino for the public comment. Following are my responses to the comments:
Comment 1 from RC1: Line 327 the work "with" is unnecessary in this sentence.
Response: We agree with the reviewer have removed "with".
Comment 2 from RC1: Figure 3 and 4 - No critical but could be helpful - would it be possible to change the colors to a color ramp?
Response: We have changed the color ramp in the revised manuscript.
Comment 3 from RC1: Figure A4 This figure requires more explanation or different graphics to better make the point.
Response: We have changed Figure A4 to add more clarity.
Comment 4 from RC1: Figure A6 & A7 – says colors are in A5 – in that legend the colors represent taxa, in this with or without validation points A6 Mammals – A7 Birds?
Response: We thank the reviewer for pointing out this error. We have corrected this by adding separate legends in Figures A6 and A7 where red represents species without validation points and green represents species with validation points.
Comment from RC2: I feel like the authors could be more practical and propositional on their interpretations along the discussion.
Response: We have added the following sentences in the discussion section to address this comment:
The AOH maps validated in this paper is the largest validation done till date in terms of number of species validated for birds and mammals. These maps will be freely available after the publication of Lumbierres et al. (2021b). We have also provided the metadata for the AOH maps of all the species along with validation statistics which can be used as a guideline by the users while using the AOH maps.
Comment 1 from CC1: Line 138: 10 point localities as a minimum seems low. Is there any reference for this quantity as an accepted threshold?
Response: Previous studies (Rondinini et al., 2011 and Ficetola et al., 2015) have used a minimum of 5 points with at least 1 point falling in a 1 km² grid. We have used a minimum of 10 points with at least 1 point falling in a 100 m² grid. Also, it must be noted that there is a trade-off between number of species included in the validation sample and the minimum number of points per species required to be considered for point validation. As the threshold for minimum number of points per species increases, the number of species in the validation sample decreases. After filtering the point with three different filters, the number of points are already reduced and setting up the minimum number of points too high will reduce the validation sample greatly. With a threshold of 10 we have a good representative validation sample size.
Comment 2 from CC1: Line 223: What does "one off pixel" mean?
Response: We mean one or few suitable habitat pixels falling inside the 300 m buffer during point validation. We have made the sentence simpler now by replacing “one off pixel” by “one or few pixels”.
Comment 3 from CC1: Sentence spanning lines 226-228: What does this mean?
Response: We realized this sentence is complex and made it simpler now and it reads as “It is therefore possible that the species included in the point validation analysis are not representative of the species not included.”
Comment 4 from CC1: Regarding "coordinate uncertainty": Did the author's include coordinate precision in their estimates of coordinate uncertainty?
Response: Yes we did consider the coordinate precision in our estimates of coordinate uncertainty. The points used in the validation have coordinate precision of 4 decimal places.

Citation: https://doi.org/10.5194/gmd-2021-245-AC1

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision

AR by Prabhat Raj Dahal on behalf of the Authors (16 May 2022) Author's response Author's tracked changes Manuscript

ED: Publish subject to technical corrections (26 May 2022) by Christian Folberth

Short summary

This paper describes the validation of area of habitat (AOH) maps produced for terrestrial birds and mammals. The main objective was to assess the accuracy of the maps based on independent data. We used open access data from repositories, such as ebird and gbif to check if our maps were a better reflection of species' distribution than random. When points were not available we used logistic models to validate the AOH maps. The majority of AOH maps were found to have a high accuracy.