SHAFTS (v2022.3): a deep-learning-based Python package for Simultaneous extraction of building Height And FootprinT from Sentinel Imagery
- 1Department of Hydraulic Engineering, Tsinghua University, Beijing 100084, China
- 2Institute for Risk and Disaster Reduction, University College London, Gower Street, WC1E 6BT, UK
- 1Department of Hydraulic Engineering, Tsinghua University, Beijing 100084, China
- 2Institute for Risk and Disaster Reduction, University College London, Gower Street, WC1E 6BT, UK
Abstract. Building height and footprint are two fundamental urban morphological features required by urban climate modeling. Although some statistical methods have been proposed to estimate average building height and footprint from publicly available satellite imagery, they often involve tedious feature engineering, which is hard to achieve efficient knowledge discovery in a changing urban environment with ever-increasing earth observations. In this work, we develop a deep-learning-based (DL) Python package–SHATFS (Simultaneous building Height And FootprinT extraction from Sentinel Imagery) to extract such information. Multi-task DL (MTDL) models are proposed to automatically learn feature representation shared by building height and footprint prediction. Besides, we integrate Digital Elevation Model (DEM) information into developed models to inform models of terrain-induced effects on the backscattering displayed by Sentinel-1 imagery. We set conventional machine-learning-based (ML) models and single-task DL (STDL) models as benchmarks and select 46 cities worldwide to evaluate developed models’ patch-level prediction skills and city-level spatial transferability at four resolutions (100 m, 250 m, 500 m and 1000 m). Patch-level results of 43 cities show that DL models successfully produce more discriminative feature representation and improve the coefficient of determination (R2) of building height and footprint prediction over ML models by 0.27–0.63, 0.11–0.49, respectively. Moreover, stratified error assessment reveals that DL models effectively mitigate severe systematic underestimation of ML models in the high-value domain: for the 100 m case, DL models reduce Root Mean Square Error of building height higher than 40 m and building footprint larger than 0.25 by 31 m and 0.1, respectively, which demonstrates the superiority of DL models on refined 3D building information extraction in highly urbanized area. For the evaluation of spatial transferability, when compared with an existing state-of-the-art product, DL models can achieve similar improvement on the overall performance and high-value prediction. Furthermore, within the DL family, comparison in building height prediction between STDL and MTDL models reveals that MTDL models achieve higher accuracy in all cases and smaller bias uncertainty for the prediction in the high-value domain at the refined scale, which proves the effectiveness of multi-task learning on building height estimation.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(24134 KB)
-
Supplement
(2510 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(24134 KB) -
Supplement
(2510 KB) - BibTeX
- EndNote
Journal article(s) based on this preprint
Ruidong Li et al.
Interactive discussion
Status: closed
-
RC1: 'Comment on gmd-2022-85', Anonymous Referee #1, 11 Sep 2022
This study develops a deep-learning (DL) based Python package-SHAFTS to extract 3D building information (average building height and footprint) from publicly available satellite imagery. Compared to conventional machine learning-based models and single-task DL models, the proposed multi-task DL models can effectively improve the prediction accuracy. This study involves the fusion of multi-source input data and many machine learning and deep learning models, which undoubtedly requires huge and solid work from the authors. Although I am not the expert in computer science, the evaluation framework presented in Section 3 is scientifically sound from my perspective – very quantitative from patch-level to city level. And I will consider using the developed package in the future. I only have the following minor comments.
Minor comments:
- Line 368, the caption of Figure 4, need to denote the source of reference values when calculating RMSE, MAE, etc.
- Figure 5, is the density of scatter points normalized to [10-4, 1] instead of [10-3, 1]?
- Line 385, this paragraph shall better describe the stratified error, which is helpful to the readers who are not familiar with it.
- Line 404-405, “Both SVR and DL models show relatively unfavourable RMSE and NMAD for the low-value domain but DL models behave slightly better.” What does it mean? I think the RMSE and NMAD are both smaller in the low-value domain (Fig. 8).
- Line 463, what is contextual information?
- 12 and 13, why the feature patterns of STDL and MTDL have large differences?
- Line 509, I think STDL model gives better predictions than others from Fig. 15?
-
AC1: 'Reply on RC1', Ruidong Li, 01 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-85/gmd-2022-85-AC1-supplement.pdf
-
AC2: 'Reply on RC2', Ruidong Li, 01 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-85/gmd-2022-85-AC2-supplement.pdf
-
AC3: 'Revised Manuscript', Ruidong Li, 01 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-85/gmd-2022-85-AC3-supplement.pdf
-
RC2: 'Comment on gmd-2022-85', Anonymous Referee #2, 25 Oct 2022
This study develops a deep-learning-based (DL) Python package–SHAFTS to extract average building height and footprint proportion at a pixelated level. I briefly read the paper, and think it is well-written and solid. I, therefore, believe it deserves publication.
-
AC2: 'Reply on RC2', Ruidong Li, 01 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-85/gmd-2022-85-AC2-supplement.pdf
-
AC1: 'Reply on RC1', Ruidong Li, 01 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-85/gmd-2022-85-AC1-supplement.pdf
-
AC3: 'Revised Manuscript', Ruidong Li, 01 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-85/gmd-2022-85-AC3-supplement.pdf
-
AC2: 'Reply on RC2', Ruidong Li, 01 Nov 2022
-
AC3: 'Revised Manuscript', Ruidong Li, 01 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-85/gmd-2022-85-AC3-supplement.pdf
Peer review completion


Interactive discussion
Status: closed
-
RC1: 'Comment on gmd-2022-85', Anonymous Referee #1, 11 Sep 2022
This study develops a deep-learning (DL) based Python package-SHAFTS to extract 3D building information (average building height and footprint) from publicly available satellite imagery. Compared to conventional machine learning-based models and single-task DL models, the proposed multi-task DL models can effectively improve the prediction accuracy. This study involves the fusion of multi-source input data and many machine learning and deep learning models, which undoubtedly requires huge and solid work from the authors. Although I am not the expert in computer science, the evaluation framework presented in Section 3 is scientifically sound from my perspective – very quantitative from patch-level to city level. And I will consider using the developed package in the future. I only have the following minor comments.
Minor comments:
- Line 368, the caption of Figure 4, need to denote the source of reference values when calculating RMSE, MAE, etc.
- Figure 5, is the density of scatter points normalized to [10-4, 1] instead of [10-3, 1]?
- Line 385, this paragraph shall better describe the stratified error, which is helpful to the readers who are not familiar with it.
- Line 404-405, “Both SVR and DL models show relatively unfavourable RMSE and NMAD for the low-value domain but DL models behave slightly better.” What does it mean? I think the RMSE and NMAD are both smaller in the low-value domain (Fig. 8).
- Line 463, what is contextual information?
- 12 and 13, why the feature patterns of STDL and MTDL have large differences?
- Line 509, I think STDL model gives better predictions than others from Fig. 15?
-
AC1: 'Reply on RC1', Ruidong Li, 01 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-85/gmd-2022-85-AC1-supplement.pdf
-
AC2: 'Reply on RC2', Ruidong Li, 01 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-85/gmd-2022-85-AC2-supplement.pdf
-
AC3: 'Revised Manuscript', Ruidong Li, 01 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-85/gmd-2022-85-AC3-supplement.pdf
-
RC2: 'Comment on gmd-2022-85', Anonymous Referee #2, 25 Oct 2022
This study develops a deep-learning-based (DL) Python package–SHAFTS to extract average building height and footprint proportion at a pixelated level. I briefly read the paper, and think it is well-written and solid. I, therefore, believe it deserves publication.
-
AC2: 'Reply on RC2', Ruidong Li, 01 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-85/gmd-2022-85-AC2-supplement.pdf
-
AC1: 'Reply on RC1', Ruidong Li, 01 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-85/gmd-2022-85-AC1-supplement.pdf
-
AC3: 'Revised Manuscript', Ruidong Li, 01 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-85/gmd-2022-85-AC3-supplement.pdf
-
AC2: 'Reply on RC2', Ruidong Li, 01 Nov 2022
-
AC3: 'Revised Manuscript', Ruidong Li, 01 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-85/gmd-2022-85-AC3-supplement.pdf
Peer review completion


Journal article(s) based on this preprint
Ruidong Li et al.
Data sets
Reference Datasets for: SHAFTS (v2022.3): a deep-learning-based Python package for Simultaneous extraction of building Height And FootprinT from Sentinel Imagery Ruidong Li, Ting Sun https://doi.org/10.5281/zenodo.6587510
Model code and software
Simultaneous building Height And FootprinT extraction from Sentinel Imagery Ruidong Li, Ting Sun https://github.com/LllC-mmd/3DBuildingInfoMap
Ruidong Li et al.
Viewed
HTML | XML | Total | Supplement | BibTeX | EndNote | |
---|---|---|---|---|---|---|
699 | 263 | 21 | 983 | 56 | 7 | 8 |
- HTML: 699
- PDF: 263
- XML: 21
- Total: 983
- Supplement: 56
- BibTeX: 7
- EndNote: 8
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(24134 KB) - Metadata XML
-
Supplement
(2510 KB) - BibTeX
- EndNote