Calibrating and validating the Integrated Valuation of Ecosystem Services and Tradeoffs (InVEST) urban cooling model: case studies in France and the United States

Understanding the cooling service provided by vegetation in cities is important to inform urban policy and planning. However, the performance of decision-support tools estimating heat mitigation for urban greening strategies has not been evaluated systematically. Here, we further develop a calibration algorithm and evaluate the performance of the urban cooling model developed within the open-source InVEST (Integrated Valuation of Ecosystem Services and Tradeoffs) software. The urban cooling model estimates air temperature reduction due to vegetation based on four predictors, shade, evapotranspiration, albedo, and building density, and was designed for data-rich and data-scarce situations. We apply the calibration algorithm and evaluate the model in two case studies (Paris, France, and Minneapolis–St Paul, USA) by examining the spatial correlation between InVEST predictions and reference temperature data at a 1 km horizontal resolution. In both case studies, model performance was high for nighttime air temperatures, which are an important indicator of human wellbeing. After calibration, we found medium performance for surface temperatures during daytime but low performance for daytime air temperatures in both case studies, which may be due to model and data limitations. We illustrate the model adequacy for urban planning by testing its ability to simulate a green infrastructure scenario in the Paris case study. The predicted air temperature change compared well to that of an alternative physics-based model (r²=0.55 and r²=0.85 for daytime and nighttime air temperatures, respectively). Finally, we discuss opportunities and challenges for the use of such parsimonious decision-support tools, highlighting their importance to mainstream ecosystem services information for urban planning.

1 Introduction

The urban heat island has been increasingly documented, varying from 1 to 10 °C in European cities with an average of 6 °C for a sample of 110 European cities (Santamouris, 2016). The phenomenon involves an increase in air and surface temperatures in urban areas due to the modification of the energy budget (Oke, 1982). This has direct health, economic, and energy consumption implications (Lehmann, 2014; Santamouris, 2020) as excessive heat has been associated with increases in energy consumption for cooling purposes, increases in ground-level ozone and particulate matter concentrations, and increases in hospital admissions due to cardiovascular conditions (Hémon and Jougla, 2004; Gosling et al., 2009; Lai and Cheng, 2009; Reid et al., 2012; Santamouris, 2015; Wang et al., 2017; Viguié et al., 2020). To reduce these effects, policymakers and urban planners are increasingly turning to blue–green infrastructure (e.g., street trees, green roofs, and urban parks), a cost-effective option for urban cooling that also produces multiple secondary benefits (Bolund and Hunhammar, 1999; Rosenzweig et al., 2006; Villanueva-Solis, 2017; Corburn, 2009; Cortinovis and Geneletti, 2019).

Blue–green infrastructure influences air temperatures and thermal comfort at several scales. At a local scale – a tree or building – shade can reduce air temperature under the canopy (Kroeger et al., 2018; McDonald et al., 2016; Shashua-Bar and Hoffman, 2000). Street trees can also indirectly improve pedestrian comfort and reduce the use of air conditioning in neighboring buildings, thus avoiding additional heat generation (Viguié et al., 2020). Green roofs and walls change the heat and energy balance of buildings; by absorbing incident solar radiation to support biological functions, vegetation acts as a screen and reduces seasonal temperature variations – although to a limited extent compared to other insulating materials (Eumorfopoulou and Kontoleon, 2009). At a larger scale, urban parks provide an “oasis effect”, reducing air temperatures by up to 6 °C (Ziter et al., 2019; Kroeger et al., 2018; Jauregui, 1990; Eliasson, 1996; Spronken-Smith and Oke, 1999; Potchter et al., 2006; Yu et al., 2020). The effect is influenced by park size (Cao et al., 2010; Yu et al., 2020), composition (Potchter et al., 2006), and local climatic conditions (Shashua-Bar and Hoffman, 2000; Yu et al., 2020). In a systematic literature review, Bowler et al. (2010) showed that parks larger than 2–3 ha are systematically cooler than the rest of the city. Recent work by Wong et al. (2021) suggests a lower threshold of 1 ha. For such large parks, the cooling capacity depends mainly on evapotranspiration and can extend to 800 m, although it is more frequently closer to 100 m (Kroeger et al., 2018; Wong et al., 2021).

Quantitative estimates of the cooling service provided by vegetation come from two main areas. Research in urban climatology examines the physical processes contributing to air cooling (Phelan et al., 2015). These studies often involve complex numerical models – mesoscale or microscale (Meili et al., 2020; Lemonsu et al., 2012) that focus on one or several processes explaining microclimate and increasingly incorporate vegetation effects (Bartesaghi Koc et al., 2018). Such models require long development and calibration processes and therefore significant human and time resources. The same applies to geostatistical models that require large geospatial datasets (e.g., Cheung et al., 2021). On the other hand, an increasing body of literature in the fields of urban ecology and ecosystem services science examines the cooling service through indicators associated with land use (Zardo et al., 2017; Larondelle and Haase, 2013; Derkzen et al., 2015; Nedkov et al., 2017; Farrugia et al., 2013) as reviewed in previous work (Hamel et al., 2021). These models further simplify the physical processes underlying the cooling dynamics, the effect of the morphological parameters of the buildings, and the daily or seasonal temperature variations, often by attributing a “cooling capacity” to a broadly defined land use type.

Cooling service models routinely measure the cooling effect of vegetation using land surface temperature, which is readily accessible from satellite data (Manoli et al., 2019; Zhao et al., 2014). Surface temperature influences thermal comfort and is often correlated with air temperature, making it an interesting proxy for microclimate studies. However, this temperature alone does not suffice to estimate the economic or health implications of the urban heat island (Martilli et al., 2020; Venter et al., 2021), reducing the relevance of models estimating land surface temperature in policy making.

These model limitations lead to a lack of decision-support tools that accurately quantify the cooling effect of vegetation on air temperatures and therefore cannot quantify its impact in socioeconomic terms. A recent review of open-source tools found that only a few ecosystem services software tools were adequate for general urban planning purposes – i.e., applicable in any location and flexible enough to represent different types of blue–green infrastructure and decision contexts (Hamel et al., 2021). InVEST (Integrated Valuation of Ecosystem Services and Tradeoffs) is one of the most popular tools developed to support ecosystem services assessments by quantifying and mapping the benefits provided by blue–green infrastructure in urban or non-urban environments (Natural Capital Project, 2022). The urban cooling model within InVEST, developed by the author team, addresses some of the limitations stated above by estimating air temperature (instead of land surface temperature only), as it is applicable in cities all around the world using readily available data.

The InVEST urban cooling model has recently been applied to an urban European dataset (Cortinovis et al., 2022) and to individual cities in Europe and India (Kadaverugu et al., 2021; Zawadzka et al., 2021; Bosch et al., 2021). However, attempts to validate the model against observed or modeled data are much rarer. In previous work, Bosch et al. (2021) found that the model performed well in Lausanne, although the observation data were sparse, with 11 weather stations over 112 km². Zawadzka et al. (2021) found that the model explained between 48 % and 60 % of the variability in land surface temperature for a summer day in three towns in the United Kingdom. Despite these notable efforts, the model has not been extensively tested and validated against spatialized air temperature data.

The goal of this paper is to evaluate the performance of the InVEST urban cooling model in different contexts: daytime and nighttime and with different levels of data availability. In doing so, we further develop an open-source calibration algorithm to facilitate future applications of the model (Bosch et al., 2021). Given the intended use of InVEST as a decision-support tool, we focus model performance assessment on two aspects: the ability to represent spatial variations in temperatures and the ability to represent temporal changes in temperature due to landscape changes. We test the model through two case studies with contrasting climates: the Paris metropolitan area, France, and the Twin Cities, Minnesota, USA. In the following sections, we present the model performance assessment and discuss the strengths and limitations of the tool and its recommended use to inform urban planning policies.

2 Methods

We assess the performance of the InVEST model for the current land use and land cover (LULC) by comparing its outputs with available air temperature and land surface temperature “best estimates”. Since daytime temperatures are highly influenced by convection and atmospheric turbulence confounding the effect of land use (Le Roy et al., 2020), we hypothesized that the InVEST daytime outputs would better capture variations in land surface temperatures rather than air temperatures. On the contrary, nighttime temperatures are strongly influenced by land cover (in particular built infrastructure) and thermal processes, so we expected to find a stronger correlation between InVEST and nighttime air temperature. Given the purpose of the model to support decision-making, we also assess the model's ability to capture the effect of a change in LULC on urban cooling.

2.1 Case studies

2.1.1 Paris metropolitan area

The Paris metropolitan area, situated in the Île-de-France administrative region, is our main case study. Île-de-France spans over 12 000 km², with the land use being predominantly agriculture (50 %), forests (24 %), and artificial areas (22.5 %) (see Fig. A1 in Appendix A). The climate in Paris is oceanic (humid temperate, no dry season, and warm summers). Average rainfall over the period 1981–2010 was 637 mm, and summer (June to August) temperatures averaged 19.7 °C with increasingly frequent temperature peaks in the middle of summer (Météo-France). The study period is the heat wave of 8–13 August 2003, described in previous work (de Munck et al., 2018), which led to maximum daytime temperatures of over 39 °C in the region. A greening scenario for the Paris metropolitan area was developed by de Munck et al. (2018) to simulate the implementation of low and high vegetation (60 % grass and small shrubs and 40 % deciduous trees) for 50 % of available ground surface (except roads and areas already covered by vegetation). This represents the greening of pavements, squares, car parks, and some roofs over a surface of 199 km², or a 23 % increase in green areas, resulting in a reduction of up to 2 °C in maximum daily mean temperatures over the study area (de Munck et al., 2018).

2.1.2 The Twin Cities

The second case study, used to validate the model in a different climate zone, is in the Twin Cities metropolitan area surrounding the cities of Minneapolis and St Paul in Minnesota, USA (see Fig. A2 in Appendix A). The climate of the Twin Cities is classified as hot-summer humid continental. We studied the regional heat wave event of 22 July 2016 and the average air temperatures over the period 2011–2014, building on previous studies of urban heat islands in the region (Smoliak et al., 2015).

2.2 Model description

The InVEST urban cooling model, hereafter “InVEST model”, is fully described in the InVEST software user manual (Natural Capital Project, 2022). We provide a summary of the key equations here to orient readers. The model computes air temperature prior to air mixing (T_nomix, in degrees Celsius) for each pixel as a function of a background rural reference temperature (T_ref) modified by a local heat factor. The latter is expressed as the maximum urban heat island intensity for the city (UHI_max) modulated by the local heat mitigation (HM) such that for a given pixel i,

$$\begin{array}{}\text{(1)}& T_{\mathrm{nomix}}\left(i\right)=T_{\mathrm{ref}}+\left(\mathrm{1}-\mathrm{HM}\left(i\right)\right)\times {\mathrm{UHI}}_{\max }.\end{array}$$

While this temperature does not account for mixing due to atmospheric turbulence, actual air temperature, T_air, is estimated from T_nomix using a spatial moving average algorithm with search radius r_mix. This radius varies with time depending on the lateral mixing due to atmospheric turbulence and can be estimated through calibration (Sect. 2.4).

The proportion of heat mitigation relative to UHI_max, with values ranging between 0 and 1, is derived from the cooling capacity (CC) of the LULC type for a given pixel and from the proximity to large parks. Following an approach proposed by others (Zardo et al., 2017; Kunapo et al., 2018), the cooling capacity during daytime is expressed as a function of shade, evapotranspiration, and albedo:

$$\begin{array}{}\text{(2)}& \mathrm{CC}\left(i\right)=W_{\mathrm{S}}\cdot S\left(i\right)+W_{\mathrm{A}}\cdot A\left(i\right)+W_{\mathrm{E}}\cdot E\left(i\right),\end{array}$$

where S(i), A(i), and E(i) are unitless indices ranging from 0 to 1 that characterize shade, albedo, and evapotranspiration for the pixel i, respectively. They are each weighted by a coefficient (W_S, W_A, or W_E) constant across the study area. S represents the proportion of shade for a given LULC type, e.g., 1 for a land cover type completely covered by canopy or high buildings and 0 for bare ground. A is the albedo value of the land cover. E is calculated from reference evapotranspiration (a raster called ET₀, in mm) and the crop coefficient (K_c, no unit) for the LULC type and then normalized by the maximum value of reference evapotranspiration in the area of interest according to

$$\begin{array}{}\text{(3)}& E\left(i\right)={\displaystyle \frac{K_{\mathrm{c}}\left(i\right)\cdot {\mathrm{ET}}_{\mathrm{0}}\left(i\right)}{max\left({\mathrm{ET}}_{\mathrm{0}}\right)}}.\end{array}$$

For nighttime temperatures, the model calculates cooling capacity as the complement of building density (B), an indicator representing urban compactness that is highly correlated with heat storage capacity (Wong et al., 2021). Building capacity is normalized across the area and can be obtained from building density data: a value of 1 implies that the pixel is covered by the buildings with the highest energy retention, often the tallest buildings, corresponding to a maximum nighttime urban heat island effect.

$$\begin{array}{}\text{(4)}& \mathrm{CC}\left(i\right)=\mathrm{1}-B\left(i\right)\end{array}$$

To account for the effect of large parks (>2 ha) in both nighttime and daytime, the heat mitigation factor, HM, equals a distance-weighted average of the CC values from surrounding areas (CC_park). The algorithm for this distance-weighted average for a pixel i is as follows:

$$\begin{array}{}\text{(5a)}& {\displaystyle }\mathrm{GA}\left(i\right)=\mathrm{area}\cdot {\sum }_{j\in c\left(d_{\mathrm{cool}}\right)}g\left(j\right),\text{(5b)}& {\displaystyle }{\mathrm{CC}}_{\mathrm{park}}\left(i\right)={\sum }_{j\in c\left(d_{\mathrm{cool}}\right)}g\left(j\right)\cdot {\mathrm{CC}}_j\cdot \exp \left(-d\left(i,j\right)/d_{\mathrm{cool}}\right),\text{(5c)}& {\displaystyle }{\mathrm{HM}}_i=\left\{\begin{array}{ll}{\mathrm{CC}}_i& \text{if}{\mathrm{CC}}_i\ge {\mathrm{CC}}_{{\mathrm{park}}_i}\vee {\mathrm{GA}}_i<\mathrm{2}\mathrm{ha}\\ {\mathrm{CC}}_{{\mathrm{park}}_i}& \text{otherwise},\end{array}\right.\end{array}$$

where GA(i) is the total area of green spaces in a buffer of radius d_cool (the distance over which a green space has a cooling effect); area is the area of a pixel in ha; c(d_cool) is the buffer area of radius d_cool; g(j) is 1 if pixel j is a green space, 0 otherwise; CC_park(i) is the cooling capacity including the influence of parks; and d(i,j) is the distance between pixels i and j.

In plain words, if the amount of green space surrounding a pixel (GA) is less than 2 ha, the value of HM for the pixel equals CC (Eq. 5c), assuming little cooling effect outside the park other than through air mixing due to atmospheric turbulence (defined by r_mix). The threshold size of 2 ha is obtained from the literature (Bowler et al., 2010; see also the “Discussion” section).

2.3 Input and calibration data

2.3.1 Paris metropolitan area

Input data. LULC data for the Île-de-France region were obtained from the regional urban planning agency Mode d'occupation du sol (MOS) for the year 2003 (Fig. S1). Reference evapotranspiration (ET₀) for August was obtained from monthly modeled climatological data for the region, averaged for the 1985–2005 period (ALADIN model; Stéfanon et al., 2015). Using long-term average reference evapotranspiration instead of the 2003 reference evapotranspiration has a limited effect on outputs given that the InVEST model only uses relative values (see Eq. 3), which have lower temporal variability than absolute values. LULC parameter values for shade, crop evapotranspiration coefficient, albedo, and building intensity were assigned based on a combination of expert opinion and literature review (see Appendix B).

Reference temperature data. In the Paris case study, we compared the InVEST model outputs with two datasets of land surface temperature and air temperature. Land surface temperature maps with a 1 km horizontal resolution were retrieved from MODIS satellite products (Wan, 2013). They were obtained for 13 August 2003 during daytime and nighttime since data were missing for the two dates closer to the peak of the heat wave (11 and 12 August). We also considered higher-resolution data from Landsat, but due to their lower frequency (16 d) there were no data from around the study period. MODIS land surface temperature data have been used in many surface urban heat island studies globally and regionally (Chakraborty and Lee, 2019; Li et al., 2017).

Predictions of air temperature maps with a 1 km horizontal resolution were obtained from the simulations performed by de Munck et al. (2018) using the physical land surface model Town Energy Balance/surface–atmosphere exchanges (TEB/SURFEX; Masson, 2000; Lemonsu et al., 2012; Masson et al., 2013). The model computes the energy and water budgets for a geographic domain of 100 km × 100 km centered on Paris with a user-defined regular grid and for natural and urban areas, taking land cover and building characteristics as inputs (Masson, 2000). The outputs used in this study were the 6 d average air temperatures from 8 to 13 August 2003. The 1st and 99th percentiles were extracted from these air temperature data to define the extreme temperature conditions over the domain during the heat wave (to exclude visible outliers). The background reference rural temperature (T_ref) was set to the 1st percentile, while the maximum urban heat island intensity (UHI_max) was the difference between both percentiles. Because the TEB/SURFEX data were available for a 100 km × 100 km window, the rest of the analyses are presented for this domain rather than for the entire Île-de-France region. The three datasets, InVEST, TEB/SURFEX, and MODIS, were resampled using bilinear interpolation to a common 1 km grid for comparison.

2.3.2 The Twin Cities

Input data. We used the 2016 LULC data from the National Land Cover Dataset (NLCD; Homer et al., 2020). Details on LULC parameter values are provided in Appendix B and in previous work (Hamel et al., 2021). Reference evapotranspiration for July 2016 was obtained from globally available data from the Consultative Group on International Agricultural Research (CGIAR) Global Aridity Index and Potential Evapotranspiration (ET0) Climate Database v2.

Reference temperature data. We compared air temperature as modeled by InVEST to two data sources: the midday land surface temperature maps of the study area during a regional heat wave on 22 July 2016, retrieved from Landsat data at a 30 m horizontal resolution for daytime and MODIS for nighttime (Wan, 2013), and rasters of average summertime (June, July, and August, 2011–2014) daytime and nighttime air temperatures across the study area, interpolated from a dense network of temperature sensors (∼ 170 stations over 5000 km², interpolated by co-kriging using impervious surfaces; Smoliak et al., 2015). We projected the reference temperature data to match the InVEST output coordinate system and resampled all datasets using bilinear interpolation to match the resolution (∼ 1 km) of the Smoliak et al. (2015) data. The background reference rural temperature (T_ref) was defined using the average air temperature in July for the metropolitan area obtained from the US National Weather Service (23.2 °C; NOAA, 2020), and the maximum urban heat island intensity (UHI_max) was taken from a global assessment of urban heat islands (2.05 °C; Chakraborty and Lee, 2019).

2.4 Model calibration and performance assessment

The model outputs were first assessed without calibration, using the default parameter values (500, 100 m, 0.2, 0.2, and 0.6, for r_mix, d_cool, W_A, W_E, and W_S, respectively). For calibration, we further developed an optimization algorithm used in previous work (Bosch et al., 2021). The algorithm starts with default parameter values and implements a simulated annealing optimization to derive the values of the five parameters for daytime (r_mix, d_cool, W_A, W_E, and W_S) or the two parameters for nighttime (r_mix and d_cool) until the algorithm converges to a solution that minimizes r² (or a limit of 100 iterations is reached). The main improvement to the calibration tool developed for this study is the ability to use either point data (e.g., a network of stations) or raster data (as is the case for the temperature data in this study) as reference temperatures. Other improvements are minor and of a technical nature. They include testing for the compatibility of user inputs, updating deprecated packages, and improving the code efficiency and readability, as is documented in the source code. The tool reports several performance metrics: mean absolute error (MAE), root mean square error (RMSE), and r². Following previous work (Bosch et al., 2021), we selected these metrics since MAE and RMSE are useful quantifications of the uncertainty in model outputs with physical quantities (expressed in °C), which is important for understanding the impact of errors. However, MAE and RMSE also depend on UHI_max, which means that performance might be artificially good for areas with small urban heat island magnitudes. For this reason, we also report r² (the default performance criterion for the optimization). The source code for the calibration tool can be found at https://github.com/martibosch/invest-ucm-calibration/tree/v0.6.0 (last access: 20 March 2024) and on Zenodo; please refer to the “Code availability ” section. A user guide is available at https://invest-ucm-calibration.readthedocs.io/en/latest/usage.html (last access: 20 March 2024). In addition, we performed one-at-a-time sensitivity analyses to further understand how the calibration parameters influence model outputs. Ranges of variation for each parameter are provided in Appendix C.

To demonstrate the application of the InVEST model in practice and assess the model's ability to represent the effect of a change in LULC, we also examined the effect of a greening strategy on urban cooling in the first case study, where comparison data were available from an alternative model. We used the greening scenario “LHV50” (50 % of impervious areas covered by low and high vegetation) developed and simulated by de Munck et al. (2018) and described in Sect. 2.1. In InVEST, we represented this scenario by changing the LULC properties (shade, albedo, and crop coefficient) for urban categories. Specifically, we estimated the proportion of available ground, as defined above, for each category of the LULC map and computed the weighted average of initial parameter values and values for an urban forest. For example, for the “parks or gardens” LULC category, it was estimated that 15 % of the ground was available, so a weighted average of the shade value for the forest (1) and the original park value (0.5) was computed. The resulting parameter values are shown in the Supplement.

3 Results

3.1 Model performance prior to calibration

3.1.1 Daytime temperatures

For the Paris case study, prior to calibration the InVEST daytime temperatures were moderately correlated with daytime land surface temperature (r²=0.60), with a MAE of 3 °C, but showed no correlation with air temperatures (Table 1). In the Twin Cities, the InVEST results had low correlation with land surface temperature (r²=0.20) but slightly higher correlation for air temperatures (r²=0.29). Of note, comparisons between InVEST and land surface temperature in the Twin Cities revealed a large MAE (11.90 °C), as surface temperatures can be much higher (up to 52 °C) than air temperatures (up to 26 °C) based on the reference temperature data.

Visual observations of the differences between the two maps suggest that the model overestimates the temperatures in forested areas (e.g., to the west and south; see Fig. S1 for the LULC map) and in the dense urban areas in the city of Paris (Fig. 1, center). In the Twin Cities, the model overestimates temperatures in agricultural areas, while underestimating temperatures around wetland areas (see Figs. 2 and S1 for the detailed LULC map). In both case studies, the model exhibited a higher variability than the reference data.

Figure 1 Differences between modeled and reference air temperatures (°C) for the Paris region for daytime and nighttime simulations pre- and post-calibration.

Figure 2 Differences between modeled and reference air temperatures (°C) for the Twin Cities region for daytime and nighttime simulations pre- and post-calibration.

3.1.2 Nighttime temperatures

As expected, the InVEST nighttime temperatures were correlated with both air temperature and surface temperature, although more strongly with air temperature in Paris (r²=0.84 for Paris and r²=0.70 for the Twin Cities; Table 1). The MAE is also significantly lower when calculated with respect to air temperatures than to land surface temperatures for both cases (0.5 vs. 3.0 °C for Paris and 0.5 vs. 2.5 °C for the Twin Cities).

Nevertheless, the maps of the differences between InVEST and nighttime air temperature show important spatial heterogeneities. For Paris, this suggests that the model severely underestimates temperatures in the core area (by about 3 °C) and overestimates temperatures in the new urban developments (by about 1 °C; Figs. 1 and S1). In the Twin Cities, InVEST systematically underestimates temperatures in the city center, outlying developed suburbs, and around bodies of water (e.g., in the west and at the confluence of the Minnesota and Mississippi rivers south of the city center), while overestimating temperatures in the surrounding agricultural countryside (Figs. 2 and S2).

3.2 Model performance after calibration

The model performance remained relatively stable after calibration for nighttime air temperatures in both Paris and the Twin Cities, reaching r² values of 0.84 and 0.73, respectively. Daytime temperatures, however, showed very low correlation with the best estimates in Paris and medium correlation (r²=0.33) in the Twin Cities. The sensitivity analysis conducted for the Paris case study confirmed that most model parameters had only a limited effect on correlations with land surface temperature data, which could explain the limited effect of calibration (Figs. C1 and C2 in Appendix C). Only r_mix had a significant effect on r² (with r² values ranging from 0.30 to 0.62 for r_mix ranging from 100 to 5000 m). Because of the very low r² values for daytime temperatures in Paris, we considered calibration unsuccessful, and we do not report the calibrated values.

The Twin Cities daytime calibrated values of the parameters are reported in Table 2 (UHI_max=11.7 °C). The shade weight remains the highest after calibration, and values only changed by <15 %. Nighttime calibrated parameter values suggest that there is less air mixing, with both r_mix and d_cool having lower values than during daytime in the Twin Cities (Table 2).

Table 1 InVEST model performance for daytime and nighttime air (T_air) and land surface temperatures (LST) before and after calibration. MAE is the mean absolute error. Post-calibration values are shown in parentheses.

Table 2 Calibrated coefficient values for air temperature in the two case studies (n/a – not applicable).

3.3 Effect of the greening scenario

Given the limited effect of calibration on daytime and nighttime air temperature data (see Sect. 3.2), we tested the correlation obtained with temperature estimates from the uncalibrated model. The results were satisfactory, with medium correlation strength between InVEST and the TEB/SURFEX model (r²=0.55 and a mean absolute error of 0.07 °C) for daytime data (Fig. 3). For nighttime data, the correlation between the two models was stronger, with r²=0.85.

Figure 3 Cooling service (°C) during daytime resulting from the greening scenario as simulated by InVEST and by the TEB/SURFEX model (reference data).

4 Discussion

4.1 Model calibration and performance

Based on the two case studies, the InVEST model performance was best (r²>0.73) for nighttime air temperature after calibration. The performance improvement due to calibration was modest in most cases (day or night, air or surface temperature), especially for nighttime air temperature in Paris where the performance was already high prior to calibration (r²=0.84). The calibration significantly improved model performance in the Twin Cities only for surface temperature, possibly due to the LULC configuration in this landscape, although our analyses do not allow us to confirm this hypothesis. The modest effect due to calibration can be explained by a relatively low sensitivity to model parameters, which is suggested by the sensitivity analyses presented in Appendix C (Figs. C3 and C4).

For the Twin Cities, nighttime calibrated parameter values (r_mix and d_cool) are lower than those for daytime, which supports the lower convection and air mixing during the night. Calibrated values of r_mix are 771 and 660 m for daytime and nighttime, respectively, which compares well with the estimate of 600 m obtained from previous studies (Schatz and Kucharik, 2014; Lonsdorf et al., 2021). Similar reasoning to verify the physical interpretation of model parameters could be done in a case study for the shade, albedo, and evapotranspiration weights if such data are available; for example, calibrated values could be compared with the relative proportion of shade to evapotranspiration that has been studied in some cities (e.g., in Singapore; Tan et al., 2018). In both case studies, model performance for surface temperatures (r² ranging from 0.43 to 0.76 post-calibration) compares with the study by Zawadzka et al. (2021), who found that the InVEST model explained between 48 % and 60 % of the variability in land surface temperature for a summer day in three towns in the United Kingdom. We highlight, however, that the model was not initially developed for land surface temperatures. The fair performance for land surface temperatures is an artifact of the model's simplified representation of air temperatures, which imperfectly represents the local energy balance, and of the strong correlation of both air and surface temperatures with LULC.

Regarding scenario assessment, the model in the Paris case study performed reasonably well (r²=0.55 and r²=0.85 for daytime and nighttime air temperatures, respectively). This confirms the potential of the tool to support urban planning studies where different scenarios might need to be compared based on their cooling potential, similar to earlier work with InVEST (Bosch et al., 2022). The effect from the greening scenario, resulting in less than 1 °C cooling, is modest but in line with the literature on such large-scale implementation of green infrastructure, not only from the reference study by de Munck et al. (2018) but also for other temperate climate studies.

4.2 Limitations of the study and future work

In both our case studies, we note that the performance was poor for daytime air temperature, which does not support the use of the model for absolute temperature estimates. This poor performance may be due to oversimplifications of the physical processes involved in urban cooling. In particular, simplification of the flow dynamics means that a model such as InVEST cannot represent urban canyon and wind effects in the city. The effect of parks is also simplified with a threshold of 2 ha for park size. While this has the advantage of reducing the number of model parameters, it also ignores the effect that smaller green spaces might have on their surroundings (Yu et al., 2020; Wong et al., 2021). The poor performance of the model for daytime air temperature may also be attributed to errors in parameterizations, in particular the use of climate data for short periods (e.g., 6–13 August 2003 for Paris) vs. averaged values over several months as was the case for some inputs (e.g., reference evapotranspiration in the Paris case study or T_ref and UHI_max in the Twin Cities; see Sect. 2.3). Further investigation of these temporal dynamics should be explored in future work, although we highlight that they did not seem to impact the fair performance of the model for nighttime air temperatures or land surface temperatures.

An important limitation of any urban climate study examining fine-resolution spatial variations in temperature is the availability of robust reference data. Because air temperature cannot be readily derived from remote sensing data, models are routinely compared to networks of weather stations (de Munck et al., 2018; Smoliak et al., 2015; Bosch et al., 2021). This means that model performance is only assessed for a limited number of points, which are not typically representative of the diversity of LULC in a region. When data are collected specifically for model validation purposes (e.g., transect data), they are also limited by practical factors such as timing considerations (e.g., a mismatch in time between the beginning and end of the transect) or a lack of reference data (Stewart, 2011; Velasco, 2018).

In our study, we have used alternative models as reference data for air temperatures: either a physics-based model or a statistical interpolation model. Both of these models have limitations and uncertainties in and of themselves, making a fine-scale understanding of the limitations of the InVEST model challenging. In other words, the differences in models observed in Figs. 1 and 2 might also be due to errors in the reference data, and future studies could examine the effect of uncertainty in reference datasets on calibration and model performance. For example, the effect of shade during daytime is poorly represented in the TEB/SURFEX model used in Paris (de Munck et al., 2018). Land surface temperature datasets, on the other hand, are less prone to such limitations and are more robust when it comes to spatial distributions. Absolute values are nonetheless challenging to ascertain due to temperature calculations requiring correction algorithms.

Finally, an important study limitation that could be explored in future work is the spatial resolution and scale of case studies. In both cases, the InVEST model was evaluated at a 1 km horizontal resolution over an area of about 100 km by 100 km. Because the model produces outputs at the same resolution as LULC inputs, it would be interesting to evaluate its performance at a different scale and resolution where such data are available. Relatedly, the use of local climate zones, which is common in urban climate studies (Aslam and Rana, 2022), could be examined as an alternative parameterization of the InVEST model. This work would also examine the influence of reference temperature (T_ref) and maximum intensity (UHI_max) as well as the biophysical parameter values (in particular crop coefficients – notably difficult to ascertain for urban land uses), which have not been explored in the present study.

Overall, this discussion of data quality and calibrated parameter values highlights an important contribution of this study to the customized calibration tool available on GitHub (https://github.com/martibosch/invest-ucm-calibration, last access: 20 March 2024) and on Zenodo; please refer to the “Code and data availability” section. Such a tool can be applied to any other city where the InVEST urban cooling model is applied with the only data requirement beyond the InVEST model input data being a reference temperature dataset (either point data or raster data). The calibration tool allows for systematic calibration and model testing, which paves the way for a better understanding of model limitations and strengths. Although in our case studies the performance improvement from calibration was modest, future work could assess the performance of the model over multiple cities with comparable datasets, examine the potential of local climate zones for improved parameterization, or explore finer temporal resolution by linking the night and day model outputs.

5 Conclusion

In this study, we have developed a custom calibration tool to assess the performance of the InVEST urban cooling model in two case studies: in Paris, France, and in the Twin Cities, MN, USA. Our analyses expand on past model testing studies by providing a much more extensive validation dataset of air temperatures (continuous data based on a reference urban climate model in Paris and on data interpolated from a dense weather station network in the Twin Cities). The model showed good performance, assessed through mean absolute error (0.52 °C) and r² (0.84) for nighttime air temperatures. Calibration only slightly improved model performance in the Twin Cities. For the case study of Paris, the use of the tool for scenario assessment was supported by moderate (daytime) and high (nighttime) correlation with change predicted by an alternative physics-based model (r²=0.55 and r²=0.85 for daytime and nighttime air temperatures, respectively). With respect to the study objectives, we conclude that the open-source model can be used to support decisions related to land use and land cover change in cities, with greater reliability of nighttime UHI applications and of relative change (i.e., comparing scenarios to one another as opposed to the using absolute values of model predictions). As these results were obtained for the case studies in Paris and in the Twin Cities, the InVEST model and calibration tools should be tested at other locations to assess model performance for urban planning applications. For research applications, future studies at other locations will help us further understand the effect of data resolution and data quality on model performance.

Appendix A: LULC input data

Figure A1 Land use and cover in the Île-de-France region, France, in 2017 (based on data developed by the Institut Paris Region).

Figure A2 Locations and land cover in the Twin Cities metropolitan area in Minnesota, USA, with the cities of Minneapolis and St. Paul outlined.

Appendix B: InVEST biophysical tables

InVEST biophysical tables can be found in the Supplement.

B1 Paris case study

Albedo values for the Paris biophysical table were derived from the following three sources.

• APUR (2020).

• Lavigne et al. (1994).

• Stewart and Oke (2012).

Specifically, most albedo values were taken from Stewart and Oke (2012). Exceptions were parks, cemeteries, all industrial land use, and all facilities except hippodromes and race tracks (derived from Lavigne et al., 1994), while outdoor courts, transportation land use, quarries, dumpsites, construction sites, hippodromes, and car tracks were derived from APUR (2020).

Crop evapotranspiration values were derived from the FAO Irrigation paper 56 (Allen et al., 1998) as is described in previous work (Tardieu et al., 2020).

• Allen et al. (1998).

• Tardieu et al. (2021).

Shade values were assumed to be 1 for forested or tree-planted land use, 0.5 for mixed land use with trees, 0.25 for commercial land use with planted trees, and 0 for all remaining land use.

B2 Twin Cities case study

Details of the Twin Cities biophysical tables are provided in the supplement of this source: Hamel et al. (2021).

Appendix C: Sensitivity analyses

For the Paris case study, we performed a local sensitivity analysis for the following five parameters (ranges for each parameter are in parentheses).

• Air mixing distance, r_mix (50 to 5000 m).

• Green area maximum cooling distance, d_cool (50 to 1000 m).

• Cooling capacity factors: W_S, W_A, and W_E (0 to 1).

We assessed results based on the correlation between daytime and nighttime temperatures and land surface temperatures. We found that the model was highly sensitive to r_mix (Fig. S1), with a local maximum of 500 m for daytime temperatures and 1000 m for nighttime temperatures (Fig. S3). The model was less sensitive to the values of maximum cooling distance and the cooling capacity factors; r² varied between 0.61 and 0.63 for d_cool and 0.59 and 0.64 for the weight factors.

These results held for nighttime temperature, where InVEST was also most sensitive to the air mixing parameter, r_mix (Fig. S4). Sensitivity to the building density parameter, B, and the maximum cooling distance, d_cool, was much lower.

Figure C1 Sensitivity of the InVEST model to the r_mix and d_cool parameters and W_E and W_S factors. Results are plotted against the correlation coefficient (r²) with daytime land surface temperature (MODIS).

Figure C2 Sensitivity of the InVEST model to the daytime weight factors, W_E (W_eti), W_A (W_albedo), and W_S (W_shade). Results are plotted against the correlation coefficient (r²) with daytime land surface temperature (MODIS).