Bidirectional coupling of the long-term integrated assessment model REgional Model of INvestments and Development (REMIND) v3.0.0 with the hourly power sector model Dispatch and Investment Evaluation Tool with Endogenous Renewables (DIETER) v1.0.2

Integrated assessment models (IAMs) are a central tool for the quantitative analysis of climate change mitigation strategies. However, due to their global, cross-sectoral and centennial scope, IAMs cannot explicitly represent the temporal and spatial details required to properly analyze the key role of variable renewable energy (VRE) in decarbonizing the power sector and enabling emission reductions through end-use electrification. In contrast, power sector models (PSMs) can incorporate high spatiotemporal resolutions but tend to have narrower sectoral and geographic scopes and shorter time horizons. To overcome these limitations, here we present a novel methodology: an iterative and fully automated soft-coupling framework that combines the strengths of a long-term IAM and a detailed PSM. The key innovation is that the framework uses the market values of power generations and the capture prices of demand flexibilities in the PSM as price signals that change the capacity and power mix of the IAM. Hence, both models make endogenous investment decisions, leading to a joint solution. We apply the method to Germany in a proof-of-concept study using the IAM REgional Model of INvestments and Development (REMIND) v3.0.0 and the PSM Dispatch and Investment Evaluation Tool with Endogenous Renewables (DIETER) v1.0.2 and confirm the theoretical prediction of almost-full convergence in terms of both decision variables and (shadow) prices. At the end of the iterative process, the absolute model difference between the generation shares of any generator type for any year is < 5 % for a simple configuration (no storage, no flexible demand) under a “proof-of-concept” baseline scenario and 6 %–7 % for a more realistic and detailed configuration (with storage and flexible demand). For the simple configuration, we mathematically show that this coupling scheme corresponds uniquely to an iterative mapping of the Lagrangians of two power sector optimization problems of different time resolutions, which can lead to a comprehensive model convergence of both decision variables and (shadow) prices. The remaining differences in the two models can be explained by a slight mismatch between the standing capacities in the real world and optimal modeling solutions based purely on cost competition. Since our approach is based on fundamental economic principles, it is also applicable to other IAM–PSM pairs.

1 Introduction

Thanks to decade-long policy support in many regions of the world and technological learning, the costs of both wind power and solar photovoltaics have plummeted (IEA, 2021; Lazard, 2021). These types of variable electricity generation are now highly cost-competitive against other alternatives, such that their deployment is increasingly driven by market forces instead of climate policies. Among the newly added renewable generations in 2020, nearly two-thirds were cheaper than the cheapest new fossil fuel (IRENA, 2020). Due to both cost declines and pressing concerns over climate change, investing in these clean and abundant resources has become a crucial part of national and regional strategies to decarbonize the power sector (The White House, 2021; Cherp et al., 2021; National long-term strategies, 2022; Rechsteiner, 2021; ICCSD Tsinghua University, 2022).

Given this dramatic development in the power sector over the past 2 decades, a universal consensus has emerged among energy transition scholars and policy makers: emissions in the power sector are relatively “easy-to-abate” (Luderer et al., 2018; Azevedo et al., 2021; Clarke et al., 2022). Compared with other primarily non-electrified end-use sectors such as buildings, transport and industry, the technologies required to transform the power sector are low-cost, mature and readily available. This trend has in recent years led to a second emerging consensus: the power sector will be the fundamental basis for a future low-cost, efficient and climate-neutral energy system (Brown et al., 2018b; Ram et al., 2018; Ramsebner et al., 2021; Luderer et al., 2022a). In addition to direct electrification, which requires end-use transformations of currently non-electrified demand, emerging technological developments in hydrogen and e-fuels produced from renewable electricity have also contributed to the broadening of potential technology portfolios for the “hard-to-abate” sectors, such as high-temperature heat and chemical productions (Parra et al., 2019; Bhaskar et al., 2020; Griffiths et al., 2021). Together, direct and indirect electrification supports a broad concept of “sector coupling”, which facilitates decarbonization by powering end-use demand with variable renewable energy sources (Ramsebner et al., 2021).

Due to the pivotal role of electrification and sector coupling in mitigation scenarios, there is an increasing demand on the scope and level of detail of energy–economy models used to guide energy transition and climate policies. The models would ideally encompass a global, multi-decadal and multi-sectoral scope such that the scenarios are relevant for international and regional climate policies while simultaneously incorporating a high level of spatiotemporal detail. The latter is important to account for the specifics of variable renewable electricity generation as well as its physical and economic interplay with the electrification of energy demand (Li and Pye, 2018; Brunner et al., 2020; Prol and Schill, 2021; Böttger and Härtel, 2022; Ruhnau, 2022). This need for improved modeling methods or frameworks, which has to overcome the trade-off between scope and detail, is a substantial methodological challenge. It entails realizing two main objectives.

•  

  Objective (1). Accurately model the power sector transformation over long time horizons in terms of investment and dispatch, especially at high shares of variable renewable energy (VRE) sources. Long-term pathways for the following power sector quantities and prices should accurately incorporate short-term hourly details:

  • a.

    capacity and generation mix of the power sector;

  • b.

    market values (annual average revenues per power generation unit) for variable and dispatchable plants;

  • c.

    capacity factors of the dispatchable plants and the curtailment rates of variable renewables; and

  • d.

    storage capacity and dispatch.

•  

  Objective (2). Accurately model direct electrification of end-use sectors as well as indirect electrification technologies such as green hydrogen production, where existing and emerging sources of power demand can in part be flexibilized.

1.1 Current modeling approaches and limitations

Current energy system models broadly fall into two distinct categories carried out by two research communities with little institutional overlap: integrated assessment models (IAMs) and power sector models (PSMs), each with their own strengths and weaknesses. IAMs are comprehensive models of a global scale and span multiple decades, linking macroeconomics, energy systems, land use and environmental impacts (Stehfest et al., 2014; Calvin et al., 2017; Huppmann et al., 2019; Baumstark et al., 2021; Keppo et al., 2021; Guivarch et al., 2022), thereby providing an “integrated assessment” of multiple factors (Rotmans and van Asselt, 2001). IAMs substantially shape the IPCC assessments of long-term climate mitigation scenarios and play an important role in policy making (Rogelj et al., 2018; UNEP, 2019; NGFS, 2022; IPCC, 2022). In comparison to IAMs, PSMs typically have narrower spatial and sectoral scopes and shorter time horizons but provide higher resolutions and increased technological detail (Palzer and Henning, 2014; Zerrahn and Schill, 2017; Brown et al., 2018a; Ram et al., 2018; Sepulveda et al., 2018; Blanford and Weissbart, 2019; Böttger and Härtel, 2022; Ringkjøb et al., 2018; Prina et al. 2020). (See also Sect. S5 in the Supplement for a comparison of model specifications of a few selected PSMs.) This allows PSMs to more accurately model the power sector under high-VRE shares (Bistline, 2021; Chang et al., 2021). Note that we use the term “power sector model” here to represent all general smaller-scope models than IAMs (usually by geographical or time horizon measures), even though many of them have sector-coupling aspects and do not only contain the traditional power sector.

IAMs and PSMs are therefore limited by a lack of spatiotemporal detail and a lack of scope, respectively. IAMs usually have a temporal resolution no shorter than a year (Keppo et al., 2021) and therefore include simplified representations of hourly power sector variability that mimic the real-world dynamics to varying degrees of success (Pietzcker et al., 2017). In general, a lack of high temporal resolutions can lead to difficulties when estimating the optimal level of variable renewable generation, often either overestimating or underestimating the market value of solar or wind generation, the challenges of variable renewable integration, the peak hourly residual demand, and the need for energy storage and base load (Pina et al., 2011; Haydt et al., 2011; Ludig et al., 2011; Kannan and Turton, 2013; Welsch et al., 2014; Luderer et al., 2017; Pietzcker et al., 2017; Bistline, 2021). While approximate methods such as parameterization via residual load duration curves (RLDCs) are able to capture the supply-side dynamics of VREs, they remain methodologically limited for representing the flexible demand-side dynamics (Ueckerdt et al., 2015, 2017; Creutzig et al., 2017). Besides limited temporal resolutions, IAMs also usually have coarse spatial resolutions, which can lead to underestimation or overestimation of transmission grid bottlenecks, geographical variability of wind and solar resources, and the flexibility requirements for balancing supply and demand (Aryanpur et al., 2021; Frysztacki et al., 2021; Martínez-Gordón et al., 2021). PSMs, on the other hand, usually lack the global and sectoral scopes required for addressing global climate mitigation, in part because of limited availability of detailed data and due to computational challenges. Furthermore, PSMs with a short-term horizon may lack the vintage tracking of standing capacities, capacity evolution over time and long-term perfect foresight, which can help policy makers and companies to look ahead beyond the short-term business cycles, to invest early and to actively drive technical progress. In contrast, in IAMs such as REMIND, proactive early investment is a built-in feature, because the optimization is done from a long-term social planner's perspective. In IAMs, investing early in the technological learning phase results in lower costs of energy expenditure later, avoiding the severity of punishment to economic growth later in time in the form of lower consumption, which raises the welfare which the model optimizes.

1.2 Iterative coupling for full model convergence

IAMs and PSMs differ in scope and resolution across the three main modeling dimensions: temporal, spatial and technological. A soft-coupling approach can tap into these complementarities and combine their strengths at potentially only a moderate increase in computational cost. The main challenge of the soft-coupling approach is to show that the two models can converge under coupling, which leads to a joint equilibrium that maximizes regional interannual intertemporal welfare in the IAM and minimizes total power system costs in the PSM. Ideally, the converged model offers the “best of both worlds”: it has both the broad scope required to assess global long-term energy transitions and the technical resolution required to capture the interplay between VREs, storage and newly electrified demand on a much shorter timescale.

Approaches aiming to bridge the “temporal resolution gap” between long-term energy system models and hourly PSMs have been proposed in the past (Deane et al., 2012; Sullivan et al., 2013; Alimou et al., 2020; Brinkerink et al., 2020; Seljom et al., 2020; Guo et al., 2022; Younis et al., 2022; Brinkerink et al., 2022; Mowers et al., 2023). While these achieved some aspects of Objective (1) with adequate results, none attempted to incorporate and achieve Objective (2). In addition, there is a methodological gap in the previous attempts at a full harmonization of the multiscale models. By a full harmonization, we mean a comprehensive coupling of the power sector dynamics and an eventual model convergence in capacities, generation and prices. In only a few previous studies, price information has been fed back into the long-term models from the short-term models: in one study only partial price information has been exchanged (Seljom et al., 2020) and in another study some subset of price information is exchanged, but they are not fully endogenized (instead, they are parameterized) and the exchange is also unidirectional (Mowers et al., 2023). Without a feedback mechanism through prices, the investment in the coupled model will very likely be suboptimal due to two effects. (1) Because of the misalignment in prices in the two models, there is a mismatch in investment incentives, resulting in a mismatch for optimal capacities if both models are completely endogenous. (2) In all previous studies, the capacities are fixed in the PSM, and only the long-term model is allowed to invest in new capacities. This implementation can further propagate and sustain the price mismatch due to (1) nontrivial shadow prices from these capacity bounds, in turn creating price distortions in the PSM that can be passed on to the IAM. Therefore, the methodological gap in previous work prevented a comprehensive convergence of the coupled models of both quantities and prices. As we show later in this study, without a comprehensive coupling of price information, no system-wide convergence can be achieved. However, with price coupling as our method proposes, we could achieve all aspects of Objective (1) and Objective (2) for one type of flexible demand with adequate numerical results, thereby representing a first step in bridging the previous methodological gap.

Compared to previous studies, our approach features three main innovations. (1) The coupling is achieved by linking market values and not hard-fixing quantities, allowing both models to invest “as endogenously as possible”. (2) The market values of all power sector technologies are coupled, not just the electricity price of the system or the market value of a particular technology, allowing models to achieve close to full convergence. (3) Under idealized coupling assumptions and for a simplified “proof-of-concept” model without storage, we can mathematically derive the necessary conditions under which comprehensive model convergence can be reached, which puts multiscale coupling on a firm theoretical footing. Our coupling approach is bidirectional, iterative and fully automated.

One should note that our methodology bears certain mathematical similarities to Benders decomposition from the discipline of operation research (Conejo et al., 2006), which is used in the long-term energy system model PRIMES (Price-Induced Market Equilibrium System) to obtain hourly detail (E3Mlab, 2018). There are however crucial differences. For example, the optimization in our work is carried out iteratively outside solver time, whereas the Benders decomposition is carried out iteratively during solver time. In addition, our approach can function even when the objective function is convex, whereas the Benders decomposition cannot, allowing our approach to be applied in more general cases. Mathematically, the subproblems in Benders decomposition have fixed capacities obtained from master problems and therefore are not endogenous, but the shadow prices of these constraints are iteratively passed on to the master problems, ensuring mathematical convergence. The exact ways in which our methodology is connected to Benders decomposition or other similar methods are yet to be fully explored.

To showcase such a framework and its ability to achieve iterative convergence, we couple the PSM Dispatch and Investment Evaluation Tool with Endogenous Renewables (DIETER), which has an hourly resolution (8760 h in a year), with the IAM REMIND for a single region (Germany). Germany is a well-suited case study for exploring high VRE shares in the power sector. The country is expected to meet stringent climate targets despite its high level of residential and industrial power demand, relatively small geographical size, and lack of solar endowment during winter seasons. Nevertheless, the German government has set very ambitious targets for the expansion and use of variable renewable energy sources (Schill et al., 2022). A viable zero-carbon power mix in Germany must include an adequate amount of storage and transmission for renewable generation as well as “clean firm generation” such as geothermal, biomass or gas with carbon capture and storage (CCS) (Sepulveda et al., 2018).

2 Models

The models used in this study are well-documented open-source models (REMIND is an open-source model but requires proprietary input data to run). A side-by-side comparison of the scope, resolution and other specifications of the two models can be found in Appendix A. The coupling scope can be found in Appendix B. Details on model input data can be found in Sect. S1.

2.1 IAM: REMIND

REMIND is a process-based IAM that describes complex global energy–economy–climate interactions (Baumstark et al., 2021). REMIND has frequently been used in long-term planning of decarbonization scenarios, most notably in the IPCC (IPCC, 2014; Rogelj et al., 2018; IPCC, 2022). The REMIND model links different modules that describe the global economy, energy, land and climate systems to a relatively detailed representation of the energy sector compared to non-process-based IAMs. The model is formulated as an interannual intertemporal optimization problem. Due to the computational complexity of nonlinear optimization, the model simulates a time span from 2005 to 2100 with a temporal resolution of either 5 years (between 2005 and 2060) or 10 years (between 2070 and 2100). The years in REMIND are representative years of the surrounding 5- or 10-year period; e.g., year “2030” represents the 5-year period 2028 to 2032. Spatially, the model represents the world composed of aggregated global regions (Fig. B1). For each region, using a nested constant elasticity of substitution (CES) production function, the model maximizes interannual intertemporal welfare as a function of labor, capital and energy use (Baumstark et al., 2021). The macroeconomic projections of REMIND come from various established global socioeconomic scenarios jointly used by social scientists and economists – the so-called shared socioeconomic pathways (SSPs) (Bauer et al., 2017).

By default, REMIND runs in a regionally decentralized iterative “Nash mode” where all regions are run in parallel and the interannual intertemporal welfare is maximized for each region for each internal “Nash” iteration. Trade flows between the regions are determined between the Nash iterations. During the Nash algorithm, REMIND regions share partial information with each other, i.e., trade variables in primary energy products and goods. The Nash algorithm is said to converge when all markets are cleared and no region has an incentive to change its behavior regarding its trade decisions; i.e., no resources can be reallocated to make one region better off without making at least one region worse off. A successfully converged run of standalone REMIND in Nash mode usually consists of 30 to 70 iterations of single-region models in parallel. Each parallel single-region model usually takes 3–6 min to solve. A typical REMIND run in Nash mode lasts 2.5–6 h, depending on the level of sectoral details included. The latest version of REMIND (v3.0.0) is published as an open-source version on GitHub (release REMIND v3.0.0 ⋅ remindmodel/remind, 2022). REMIND is implemented as a nonlinear programming (NLP) mathematical optimization problem. In REMIND, the nonlinearity consists of the welfare function, the CES production functions, adjustment costs, technological learning, the extraction cost functions, the bioenergy supply function and nonlinear constraints, among others.

2.2 PSM: DIETER

DIETER is an open-source power sector model developed for Germany and Europe. In a long-running equilibrium setting (i.e., a competitive benchmark), the model minimizes the overall system costs of the power sector for 1 year. DIETER determines the least-cost investment and hourly dispatch of various power-generation, storage and demand-side flexibility technologies. In the previous literature, different versions of the model have been used to explore scenarios with high-VRE shares, where storage (Zerrahn et al., 2018; Zerrahn and Schill, 2017; Schill and Zerrahn, 2018), hydrogen (Stöckl et al., 2021), power to heat (Schill and Zerrahn, 2020) or solar prosumage (Say et al., 2020; Günther et al., 2021) are evaluated with a high degree of technological detail. DIETER recently also contributed to model comparison exercises that focused on power sector flexibility for VRE integration and sector coupling (Gils et al., 2022b, a; van Ouwerkerk et al., 2022).

As a first step in building a model-coupling infrastructure, we implemented an earlier and simpler version of DIETER (v1.0.2) that is purely based on the General Algebraic Modeling System (GAMS). It has limited features in ramping constraints, flexible demand and storage. The model minimizes the total investment and dispatch cost of a power system for a single region considering all consecutive hours of 1 full year. The technology portfolio contains conventional generators such as coal and gas power plants, nuclear power as well as renewable sources such as hydroelectric power, solar photovoltaics (PV) and wind turbines. Endogenous storage investment and dispatch as well as demand flexibilizations are offered as additional features that can be turned on or off. DIETER, like many PSMs, is a linear program (LP). A typical standalone run (with essential features) lasts from several seconds to several minutes for a single region. See Zerrahn and Schill (2017) for detailed documentation of the initial model, which was implemented purely in GAMS. Later, DIETER's GAMS core was embedded in a Python wrapper for enhanced scenario analysis and postprocessing, but the model can still be run in GAMS-only mode (Gaete-Morales et al., 2021).

3 A novel coupling approach

It is central to our approach that the price-based variables, such as the market values of electricity generation, are exchanged between the models. This approach ensures full convergence – including both quantity convergence and price convergence in the market equilibrium. Here, we first introduce the intuition behind this approach and then conduct a deep dive into the economic theory behind energy system modeling.

Economic concepts such as market values or capture prices (Böttger and Härtel, 2022), as key variables in our coupling, translate the physical characteristics of variable power generation or flexible consumption into economic ones. For example, generation technologies differ with respect to physical features and constraints – solar and wind generation depends on current weather conditions as well as diurnal and seasonal patterns, whereas this is less the case for dispatchable power plants such as coal, gas, biomass, nuclear or storage (López Prol and Schill, 2021). One consequence of this is that, for example, prices in hours where PV does not produce will essentially be set by other and usually more expensive forms of generation. In cost-minimizing PSMs, the shadow prices of the energy balance are interpreted as wholesale market prices (Brown and Reichenberg, 2021; López Prol and Schill, 2021). Therefore, in general, hourly resolution PSMs are well equipped to translate such physical constraints of generation into (wholesale) power market price time series. By providing such prices generated by PSMs (among other variables of the power sector dynamics) to IAMs, the latter can be indirectly informed about power market dynamics happening on much shorter timescales, even if they lack hourly resolutions. Over iterations, the prices from PSMs act as “price signals” to induce investment decision changes in IAMs, which can in turn provide feedback to the PSMs until the two models converge.

One innovation of our method is that the prices used for the model coupling can be symmetrically applied on the power supply side and on the demand side. On the supply side, the coupling method mainly utilizes the concept of a market value (i.e., the annual average revenue per energy unit of a generator) in a competitive market at equilibrium. Generally speaking, market values of generation usually convey the degree of variability intrinsic to a given source of power supply and reflect the generator's ability to meet an inflexible hourly demand, especially given the lower cost of variable generation compared to dispatchable technologies. Mirroring the concept of the market value, on the demand side, there is the concept of the “capture price” of electricity demand, which conveys the degree of demand-side flexibility. Note that there may be multiple terminologies for demand-side electricity prices; we use “capture price” to be consistent with one example of the literature on this topic. The capture price is the average electricity price that a flexible demand technology pays over a year. For example, flexible demand technologies such as heat pumps, electrolyzers or electric vehicles (EVs) can take advantage of electricity at hours when the generation is cheap in order to obtain a lower capture price, whereas inflexible demand has to pay a higher price on average. Price information given from a PSM to an IAM from both the supply and demand sides can change the IAM's inherent investment and dispatch decisions of power generation as well as inflexible and flexible demand-side technologies.

For an intuitive understanding of our innovative coupling scheme, we take the supply side as an example and use a toy model to visualize the approach of coupling via market values. The market values of electricity-generating technologies have been studied in depth (Sensfuß, 2007; Sensfuß et al., 2008; Hirth, 2013; Mills and Wiser, 2015; Hildmann et al., 2015; Koutstaal and va. Hout, 2017; Figueiredo and Silva, 2018; Hirth, 2018; Brown and Reichenberg, 2021). The general idea of the coupling is illustrated in Fig. 1 for a simplified case of only two types of generators – dispatchable gas turbines and solar photovoltaics with variable output. Note that we assume the system to be at a solar share of > 50 % and with no storage, such that the solar market value is below the average electricity price and that of gas generation is above it. Before the coupling, for a general IAM with coarse temporal resolution and without any VRE integration cost parameterizations, there is no differentiation between the market values of gas and solar generators – they are both equal to the electricity price. Thus, there is no differentiated revenue for 1 MWh generated by variable sources and dispatchable sources. The lack of market value differentiation is a direct consequence of the limited temporal resolution in IAMs, which cannot represent hourly dynamics. However, through a market-value-based coupling, the IAM can be informed by the PSM via a price “markup”. The annual price markup is defined as the difference between the market value of a specific technology and the annual average revenue that all generators together earn for one unit of generation (i.e., the annual average electricity price that a user pays). Under our soft-coupling approach, the markups from the PSM act as price signals that change the composition of the energy mix in the next iteration of the IAM. In this simple example with a lot of PV and no storage, since the gas generator can generate electricity in times of scarcity (night), it is “more valuable” to the system and thus will receive a positive markup. When this positive price incentive is transferred from the PSM to the IAM, it increases the optimal level of investment in gas generation in the next IAM iteration. At the same time, solar generation receives a negative price incentive, reducing the optimal level of investments in the next iteration. Ultimately, the higher market value of gas turbines is due to (1) their higher cost compared to solar (when gas is at < 50 % market share) and (2) their ability to set prices in hours of low solar output and inflexible electricity demand. As we later show through the mathematical theory of model convergence, other information besides markups also needs to be transferred, such as capacity factors (the annual average utilization rates of the generators).

Figure 1 Schematic illustration of the coupling approach for a simple power system in an IAM with coarse temporal resolution, consisting of only gas and solar generators (no storage). (a, c) Before coupling; (b, d) after coupling. (a, b) Endogenous prices (electricity price, market values of solar and gas generators); (c, d) endogenous quantities (generation mix). The markups (as part of a larger set of interfaced variables) are the differences between market values and electricity prices and are given by the PSM of high temporal resolution as price signals to the IAM. Usually, it is called a “markup” when the market value is higher than the annual average electricity price and a “markdown” if it is the other way around. For simplicity, in the rest of the text we only refer to “markup” and “markdown” collectively as “markup”, regardless of whether the market value is higher or lower than the average electricity price.

There are several advantages to this new coupling approach centered on linking prices. First, instead of simply prescribing quantities such as yearly generation and capacities, the approach allows endogenous investment decisions to be made by both models as they converge towards a joint solution. This gives maximal freedom to the coupled models while minimizing unnecessary distortions from one model to the other when some necessary quantities are being prescribed. Second, our coupling scheme provides an elegant treatment of both supply- and demand-side technologies using the concepts of market values on the one hand and capture prices on the other. Third, from a theoretical point of view, transferring the market values of all the generation types in a system alongside mappings of other relevant system parameters can lead to a convergence of the solutions of the two models under idealized coupling circumstances. It can be rigorously shown that our method contains an exhaustive list of interfacing parameters and variables for full model convergence of both quantities and prices. To the best of the authors' knowledge, the last point has not been explored or shown in any previous work.

In certain IAMs, VRE integration cost parameterization has been implemented to mimic the economic consequences of variability of VREs, especially when the models have a lower temporal resolution. Such VRE integration costs are contained in the uncoupled default REMIND power sector modeling. However, the exact parameterization always depends on a particular set of technological costs and parameters which might be subject to changes (Pietzcker et al., 2017), and the parameterization often needs to be carried out anew under new assumptions and scenarios. In contrast, the model-coupling approach is more general, and no such bespoke parameterization is needed.

Inspired by the theoretical framework based on the Karush–Kuhn–Tucker (KKT) conditions for power sector optimization problems (Brown and Reichenberg, 2021), we develop the theoretical basis for the coupling method in this section, which we use for validating convergence in numerical coupling in later sections. In Sect. 3.1, we analytically formulate the fundamental economic theory of the coupling approach. We first introduce the power sector formulations in the two uncoupled models (Sect. 3.1). Then we carry out a derivation of the convergence conditions and criteria, where we map the Lagrangians of the two power sector problems at different time resolutions and derive the equilibrium condition for the coupled models (Sect. 3.2). In Sect. 3.3, we introduce the iterative coupling interface which contains all the previously derived convergence conditions. For REMIND information being passed on to DIETER (Sect. 3.3.1) and DIETER information being passed on to REMIND (Sect. 3.3.2), we list and define the variables and parameters being exchanged at the interface as well as additional constraints and implementations which serve to improve the coupling.

Complete lists of the mathematical symbols and abbreviations can be found in the Appendices.

In the following sections, we first formulate the two uncoupled models and then move on to discussing coupled models. The theoretical tools we develop here are the foundation for the numerical implementation of coupling and serve to validate and assess the model convergence in the Results section.

3.1 Descriptions of uncoupled models

REMIND and DIETER are both optimization models. REMIND maximizes interannual global welfare from 2005 to 2150, whereas DIETER minimizes the power sector system cost for a single year and a single region. For a given REMIND Nash iteration (see Sect. 2.1), the single-region economy is in long-term equilibrium after the optimization problem is solved. Since, given a fixed national income, lower energy system costs mean higher consumption, which leads to increased welfare (see Appendix C for details), maximizing welfare can be assumed to correspond to minimizing energy system costs, a part of which is power sector costs. Therefore, to reduce the complexity of our analysis, we formulate an uncoupled REMIND model based solely on the power sector cost minimization and not the total welfare maximization. For the standalone REMIND, the multi-year power system cost for a single region equals the sum of all variable and fixed costs of generation,

$$\begin{array}{}\text{(1)}& Z={\sum }_{y,s}(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}),\end{array}$$

where c represents the fixed cost for capacity, o represents the variable cost of running power generation, P denotes endogenous capacity, and G denotes endogenous generation (defined as including curtailment in REMIND). P and G are the decision variables of the problem. The sum in the objective function is over time index y and power-generating technology type s. The REMIND time index y stands for 1 representative year, which represents 5 or 10 years centered around it. So, even though the time step is 5 to 10 years, the time resolution is 1 year. For example, “y=2020” represents the years 2018–2022. Capital letters (both Latin and Greek) denote independent decision variables of the optimization problem. We classify an endogenous decision variable as independent if it is not uniquely determined by one or more other decision variables and has no binding constraints applied to itself that are not already accounted for by the constraints on the decision variable(s) it depends on. Note that, for simplicity, we treat all costs in REMIND in this formulation as if they are exogenous. In reality, REMIND has endogenous fixed costs due to technological learning as well as an endogenous interest rate. Some types of variable costs such as fuel costs are also endogenous and are determined based on primary energy balance equations for oil, gas and biomass. CO₂ prices can also be endogenous under emission constraints.

Under the simplifying assumptions made for the derivation in this paper, the only independent decision variables are capacities, generations and curtailments. Small letters denote either exogenously given parameters or endogenous shadow prices.

For standalone DIETER, which has a year-long time horizon, the power system cost is

$$\begin{array}{}\text{(2)}& \stackrel{\mathrm{‾}}{Z}={\sum }_s{\stackrel{\mathrm{‾}}{c}}_s{\stackrel{\mathrm{‾}}{P}}_s+{\sum }_h\left[{\stackrel{\mathrm{‾}}{o}}_s\left({\stackrel{\mathrm{‾}}{G}}_{h,s}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}\right)\right],\end{array}$$

where ${\stackrel{\mathrm{‾}}{G}}_{h,s}$ is the endogenous hourly power generation (excluding curtailment – note that this is different from the generation variable definition in REMIND), h is the hourly index in a year from 1 to 8760, and s is the index for the power-generating technology in DIETER. $\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}$ is hourly curtailment, only applicable in the case of variable renewables vre (vre⊂s). Technology type s can be subdivided into two subsets: vre and dis (“dispatchables”). For simplicity, we abbreviate the index subscript from s|s=vre to vre and from s|s=dis to dis. Here, in order to differentiate from REMIND notations, we use overline $\stackrel{\mathrm{‾}}{.}$ to denote DIETER parameters and variables. Note that, for simplicity, in the derivation we treat the technology types in both models as identical, although in fact the technologies in the two models are not one-to-one mapped (Fig. B2). During the coupling all interface parameters and optimal decision variables need to be upscaled or downscaled when transferred from one model to the other.

The cost minimization of the total power sector cost Z and $\stackrel{\mathrm{‾}}{Z}$ under constraints yields the optimal values of the decision variables, denoted as ($P_{y,s}^{\ast }$, $G_{y,s}^{\ast }$) and (${\stackrel{\mathrm{‾}}{P}}_s^{\ast }$, ${\stackrel{\mathrm{‾}}{G}}_{h,s}^{\ast }$, ${\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,s}^{\ast })$.

Without coupling, and under a baseline scenario, there are several constraints for each model. In the following equations we denote the shadow price (i.e., the Lagrangian multiplier) of a constraint by the symbol following ⟂. We use small Greek letters to denote endogenous shadow prices and small Latin and Greek letters to denote exogenous parameters. The major constraints are as follows (“c” stands for “constraint”).

• c1.

  Constraint on generation for meeting demand, a.k.a. “supply–demand balance equation” or “balance equation” for short:

  $$\begin{array}{lll}{\displaystyle }& {\displaystyle }\text{REMIND (annual)}:d_y={\sum }_s{G}_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)& {\displaystyle }⟂{\mathit{\lambda }}_y,\\ {\displaystyle }& {\displaystyle }\text{DIETER (hourly)}:{\stackrel{\mathrm{‾}}{d}}_h={\sum }_s{\stackrel{\mathrm{‾}}{G}}_{h,s}& {\displaystyle }⟂{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h,\end{array}$$

  where d_y denotes the annual REMIND power demand and ${\stackrel{\mathrm{‾}}{d}}_h$ denotes the DIETER hourly demand. The shadow prices (Lagrangian multipliers) λ_y and ${\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h$ represent the annual and hourly electricity prices in REMIND and DIETER, respectively, and are equal to the marginal cost of one additional unit of electricity generation. α_y,s is the annual VRE curtailment ratio in REMIND. Note that, technically speaking, REMIND electricity demand d_y is determined endogenously, partially via competition with other energy carriers at the final energy consumption level, such as the competition between electricity and gaseous carriers such as natural gas or hydrogen in household heating. However, because here we have reduced REMIND to only intra-power sector dynamics for the purpose of mathematical analysis, we treat demand as exogenous.

• c2.

  Constraint on maximum capacity by the available annual potential ψ_s in a region:

  $$\begin{array}{lll}{\displaystyle }& {\displaystyle }\text{REMIND}:P_{y,s}\le {\mathit{\psi }}_s& {\displaystyle }⟂{\mathit{\omega }}_{y,s},\\ {\displaystyle }& {\displaystyle }\text{DIETER}:{\stackrel{\mathrm{‾}}{P}}_s\le {\stackrel{\mathrm{‾}}{\mathit{\psi }}}_s& {\displaystyle }⟂{\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s.\end{array}$$

  Note that the resource constraint in REMIND is only relevant for wind, solar and hydro and is assumed to be constant over the model horizon. Biomass availability is not modeled via a regional potential constraint. Instead, the availability of biomass is priced in through the soft-coupling to the land use model MAgPIE via a supply curve.

• c3.

  Constraint on generation being non-negative:

  $$\begin{array}{lll}{\displaystyle }& {\displaystyle }\text{REMIND}:-G_{y,s}\le \mathrm{0}& {\displaystyle }⟂{\mathit{\xi }}_{y,s},\\ {\displaystyle }& {\displaystyle }\text{DIETER}:-{\stackrel{\mathrm{‾}}{G}}_{h,s}\le \mathrm{0}& {\displaystyle }⟂{\stackrel{\mathrm{‾}}{\mathit{\xi }}}_{h,s}.\end{array}$$

  Note that there are several other similar constraints on other positive variables such as capacities and curtailment. In practice, during the derivation they behave similarly to this positive generation constraint. Therefore, for simplicity, we do not include them in the derivation.

• c4.

  Constraint on maximum generation from capacity:

  $$\begin{array}{rlr}\text{REMIND}& :G_{y,s}={\mathit{\varphi }}_{y,s}{P}_{y,s}\cdot \mathrm{8760}& ⟂{\mathit{\mu }}_{y,s},\\ \text{DIETER}& :\text{(variable renewables)}{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{vre}}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}\\ & ={\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,\mathrm{vre}}{\stackrel{\mathrm{‾}}{P}}_{\mathrm{vre}}& ⟂{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{vre}}\\ & \text{(dispatchables)}{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{dis}}\le {\stackrel{\mathrm{‾}}{P}}_{\mathrm{dis}}& ⟂{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{dis}},\end{array}$$

  where ϕ_y,s is the exogenous annual average capacity factor of the power plant s in REMIND in year y and ${\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,\mathrm{vre}}$ is the exogenously given hourly theoretical capacity factor (i.e., before curtailment) of VRE in DIETER. Note that, strictly speaking, curtailments in the uncoupled REMIND and DIETER are endogenous decision variables but are not independent variables. However, here we use capital letters to denote hourly curtailment in DIETER as an independent decision variable to account for curtailment costs and other curtailment constraints that can arise from a more general formulation of the model.

• c5.

  “Historical” constraints on capacities in REMIND. This makes REMIND a so-called “brown-field model”, i.e., a model accounting for the standing capacities in the real world. Past capacities (y< 2020) are “hard-fixed”; i.e., the variable capacities are fixed to certain numeric values. Current capacities (y=2020) are “soft-fixed”; i.e., the variable capacities are fixed to a corridor around certain standing numeric values: the lower bounds guarantee the already planned capacities, and the upper bounds reflect the finite physical capabilities of scaling up, defined by 5 % above the 2020 real-world data. For simplicity, we use only one constraint for both past and current capacities,

  $${\displaystyle }{\displaystyle }{P}_{y,s}\ge p_{y,s}{\displaystyle }⟂{\mathit{\sigma }}_{y,s}\text{for}y\le \mathrm{2020},$$

  where p_y,s represents the standing capacities of technology s at time y in REMIND in the past and present years.

• c6.

  Near-term upscaling constraint on VRE capacity expansion, represented by an upper bound on near-term capacity addition in model period $\left(y-\mathrm{\Delta }y,y\right)$, $\mathrm{\Delta }{P}_{y,s}:=P_{y,s}-P_{y-\mathrm{\Delta }y,s}$, where Δy is the REMIND model time step:

  $${\displaystyle }{\displaystyle }\mathrm{\Delta }{P}_{y,s}\le q_{y,s}{\displaystyle }⟂{\mathit{\gamma }}_{y,s}\text{for}y=\mathrm{2025},$$

  where q_y,s is equal to twice the added capacity during the 2010–2020 period (only applied to Germany in the default REMIND).

Note that constraints (c5) and (c6) introduce interannual intertemporality into the power sector of REMIND. This additional interannual intertemporality determines that the model equilibrium can only be strictly satisfied across the sum of all the model periods and not for a single period. Another source of intertemporality in REMIND is the adjustment cost, which we ignore in the main text of this study since it introduces nonlinearity into the power sector and also plays a relatively small role in the overall dynamics.

Note that, regarding the simplification of REMIND above, to the best of the authors' knowledge, there is no theoretical or empirical concept that addresses the validity of drawing equivalence between welfare maximization and energy system cost minimization in IAMs. Naively, given that the gross domestic product (GDP) is unchanged, decreasing energy system costs raises consumption and therefore welfare. However, this is only valid under the assumption that energy is a substitute for (and not a complement to) capital and labor; i.e., one usually cannot raise economic output (GDP) simply by spending more on higher energy expenditure (while satisfying the same level of energy demand). Nevertheless, this is likely a necessary condition and not a sufficient one for proving the equivalence. More theoretical research will be needed to draw a precise and rigorous equivalence. However, in practice, we see that during our numerical calculation the model is well behaved according to this reduced theory, which means that the parameters in the models are in a regime where such an assumption is valid, at least in the case of IAM REMIND.

3.2 Economic theory of model convergence

In the last section we discussed the standalone uncoupled power sector formulations in REMIND and DIETER. In this section we discuss the coupled models and their convergence. Under simplified assumptions, we first derive the mapping between the models that is necessary for a convergence (Sect. 3.2.1 and 3.2.2), and then we derive theoretical relations that are later used to validate the numerical results of the coupled run (Sect. 3.2.3).

3.2.1 Derivation of convergence conditions

Our aim is to develop a method under which comprehensive convergence can be reached for soft-coupled multiscale models. We achieve this by deriving a mapping of the two problems such that their decision variables have identical optimal solutions and the endogenous shadow prices are also equal across the models. The convergence conditions of the coupled REMIND–DIETER model for the power sector are the result of such a mapping. Below, we first define what is meant by a “comprehensive model convergence” and then sketch the workflow of the derivation of a coupling framework which would result in a comprehensive model convergence of both decision variables and shadow prices. The detailed derivation is in Appendix D.

Here, we derive the conditions under which the endogenous decision variables are identical at each model's optimum, i.e., $P_{y,s}^{\ast }={\stackrel{\mathrm{‾}}{P}}_{y,s}^{\ast }$ and $G_{y,s}^{\ast }\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}^{\ast }\right)={\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}^{\ast }$ – or, equivalently, pre-curtailment generation $G_{y,s}^{\ast }$ and ${\sum }_h\left({\stackrel{\mathrm{‾}}{G}}_{y,h,s}^{\ast }+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{y,h,s}^{\ast }\right)$. A convergence of the solutions of these two sets of annual decision variables for each technology s and for each year y, along with the convergence of shadow prices, gives rise to comprehensive model convergence. We show below that this can only be achieved if there is a harmonization at the level of the KKT Lagrangians of the two problems, following the methods first developed by Karush, Kuhn and Tucker (Karush, 1939; Kuhn and Tucker, 1951).

Our coupling approach fundamentally relies on mapping the parameterization of the Lagrangians for both optimization problems. It is trivial to show that, as long as the KKT Lagrangians are identical with respect to the decision variables, the solutions of the problem are identical, e.g., if an optimization problem A has Lagrangian ${\mathcal{L}}_{\mathrm{1}}=a_{\mathrm{1}}\cdot x+b_{\mathrm{1}}\cdot y$ and another problem B has Lagrangian ${\mathcal{L}}_{\mathrm{2}}=a_{\mathrm{2}}\cdot x+b_{\mathrm{2}}\cdot y$, where x and y are decision variables of the optimization problems. Then, if we let $a_{\mathrm{1}}=a_{\mathrm{2}},b_{\mathrm{1}}=b_{\mathrm{2}}$, the two problems are identical, and they must have identical optimal solutions for the decision variables x^* and y^*. This is the basic logic behind the Lagrangian-based method. The challenge in the case of REMIND and DIETER is to show that when a decision variable representing the same physical quantity, for example, the annual power generation from a technology, is defined with low resolution in one problem and high resolution in another, there is nevertheless a viable mapping between the two Lagrangians. In this case, the parameterization of the Lagrangian is not only limited to exogenous parameters of the model, but also includes endogenous shadow prices and endogenous decision variables from the other model. Due to the endogenous nature of the latter two, the parameterization in the current-iteration model A must come from the solved results from the last iteration from model B and vice versa. Figure 2 illustrates the workflow of the analytical derivation of the convergence conditions.

Figure 2 The schematics of the Lagrangian-based derivation procedure for a simplified version of REMIND–DIETER iterative convergence. After simplifying assumptions, we can construct the Lagrangians of the reduced REMIND model and the full DIETER model for a single year (Eqs. 3–4). Comparing and mapping terms in the Lagrangians (a key step in bold), we discover that iterative exchange of a broad range of information is needed for a fully harmonized parameterization of the Lagrangians. Under the harmonization specified in the seven convergence conditions (color-coded for directions of information flow), the coupled models can give rise to identical optimal solutions of the models' respective (annually aggregated) decision variables and hence a full quantity convergence. The necessary shadow price convergence is shown in the detailed derivation of the harmonization conditions (h1–h7) in Appendix D.

The analytical derivation workflow, as shown in Fig. 2, is described in detail as follows. First, we apply simplifying assumptions to reduce the complexity of the uncoupled models (before the key step in blue in Fig. 2). Assumptions have to be made to justify reducing the scope of the REMIND model, such that for the purpose of the analysis it is on equal footing to DIETER. We achieve this by reducing the global REMIND model to a single sector (the power sector), single year and single region. To reduce the REMIND model from a macroeconomic–energy model to a power-sector-only model, we make similar assumptions to before when formulating the uncoupled REMIND power sector (see Sect. 3.1). To reduce the REMIND model further to a single year, we assume that the models only contain constraints in the power sector that are not intertemporal; i.e., we ignore the brown-field and near-term constraints for now. Since for each iteration of the REMIND model in Nash mode interregional trading happens between the iterations, the single-iteration optimization model is already for a single region and therefore does not require simplification. After these simplifying steps, in this part of the derivation, we can treat REMIND's power sector as “separate” from the rest of the model and treat the dynamics of a single year in REMIND as independent of the dynamics of other years. Later, the numerical results of the convergence can confirm to a large degree the validity of these assumptions, especially in the green-field temporal ranges, i.e., where the intertemporal brown-field constraints have little influence on the dynamics. Note that, with the inclusion of these intertemporal constraints in the derivation, the mapping becomes more complicated, especially for the near-term range, i.e., before 2035. So, in practice, this derivation of the coupling interface is only an approximation of what is needed for a full convergence of DIETER and REMIND, since it deliberately ignores such constraints. See also Sect. 6.1.

After the necessary simplification assumptions, we construct the Lagrangians for the simplified model REMIND and for DIETER (after the blue block in Fig. 2) (Gan et al., 2013). For a single-year reduced REMIND power sector model, the Lagrangian is

$$\begin{array}{}\text{(3)}& \begin{array}{rl}{\mathcal{L}}_{\text{y}}& =\underset{\text{REMIND objective function}}{\underbrace{\sum _s\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right)}}\\ & +\underset{\text{annual electricity balance equation constraint}}{\underbrace{{\mathit{\lambda }}_y\left[d_y-\sum _s{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})\right]}}\\ & +\underset{\text{maximum generation from capacity constraint}}{\underbrace{\sum _s{\mathit{\mu }}_{y,s}\left(G_{y,s}-\mathrm{8760}\cdot {\mathit{\varphi }}_{y,s}{P}_{y,s}\right)}}.\end{array}\end{array}$$

We would like to map it to the single-year DIETER Lagrangian $\stackrel{\mathrm{‾}}{\mathcal{L}}$:

$$\begin{array}{}\text{(4)}& \begin{array}{rl}\stackrel{\mathrm{‾}}{\mathcal{L}}& =\underset{\text{DIETER objective function}}{\underbrace{\sum _s\left[{\stackrel{\mathrm{‾}}{c}}_s{\stackrel{\mathrm{‾}}{P}}_s+{\stackrel{\mathrm{‾}}{o}}_s\sum _h\left({\stackrel{\mathrm{‾}}{G}}_{h,s}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}\right)\right]}}\\ & +\underset{\text{hourly electricity balance equation constraint}}{\underbrace{\sum _h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h\left({\stackrel{\mathrm{‾}}{d}}_h-\sum _s{\stackrel{\mathrm{‾}}{G}}_{h,s}\right)}}\\ & +\underset{\text{maximum dispatchable generation from capacity constraint}}{\underbrace{\sum _{h,\mathrm{dis}}{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{dis}}\left({\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{dis}}-{\stackrel{\mathrm{‾}}{P}}_{\mathrm{dis}}\right)}}\\ & +\underset{\text{maximum renewable generation from capacity and weather constraints}}{\underbrace{\sum _{h,\mathrm{vre}}{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{vre}}\left({\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{vre}}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}-{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,\mathrm{vre}}{\stackrel{\mathrm{‾}}{P}}_{\mathrm{vre}}\right)}}.\end{array}\end{array}$$

The algebraic derivation of mapping the two Lagrangians term by term is presented in Appendix D. From this algebraic mapping, we can derive seven harmonization conditions (h1–h7) required for a full convergence. Conditions (h1)–(h7) are the subsequent basis for most of the information exchanged at the coupling interface. Among them, conditions (h3) and (h5)–(h7) (purple blocks in Fig. 2) indicate conditions which contain endogenous information that must come from the previous iteration of DIETER that is passed on to REMIND, such as markup and capacity factors. Conditions (h1)–(h2) and (h4) (yellow blocks) indicate conditions which contain information that comes from the previous iteration of REMIND and is passed on to DIETER. For schematics of the coupled iterations, see Appendix E.

This Lagrangian-mapping-based derivation can theoretically show that our approach (in its simplest form) necessarily leads to model convergence and has the advantage of being mathematically straightforward and rigorous. The necessary information from the power sector dynamics is all contained in the list of conditions derived from such a mapping. If the coupling contains less information, a convergence is not possible; at the same time, for a model convergence, one does not need to pass on any additional information beyond what is contained in this list of conditions. The list of information derived here is therefore complete and exhaustive for a coupled convergence.

3.2.2 List of convergence conditions

The convergence conditions (h1–h7), which are derived in detail in Appendix D following the procedure in Sect. 3.2.1, are summarized here.

• h1.

  Annual fixed costs are harmonized: $c_{y,s}={\stackrel{\mathrm{‾}}{c}}_{y,s}$.

• h2.

  Annual variable costs are harmonized: $o_{y,s}={\stackrel{\mathrm{‾}}{o}}_{y,s}$.

• h3.

  Annual average market values for each generation type s are harmonized via markups from DIETER. We let ${\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}\left(i-\mathrm{1}\right)$ denote the markup for technology s in year y in the last-iteration DIETER, i.e., the difference between the market value and the annual average price of electricity.

  $$\begin{array}{}\text{(5)}& {\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}=\underset{\stackrel{\mathrm{‾}}{{\text{market value}}_s}}{\underbrace{{\displaystyle \frac{\sum _s{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}{\sum _h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}}}}-\underset{{\stackrel{\mathrm{‾}}{\text{annual average electricity price}}}_s}{\underbrace{{\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{d}}_{y,h}}{\sum _h{\stackrel{\mathrm{‾}}{d}}_{y,h}}}}}\end{array}$$

  This is the heart of our coupling approach, using markups as the price signals. Intuitively, the markups represent the market value differences between REMIND and DIETER. The harmonization of market values is implemented by iteratively adjusting the market value for each generator type in REMIND to be the same as that in DIETER. As long as the market values (or per-unit-generation revenues) and costs are harmonized, the economic structures of the power market are identical and the models can converge.

  Using markup Eq. (5), we modify the original objective function Z in the coupled version of REMIND by subtracting the product of markups and generations summed over all technologies and all years:

  $$\begin{array}{}\text{(6)}& Z^{\prime }=Z-{\sum }_{y,s}{\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}\left(i-\mathrm{1}\right)G_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right),\end{array}$$

  where Z^′ is the modified REMIND objective function in the coupled version, and i is the iteration index of the iterative soft-coupling.

• h4.

  Annual power demands are harmonized: ${\sum }_h{\stackrel{\mathrm{‾}}{d}}_{y,h}=d_y$.

• h5.

  Annual average prices of electricity are harmonized:

  $$\begin{array}{}\text{(7)}& {\mathit{\lambda }}_y={\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}\left(i-\mathrm{1}\right){\stackrel{\mathrm{‾}}{d}}_{y,h}\left(i-\mathrm{1}\right)}{{\sum }_h{\stackrel{\mathrm{‾}}{d}}_{y,h}\left(i-\mathrm{1}\right)}},\end{array}$$

  where (i−1) indicates that the endogenous results are from the last iteration. This is shown in Appendix D to be a direct consequence of (h3) and (h4).

• h6.

  Annual average capacity factors for each generation type s are harmonized:

  $$\begin{array}{}\text{(8)}& {\mathit{\varphi }}_{y,s}={\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,s}\left(i-\mathrm{1}\right)/\mathrm{8760},\end{array}$$

  where ${\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,s}\left(i-\mathrm{1}\right)=\frac{{\stackrel{\mathrm{‾}}{G}}_{y,h,s}\left(i-\mathrm{1}\right)}{{\stackrel{\mathrm{‾}}{P}}_{y,s}\left(i-\mathrm{1}\right)}$ is the hourly capacity factor in DIETER, determined by endogenous hourly generation and annual capacities in the last iteration.

• h7.

  Annual curtailments are harmonized:

  $$\begin{array}{}\text{(9)}& G_{y,\mathrm{vre}}{\mathit{\alpha }}_{y,\mathrm{vre}}={\sum }_h{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{y,h,\mathrm{vre}}\left(i-\mathrm{1}\right).\end{array}$$

In mapping the Lagrangians (Eqs. 3–4), except for the objective function, the rest of the parameterization contains endogenous shadow prices and endogenous quantities. Since endogenous values can only be known ex post, this imposes a strict requirement on the coupling that it must be iterative, with the endogenous part of the parameterization coming from previous iteration optimization results – usually from the other model. The mapping of the endogenous information requires careful argument in each case (i.e., the derivation of h3–h7). In the case of the balance equation constraint Lagrangian term (corresponding to c1), the shadow prices of the constraint in the current-iteration REMIND model are exogenously corrected by a set of technology-specific markups (see the introduction in Sect. 3.1), such that the new “corrected” market value in REMIND is manipulated to match the market value of the previous iteration of DIETER. This is the heart of our coupling approach using markups as the price signals. In the case of the constraint on maximum generation from capacity (corresponding to c4), the endogenous shadow prices in the current iteration REMIND can be shown to be automatically mapped to those in the previous iteration of DIETER given that the annual average capacity factors in the constraints are harmonized (h6–h7).

In actual implementation, most of the above mappings are modified for numerical stability (Sect. 3.3.2, Appendix H).

3.2.3 Theoretical tools for validating convergence

Here we first state the convergence criteria, which are mathematical relations that are being satisfied under model convergence. Then we also discuss equilibrium conditions of the coupled models that alongside the convergence criteria can be used to check numeric results to validate and assess the convergence outcome.

Under a theoretical full convergence of the coupled model,

• v1.

  annual average electricity prices,

• v2.

  capacities and

• v3.

  (post- or pre-curtailment) generations

should all be identical at the end of the coupling in both models. These are the most important criteria by which we validate full model convergence. Technically, electricity price convergence (v1) (i.e., convergence condition h5) can be derived from (h3) to (h4). Nevertheless, we check this ex post, together with quantity convergence (v2–v3). In actual coupled model runs, following only the convergence conditions (h1–h7), the convergence criteria (v1–v3) might not be exactly fulfilled. Therefore, in practice, in order to validate the degree of numerical convergence, the alignment between REMIND and DIETER generation shares is set to be within a few percentage points before coupled runs terminate.

Besides using convergence criteria (v1–v3), we also use a type of equilibrium condition – the so-called “zero-profit rules” (ZPRs) – to validate the numerical model convergence. ZPRs are mathematical relations which state that, under market equilibrium, prices are equal to the costs for electricity. This is not always the case, especially in situations where there are extra constraints on the model that distort this equality. ZPRs contain model parameters and decision variables at market equilibrium, and they can be derived from the KKT conditions of the model (Appendix F). ZPRs are therefore reliable tools in ascertaining the sources of market values or the price of electricity of the power sector because, according to the ZPRs, one can always decompose the prices into the cost components, i.e., so-called levelized costs of electricity (LCOEs). The decomposition of prices into cost components is important because the prices of electricity in the power market are overdetermined by the energy mix, so it is possible that two different power mixes correspond to the same electricity price. In numerical results, a slight mismatch of the energy mix at the end of the coupling is unavoidable, so, alongside comparing the prices, it is often helpful to compare the makeup of the LCOEs across the models, such that they also appear harmonized at the end of the iterative convergence. Overall, ZPRs are a helpful tool for visualizing and understanding the power market dynamics, both from the point of view of each generator type and from the point of view of the entire electricity system. It is worth noting that the ZPRs, which are mathematical conditions derived from an idealized modeling of the power sector as fully competitive, are only an approximation of the real-world markets, where firm profits exist. ZPRs in their technical definition simply mean that, at model equilibrium, cost equals revenue. Given that the profits are defined as the difference between revenue and cost, the profits are zero in this situation. The name “zero-profit rule” therefore should not be overinterpreted beyond its technical contents, and one should be aware of its theoretical origin and the assumptions under which it is valid.

The ZPRs of the coupled model can be derived based on (1) the uncoupled models, (2) the modification made to the model due to the coupling interface (h1–h7) and (3) any additional modifications made to the model during our numerical implementation. In the last category, for a complete numerical implementation of the coupling, we add one additional capacity constraint, i.e., (c7) and (c8) for each model. The first capacity constraint (c7) is created in REMIND to circumvent the issue of extremely high markups from peaker gas plants in the scarcity hour of the year in the DIETER model, which otherwise causes instability during the iterative coupling. The second constraint (c8) is a simple brown-field constraint implemented in DIETER to address the fact that DIETER is a green-field model that is otherwise ignorant of standing capacities in the real world. For simplicity, (c7) and (c8) are not included in the convergence condition derivations in Sect. 3.2.1. The derivations of the ZPRs outlined by the above three steps have been carried out in Appendix F (uncoupled models), Appendix G (coupled REMIND only including the coupling interface, coupled DIETER including constraint c8) and Appendix H (coupled REMIND, including constraint c7).

In summary, the ZPRs for both coupled models are as follows.

• a.

  Coupled REMIND

  • i.

    Technology-specific ZPR:

    $$\begin{array}{}\text{(10)}& \begin{array}{rl}& \underset{{\text{pre-curtailment LCOE}}_s}{\underbrace{{\displaystyle \frac{\sum _y\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right)}{\sum _y{G}_{y,s}}}}}\\ & +\underset{{\text{curtailment LCOE}}_s}{\underbrace{{\displaystyle \frac{\sum _y\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right){\mathit{\alpha }}_{y,s}}{\sum _y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}\\ & =-\underset{{\text{capacity shadow price}}_s^{\prime }}{\underbrace{{\displaystyle \frac{\sum _y\left({\mathit{\omega }}_{y,s}-{\mathit{\sigma }}_{y,s}+{\mathit{\gamma }}_{y,s}+{\mathit{\nu }}_{y,s}\right)P_{y,s}}{\sum _y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}\\ & +\underset{{\text{market value}}_s^{\prime }}{\underbrace{{\displaystyle \frac{\sum _y({\mathit{\lambda }}_y+{\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}^{\prime })G_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}{\sum _y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}.\end{array}\end{array}$$
  • ii.

    System ZPR:

    $$\begin{array}{}\text{(11)}& \begin{array}{rl}& \underset{{\text{pre-curtailment LCOE}}_{\mathrm{system}}}{\underbrace{{\displaystyle \frac{\sum _{y,s}\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right)}{\sum _{y,s}{G}_{y,s}}}}}\\ & +\underset{{\text{curtailment cost}}_{\mathrm{system}}}{\underbrace{{\displaystyle \frac{\sum _{y,s}\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right){\mathit{\alpha }}_{y,s}}{\sum _{y,s}{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}\\ & =-\underset{{\text{capacity shadow price}}_{\mathrm{system}}^{\prime }}{\underbrace{{\displaystyle \frac{\sum _{y,s}\left({\mathit{\omega }}_{y,s}-{\mathit{\sigma }}_{y,s}+{\mathit{\gamma }}_{y,s}+{\mathit{\nu }}_{y,s}\right)P_{y,s}}{\sum _{y,s}{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}\\ & +\underset{{\text{electricity price}}_{\mathrm{system}}^{\prime }}{\underbrace{{\displaystyle \frac{\sum _{y,s}({\mathit{\lambda }}_y+{\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}^{\prime })G_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}{\sum _{y,s}{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}.\end{array}\end{array}$$
• b.

  Coupled DIETER

  • i.

    Technology-specific ZPR:

    $$\begin{array}{}\text{(12)}& \begin{array}{rl}& \underset{\stackrel{\mathrm{‾}}{{\text{LCOE}}_s}}{\underbrace{{\displaystyle \frac{{\stackrel{\mathrm{‾}}{c}}_s{\stackrel{\mathrm{‾}}{P}}_s+{\stackrel{\mathrm{‾}}{o}}_s\sum _h\left({\stackrel{\mathrm{‾}}{G}}_{h,s}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}\right)}{\sum _h{\stackrel{\mathrm{‾}}{G}}_{h,s}}}}}\\ & =-\underset{\stackrel{\mathrm{‾}}{{\text{capacity shadow price}}_s^{\prime }}}{\underbrace{{\displaystyle \frac{({\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s+{\stackrel{\mathrm{‾}}{\mathit{\zeta }}}_s){\stackrel{\mathrm{‾}}{P}}_s}{\sum _h{\stackrel{\mathrm{‾}}{G}}_{h,s}}}}}+\underset{\stackrel{\mathrm{‾}}{{\text{market value}}_s}}{\underbrace{{\displaystyle \frac{\sum _h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h{\stackrel{\mathrm{‾}}{G}}_{h,s}}{\sum _h{\stackrel{\mathrm{‾}}{G}}_{h,s}}}}}.\end{array}\end{array}$$
  • ii.

    System ZPR:

    $$\begin{array}{}\text{(13)}& \begin{array}{rl}& \underset{\stackrel{\mathrm{‾}}{{\text{LCOE}}_{\mathrm{system}}}}{\underbrace{{\displaystyle \frac{\sum _s[{\stackrel{\mathrm{‾}}{c}}_s{\stackrel{\mathrm{‾}}{P}}_s+{\stackrel{\mathrm{‾}}{o}}_s{\sum }_h({\stackrel{\mathrm{‾}}{G}}_{h,s}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}\left)\right]}{\sum _{h,s}{\stackrel{\mathrm{‾}}{G}}_{h,s}}}}}\\ & =-\underset{\stackrel{\mathrm{‾}}{{\text{capacity shadow price}}_{\mathrm{system}}^{\prime }}}{\underbrace{{\displaystyle \frac{\sum _s({\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s+{\stackrel{\mathrm{‾}}{\mathit{\zeta }}}_s){\stackrel{\mathrm{‾}}{P}}_s}{\sum _{h,s}{\stackrel{\mathrm{‾}}{G}}_{h,s}}}}}\\ & +\underset{\stackrel{\mathrm{‾}}{{\text{annual average electricity price}}_{\mathrm{system}}}}{\underbrace{{\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h{\stackrel{\mathrm{‾}}{d}}_h}{\sum _h{\stackrel{\mathrm{‾}}{d}}_h}}}}.\end{array}\end{array}$$

The prime sign indicates that the term has been modified from the uncoupled versions due to implementation in the coupling. ν and ζ are capacity shadow prices introduced from the additional constraints (c7–c8) (Appendices G–H). It is worth noting that constraints (c7)–(c8) introduced due to coupling can impact the Lagrangians of the two models which we used to derive convergence conditions and criteria. However, in actual coupled runs, evidently there is only a moderate distortion due to these extra constraints. Condition (c8) even helps with convergence because it also puts most of the brown-field and near-term constraints which REMIND sees into DIETER (see Sect. 6.1).

Due to the fact that several sources of shadow prices cannot be incorporated during the derivation for convergence (Sect. 3.2.1), in numerical experiments of the coupled run it is appropriate to compare the following two types of prices across the two models for price convergence:

1. electricity price convergence, not including any capacity shadow prices;

2. sum of electricity prices and all respective capacity shadow prices converging.

Under the simplified analysis of convergence (discounting brown-field constraints, scarcity prices, etc.), price convergence in (1) is predicted by theory (see also convergence condition h5). However, this is only under the most idealized situation. Convergence in (2) on the other hand includes all the prices, which should match if LCOEs match across the system. We use the first type to check price convergence over iteration and use the second type only in the context of checking the system ZPRs across the models because of the theoretical relations between full prices and LCOEs.

3.3 Implementation via interface: exchange of variables

In this section we list parameters and endogenous variables that are exchanged between REMIND and DIETER. This already satisfies most convergence conditions, while the remaining condition (h5) is checked in Sect. 4 as part of the convergence criteria (v1–v3). An overview of the model coupling and the flow of information under convergence conditions is shown in Fig. 3.

Figure 3 The schematics of the REMIND–DIETER iterative soft-coupling. The power sector module of IAM REMIND, which is between the layer of primary-to-secondary energy transformation, is hard-coupled with other modules inside REMIND such as macro-economy, industry and transport. In PSM DIETER, the power market with generators of various types is modeled with hourly resolution, with options for storage and flexible demand. The information exchanged between the models (block arrows) is determined via the convergence conditions (h1–h7) derived before (Sect. 3.2.1). In order to improve performance and facilitate convergence, additional constraints (c7 and c8) are included in the coupling interface. The coupling interface for REMIND to DIETER is programmed as part of modified DIETER code and vice versa. Both interfaces are written in GAMS. For a single region, the scheduling of coupled iterations is illustrated in Fig. E1 in Appendix E. Sixteen DIETER optimization problems are solved for each representative year of REMIND in parallel, scheduled after each internal REMIND Nash iteration (see Sect. 2.1 for a description of the iterative Nash algorithm).

During the coupling, the following exchanges of parameters and variables take place iteratively in both directions via the interface.

3.3.1 REMIND to DIETER

The following information flow is from REMIND to DIETER.

1. Technology fixed costs (convergence condition h1)

   • a.

     Annualized capital investment cost: this is calculated from the endogenously determined overnight investment cost, plant lifetime and the endogenously determined interest rate. The overnight investment cost is determined from floor cost, learning rate and the endogenous global accumulated deployment. Note that investment costs decrease according to the endogenous learning rate. The interest rate is about 5 % on average but is endogenous and time-dependent in REMIND.

   • b.

     Annualized operation and maintenance (“O&M”) fixed costs (OMF): they are a fixed share of the capital costs.

   • c.

     Adjustment cost: this is technology-specific and is proportional to the capital investment cost. See Appendix I for its implementation.

2. Technology variable costs (convergence condition h2)

   • a.

     Primary energy fuel costs: they are endogenously determined as the shadow prices of the primary fuel balance equations in REMIND. Import prices, domestic prices of extraction, the amount of regional reserves and the amount of fuel demand can all influence the fuel cost. The relevant fuel costs include coal, gas, biomass and uranium. The fuel costs can have interannual intertemporal oscillatory components which can cause instability during iteration if coupled directly. We mitigate this by conducting a linear fit to the time series before passing them on to DIETER.

   • b.

     Conversion efficiency of each generation technology

   • c.

     O&M variable costs (OMV)

   • d.

     CO₂ emission cost: an exogenous or endogenous CO₂ price from REMIND multiplied by the carbon content of a type of fossil fuel and divided by the conversion efficiency of a generation technology gives the CO₂ cost of 1 MWh of generation. Note that, in REMIND, biomass is considered to contain zero carbon emission when combusted.

   • e.

     Grid cost: in REMIND, the stylized grid capacity equation is proportional to the amount of pre-curtailment VRE generation. So, the grid cost is effectively a variable cost. Note that, in future work, grid costs can be modeled in more detail either in DIETER or in another PSM. Here, we use the parameterized grid costs which are implemented in the default REMIND as an approximation to the necessary grid cost.

3. Power demand (convergence condition h4). REMIND informs DIETER of the total power demand d_y of a representative year y. In the next iteration of DIETER, the exogenous time series for the hourly demand from a historical year (2019) is scaled up to the demand of the last-iteration REMIND, d_y(i−1), such that the annual total power demand in DIETER is equal to that of REMIND for each coupled year: ${\stackrel{\mathrm{‾}}{d}}_h={\stackrel{\mathrm{‾}}{d}}_{\mathrm{2019},h}\cdot \frac{d_y\left(i-\mathrm{1}\right)}{{\sum }_h{\stackrel{\mathrm{‾}}{d}}_{\mathrm{2019},h}}$.

4. Pre-investment capacities $P_{y-\mathrm{\Delta }y/\mathrm{2},s}/\left(\mathrm{1}-\mathrm{ER}\right)$ as an additional brown-field constraint (see constraint (c8) in Appendix G). ER is the endogenous early retirement rate in REMIND.

5. Total regional renewable resources for wind, solar and hydro (constraint c2), such that DIETER capacities are constrained by the same total available resources as in REMIND

6. Annual average theoretical capacity factors of VREs and hydroelectric ones in REMIND (convergence condition h6). We denote the pre-curtailment utilization rates of VRE capacity as “theoretical capacity factors”, as these can be achieved in theory if there is no curtailment. They are usually determined by meteorological factors such as wind and solar potential as well as the efficiency of the turbines or solar photovoltaic modules. In contrast, the post-curtailment utilization rates of VREs are “real capacity factors”, as these are the real utilization rates after optimal endogenous dispatch. The time series of theoretical utilization rates of VRE generations of 1 historical year in DIETER are scaled up such that the annual average theoretical capacity factors in DIETER equal the exogenous parameters in REMIND:

   $$\begin{array}{ll}{\displaystyle }& {\displaystyle }{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,\mathrm{vre}}\left(y\right)\\ {\displaystyle }& {\displaystyle }=min\left(\mathrm{0.99},{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,\mathrm{vre}}\left(y=\mathrm{2019}\right)\cdot {\displaystyle \frac{{\mathit{\varphi }}_{\mathrm{vre}}}{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,\mathrm{vre}}\left(y=\mathrm{2019}\right)}}\right).\end{array}$$

   In DIETER, to be realistic, the rescaled hourly capacity factor for solar and wind has an upper bound at 99 %. The slight mismatch of the capacity factors due to this additional upper bound is negligible.

3.3.2 DIETER to REMIND

The following information is passed from the last-iteration DIETER to REMIND.

1. Market values are ${\stackrel{\mathrm{‾}}{\mathrm{MV}}}_{y,s}^{\prime }$, and the annual average electricity price is ${\stackrel{\mathrm{‾}}{J}}_y^{\prime }$ (convergence condition h3), where ${\stackrel{\mathrm{‾}}{\mathrm{MV}}}_{y,s}^{\prime }$ is the annual average market value without the surplus scarcity hour price and ${\stackrel{\mathrm{‾}}{J}}_y^{\prime }$ is the annual average electricity price without the surplus scarcity hour price.

2. The peak hourly residual power demand ${\stackrel{\mathrm{‾}}{d}}_{\mathrm{residual}}$ is a fraction of the total annual demand ${\sum }_h{\stackrel{\mathrm{‾}}{d}}_h$ (constraint c7). This produces the peak residual demand in REMIND d_residual,y that is proportional to the last-iteration DIETER peak to total demand ratio $\frac{{\stackrel{\mathrm{‾}}{d}}_{\mathrm{residual}}\left(y,i-\mathrm{1}\right)}{{\sum }_h{\stackrel{\mathrm{‾}}{d}}_h\left(y,i-\mathrm{1}\right)}$ together with the in-iteration total annual demand d_y(i):

   $$d_{\mathrm{residual},y}\left(i\right)={\displaystyle \frac{{\stackrel{\mathrm{‾}}{d}}_{\mathrm{residual}}\left(y,i-\mathrm{1}\right)}{{\sum }_h{\stackrel{\mathrm{‾}}{d}}_h\left(y,i-\mathrm{1}\right)}}\cdot d_y\left(i\right),$$

   where ${\stackrel{\mathrm{‾}}{d}}_{\mathrm{residual}}$ was defined in Appendix H (Eq. H1).

3. Annual capacity factors of dispatchable plants ${\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{\mathrm{dis}}=\frac{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{dis}}}{{\stackrel{\mathrm{‾}}{P}}_{\mathrm{dis}}\cdot \mathrm{8760}}$ (convergence condition h6)

4. Annual solar and wind curtailment ratio: curtailment as a fraction of total annual post-curtailment generation $\frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}}{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{vre}}}$ (convergence condition h7)

For the information flowing from DIETER to REMIND, we use an innovative method of multiplicative “prefactors”, which can stabilize the coupling and increase the speed towards model convergence. The prefactors are automatic linear stabilizers of the current-iteration variables in REMIND. They depend on current-iteration endogenous variables in REMIND and are usually multiplied usually by the last-iteration endogenous DIETER results that are exogenously passed on to REMIND. This allows some degree of endogeneity in these exchanged variables, and their values can be adjusted according to the updated dynamics in the current REMIND iteration, such as interregional trading or price elasticity of demand, under which the exogenous last-iteration DIETER optimality can be used as an approximate starting point but does not necessarily hold exactly.

The prefactors usually depend on the differences between generation shares in the two models: for example, the prefactor for markup is a linear function of the difference between the current-iteration REMIND endogenous generation share and the last-iteration DIETER generation share. We illustrate the mechanism of prefactors using the markup for solar as an example: a lower market value for solar is consistent with a higher solar share according to the well-known self-cannibalization effect of a decreasing VRE market value as the VRE share increases (Hirth, 2018). Therefore, we can introduce an automatic stabilization measure through a negative feedback loop: if the REMIND endogenous share is larger than in the last DIETER iteration, in which case the in-iteration market value should be lower than the last-iteration DIETER market value, the multiplicative prefactor for the market value should be constructed such that it is smaller than 1. This lowers the market value for solar and decreases the in-iteration REMIND markup η_y,s(i), hence preventing over-incentivization of solar generation using the old market value based on the last-iteration energy mix. Overall, this produces a stabilizing effect on the system by making the markup as a price signal responsive to endogenous quantity change. We use prefactors ubiquitously when passing variables from DIETER to REMIND, such that during the iteration REMIND can adjust more smoothly and easily. We discuss the implementation of these prefactors in detail in Appendix H2.

4 Numerical convergence under the proof-of-concept baseline scenario

In this section, we check the convergence behavior for prices and quantities (capacity and generation) in coupled model runs using the convergence validation criteria from the last section. Comparing the numerical results with the theoretical prediction, we can confirm that REMIND–DIETER soft-coupling indeed produces almost full convergence.

Throughout this section, we only use one scenario – a proof-of-concept baseline scenario. Under the proof-of-concept scenario of the coupled run, we disable storage (i.e., batteries and hydrogen) and flexible demand (i.e., electrolyzers) in both models, as this allows us to use the theoretically derived convergence criteria from Sect. 3, which would become overly complex in a model with storage and flexible demand. The coupled run is under a baseline scenario; i.e., there is no additional climate policy implementation. Since this is a configuration created only for comparison with the theoretical prediction, it is not meant to be a policy-relevant configuration. In more policy-relevant coupled runs, we turn on storage and flexible demand (see Sect. 5). For schematics and computational run times of the coupled iterations, see Appendix E.

For the coupled runs, we define a baseline scenario for the single region Germany under SSP2 assumptions corresponding to the “middle-of-the-road” scenario (for a definition of the SSPs, see Koch and Leimbach, 2023). Specifically, this means that REMIND runs for all global regions in parallel but that DIETER only runs for Germany. Only information in the German power sector is exchanged for the two models. We use a low CO₂ price to represent “no additional policy”, which is USD 30 per tCO₂ in 2020 and USD 37 per tCO₂ for years beyond 2020. According to the 2011 Nuclear Energy Act of Germany, remaining nuclear capacities are set for early retirement in REMIND within the time period until 2022. We assume hydroelectric generation in Germany to come from running rivers. In DIETER, we cap the dispatchable generation's annual capacity factors at 80 % for non-nuclear power plants and at 85 % for nuclear power plants, so the dispatch results are in line with real-world power sectors. This constraint only adjusts the capacity factor constraint (c4), which would pose no additional distortion of our mathematical analysis.

Due to the particular implementation of offshore wind in REMIND, DIETER wind offshore capacities are fixed to those of REMIND to avoid too much distortion. Since in our scenarios offshore wind capacity in Germany is relatively small compared to other generators, this fixing represents only a minor distortion of the coupling. Hydroelectric generation in REMIND is assumed to have an average annual capacity factor of around 25 %. This capacity factor is implemented as a bound in DIETER. For simplicity, instead of a time series profile for hydroelectric generation, we allow the hourly capacity factor to be no higher than 90 %, meaning hydro is close to being dispatchable in all our scenarios. In the German context, hydro usually means run-of-the-river hydroelectricity, which has a variable output. Nevertheless, we find the 90 % maximum hourly capacity factor a reasonable assumption to make, since in our runs we do not yet consider pumped hydro as a technology in this study, so a more dispatchable quality of hydro can be assumed. Results presented in this section belong to the same coupled run under the proof-of-concept scenario.

4.1 Electricity price convergence

According to theoretical convergence criteria (under simplifying assumptions, Sect. 3.2.1–3.2.3), at numerical convergence, the electricity price of REMIND should be equal to the price of DIETER. However, REMIND is interannual intertemporal, whereas DIETER is only year-long, so we compare the differences over time as well as the interannual average of the price differences (Fig. 4).

Figure 4 Annual average electricity price convergence behavior of a coupled run for Germany under a “proof-of-concept” baseline scenario. (a) The difference between the annual electricity price time series of REMIND and the annual average electricity price time series in DIETER as a function of coupled iteration. (b) The interannual average of the differences in panel (a) as a share of the REMIND price. Due to the interannual, intertemporal nature of REMIND, in panel (a) the price difference can appear to have oscillatory components, obscuring the visual assessment of convergence. As a result, we show the trend of price convergence over iterations more clearly in panel (b) by taking the temporal average of the price differences. The REMIND price in both plots is a running average of three neighboring time periods to visually smooth out oscillations.

In Fig. 4a, the price difference oscillates from period to period. As the coupling starts, the REMIND price is much higher than DIETER, especially in the earlier years. After around the 10th iteration, the difference in the early years starts to reverse: DIETER's price becomes higher than REMIND. Around 2040–2060, REMIND has a higher average price than DIETER due to the VRE market values being higher than their LCOEs. This is discussed later in Sect. 4.3.2.

In Fig. 4b, we calculate the difference between two time series – the time-averaged power prices in the two models. We observe that the difference between them decreases over the iterations, showing a clear converging trend, and stabilizes at around 3 % of the REMIND price. There are two observations regarding the price convergence of the coupled run. First, the convergence happens rather quickly within 10 iterations. Second, the converged value of the price difference is not exactly 0 but is slightly above 0, a few percent of the full price (a few USD per megawatt hour). Under ideal convergence conditions, according to (v1), the two prices should be equal at full convergence for every coupled year. However, in practice, the average prices do not match perfectly, as there are several sources of distortions from capacity shadow prices. The capacity shadow prices come from many sources in both models: extra constraints such as (c7)–(c8) that are not part of the analysis leading to (v1), constraints that are in REMIND but not in DIETER (c5–c6), and exogenous wind offshore capacity in DIETER. Some of these capacity shadow prices in both models can be more or less consistent with each other (such as the standing capacity constraint in DIETER and brown-field constraints in REMIND), but others are not and can distort two models in different ways, causing some degrees of misalignment in prices. As discussed before, prices can be overdetermined by the energy mix (Sect. 3.2.3). Therefore, some of the capacity shadow prices – even though not aligned between the two models – can nevertheless cancel each other out (especially when averaged over time), potentially causing the price differences to be moderate. To examine exactly how well the prices at the end of the coupling match, we need to check the cost decomposition of the prices. This is discussed later in Sect. 4.3.

Also note that Fig. 4b presents a time-averaged price comparison, and on average the difference between the prices in the two models is small at the end of the coupling. However, when one compares the maximal deviation for any single year at the end of the coupling, it can be as high as USD 10 per megawatt hour, e.g., around 2050 (Fig. 4a). This is much larger than the 3 % averaged deviation in Fig. 4b. However, compared to default REMIND prices (which we cannot show due to limited space), we are fairly confident that the oscillation of the coupled REMIND results from internal dynamics that are also visible in the default uncoupled version. So, a time-averaged treatment is adequate in displaying the total price convergence here.

4.2 Quantity convergence

Besides price convergence, the capacity and generation decision variables must also converge within a certain tolerance at the end of the coupling. This is reflected in the generation mix (Fig. 5) and the capacity mix (Fig. 6) at the end of the coupled run. Due to the existence of several sources of mismatch between the two models already mentioned in the last section, which is already manifested in the mismatch in electricity prices of the two models, a certain degree of mismatch in quantities is also to be expected. Nevertheless, the agreement between the two endogenous sets of decision variables is satisfactory. For this coupled run, the differences of the generation share of any single technology between the two models are smaller than 4.4 % for each year until 2100. Figure 5b highlights some subtle model differences in generation. For example, after 2040, REMIND favors solar and coal, whereas DIETER tends to have more combined-cycle gas turbines (CCGTs) and wind onshore. Due to the low capacity factor of open-cycle gas turbines (OCGTs) and solar compared to the capacity factors of the other generators, the capacity mix differences between models are amplified for these two technologies (Fig. 6). However, overall, the generation mixes and the capacity portfolios at the end of the coupled run are generally similar.

Figure 5 Annual electricity generation convergence at the final iteration of a coupled run for Germany under the proof-of-concept baseline scenario. (a) Side-by-side comparison of the two generation portfolios at the end of the coupled run. (b) The difference between the generation mix in the two models as a share of the total REMIND generation.

Figure 6 Capacity convergence at the final iteration of a coupled run for Germany under the proof-of-concept baseline scenario. (a) Side-by-side comparison of the two models' capacity mix at the end of the coupled run. (b) The capacity difference between the two models as a share of the total REMIND capacity.

For periods that are policy-relevant in the short to medium term (i.e., before 2070), the convergence for quantities is generally slightly worse in the near term, i.e., in the 2020s and 2030s, likely due to the capacity bound mismatch in the near term (such as the capacity bounds c5–c6 in REMIND not being completely replicated by the standing capacity constraint c8 in DIETER). If DIETER does not contain identical bounds to REMIND, then its endogenous decision will have more of a green-field rationale than REMIND does, the latter of which is more constrained in the near term. In case an improvement in near-term convergence is desired, these bounds could be implemented more carefully and be more technology-specific. Due to the limited scope, we only apply a generic standing capacity constraint (c8) in DIETER to represent the basket of various constraints. The convergence of quantities is also not perfect in the green-field periods, such as after 2040, when both models are less constrained by near-term dynamics. The reason for this is likely the fact that, in DIETER, hydroelectric generation is not economically competitive against other, cheaper forms of generation such as solar and wind. However, in REMIND it is economically competitive, likely due to the long lifetime of the plants. Semi-exogenous wind offshore capacities in both models could also play a role. This is discussed in more detail in Sect. 6.1.

4.3 Zero-profit rules for the coupled model

As our analytical discussion showed before in Sect. 3.2.3, model equilibria in the form of ZPRs are useful for validating convergence in a more detailed way by decomposing prices into cost components as well as any perturbation from capacity shadow prices. In this section, we first compare the system LCOEs, price and capacity shadow prices of the two models for ZPRs on the system level, and then we show the technology-specific ZPRs. Using this validating step, we can visually ascertain that the cost components and prices or market values in the two models are remarkably similar on the system level as well as on the technological level, demonstrating that the underlying principle behind the coupled convergence holds to a good degree.

4.3.1 System-level zero-profit rule

At the convergence of the soft-coupled model, we expect ZPRs to be satisfied for the two systems individually (Eq. 11 for REMIND and Eq. 13 for DIETER); i.e., each price time series also matches the LCOE time series to a good degree, barring distortions from the capacity shadow prices. This is to say, under full convergence, that the time series of system LCOEs and the sum of the time series of the electricity prices and time series for capacity shadow prices for both models should overlap one another within numerical tolerance. The costs and prices at the last iteration of the coupled run are summarized in Fig. 7. The electricity prices derived from the shadow prices of the balance equations are shown in dark grey: (a) REMIND electricity price λ_y and (b) DIETER annual average electricity price ${\stackrel{\mathrm{‾}}{J}}_y=\frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{d}}_{y,h}}{{\sum }_h{\stackrel{\mathrm{‾}}{d}}_{y,h}}$. Adding all the sources of capacity shadow prices, we obtain the blue lines: (a) REMIND capacity constraints (c5–c7) and (b) the DIETER capacity constraint (c8). All capacity shadow prices have been converted to per energy unit via capacity factors. (Note: Fig. 4 shows the difference between the black lines without considering the capacity shadow prices. See Sect. 3.2.3.)

From Fig. 7, we can conclude that the ZPR for DIETER is satisfied to very good accuracy for every year (the blue line: the sum of the electricity price and the capacity shadow price has exactly the same value as the sum of the LCOE bars). For REMIND, the ZPR is satisfied year on year to a lesser degree but on average to a good degree given the interannual fluctuations. The prices in the coupled REMIND become very erratic for the early years (2020–2025), likely due to the interaction between the historical or near-term bounds in REMIND and the exchanged information from DIETER for those years. The LCOE component structures match well across the models for most years, which serve as additional visual support for the price convergence shown in Fig. 4; i.e., the cost structures behind the prices are harmonized as well at the end of the coupling. The origins of the differences between LCOEs and prices as well as the degree with which capacity shadow prices account for them can be found when one examines the LCOE and market values of specific technologies, which are analyzed next.

Figure 7 Cost components of the system LCOEs (bars), electricity prices (grey lines), and the sum of electricity prices and capacity shadow prices for (a) REMIND and (b) DIETER under the proof-of-concept baseline scenario. Visually, the ZPRs for both models are satisfied within numerical tolerance. The intertemporal structure of the LCOE breakdown is very similar for most of the coupled periods. For DIETER, a small remaining difference exists between the price (grey line) and the LCOE (bars), which can be entirely explained by the capacity shadow price due to the standing capacity constraint. The REMIND price time series is a rolling average of three time periods. The large negative adjustment costs in 2020 are due to coal and nuclear phase-out.

4.3.2 Technology-specific zero-profit rules

After validating ZPRs on the system level, we further dive into each technology and check the ZPRs for each technology in both models at the last iteration of the coupled run (Fig. 8).

Figure 8 Technology-specific costs and market values for (a) REMIND and (b) DIETER under the proof-of-concept scenario. Cost components of the technology LCOE are plotted in stacked bars. Market values are shown in solid black lines. The sums of market values and all sources of capacity shadow prices are shown in dashed lines: for DIETER (two-dash blue lines), they contain mostly the standing capacity shadow price and to a small extent the capacity shadow prices of the resource constraint; for REMIND (dashed blue lines), they contain mostly the peak demand capacity shadow price and small capacity shadow prices due to brown-field and resource constraints. Electricity prices are shown in purple solid lines as references. Due to large positive shadow prices in 2020 due to fixings to the historical capacities, only periods beyond 2020 are shown. REMIND market values and capacity shadow prices are a rolling average of three time periods.

In Fig. 8b, DIETER LCOE and market values for the eight types of generators are shown. As expected from the ZPR, the LCOE always matches the sum of the market value and capacity shadow prices for each technology and for each year (Eq. 12). The differences between the dashed and solid lines are largely the generation capacity shadow prices. It is worth noting that, at the end of convergence, the sizes of the shadow prices are in general small for the main generator types, e.g., solar, wind onshore, CCGTs and OCGTs. This indicates the fact that, for these technologies for most periods, the optimal DIETER generation mix is close to that of a green-field model. That is, DIETER hardly faces any exogenous constraints (except resource constraints that are aligned with those of REMIND) and can make fully endogenous investment and dispatch decisions based on cost information alone. On the whole, DIETER at the coupled convergence experiences only a small amount of distortion from the brown-field model REMIND, especially concerning the “model-suboptimal” real-world standing capacities from biomass, hydro and coal.

In Fig. 8a, we show the REMIND LCOE and market values for the same generation technologies. Due to the intertemporal nature of REMIND, the sum of the market value and the capacity shadow price for each technology and for each year generally matches the LCOE slightly less well than DIETER. This means that for REMIND the ZPR (Eq. 10) for each generator type is also satisfied to a good degree for the main generator types, e.g., solar, wind onshore, coal, CCGTs and OCGTs. The mismatch in biomass and hydro might come from the shadow price from historical capacities.

Since the differences between market values and costs are accounted for by the capacity shadow price to a large degree, it is worth interpreting physically the sources of these “hidden” costs and revenues. For REMIND, the capacity shadow prices consist of those in (c2), (c5) and (c6) as well as the “peak residual demand constraint” from DIETER (c7). Constraint (c7) is created to circumvent high markups, especially from peaker gas plants (Appendix H1), because peaker gas plants generate power mostly only at hours with high prices (especially the scarcity hour price) and therefore have very high market values compared to the annual average electricity price. The high market values of OCGT – usually more than 5 times the average annual electricity prices – act as a large incentive in the next-iteration REMIND and lead to over-investment in capacities. Over iterations, this causes oscillations in the quantities and prices in the coupled model and prevents model convergence. To circumvent the issue of high markup, we implement (c7) as an equivalent peak residual demand constraint. As can be shown mathematically (Appendix H), (c7) generates essentially the scarcity hour price, and it is very easy to validate this for OCGT in Fig. 8a. The capacity shadow price derived from this peak residual demand constraint, when translated to energy terms and added to the market value, correctly recovers the LCOE for OCGT, recovering the original ZPR (Appendix H1.2). This indicates that, under multiscale model coupling, an extra constraint is an effective way of circumventing potential issues of numerical divergence due to the large impact from short-term dynamics, such as the large market value of peaker gas plants.

For DIETER, the two sources of the capacity shadow price are the total renewable potential limit (constraint c2 in Sect. 3.1) and the standing capacity constraint from REMIND (constraint c8 in Sect. 3.2.3). For the first type, the resulting capacity shadow price is a hidden “positive cost” from the perspective of the power user since endogenously DIETER would like to invest more but is limited by the natural resources available. An example of this first type is hydroelectric power between 2020 and 2035 due to the limited resource (run-of-the-river) in Germany. It is worth noting that, from the generator's perspective, the capacity shadow price from the resource constraint can be interpreted as an extra resource rent. The second type of capacity constraint originates from the standing capacity, and the latter is received by DIETER from REMIND as a lower bound. This constraint usually results in a hidden “negative cost” from the perspective of a power user; i.e., a part of the cost (LCOE) does not get passed on to the electricity price, so the users get part of the capacity “for free”. (This can also be interpreted as subsidies for generators to sustain these unprofitable capacities.) This is because, based on green-field cost optimization, DIETER endogenously would invest less in certain technologies. However, since the standing capacities account for the existing generation assets in the real world, which can be model-suboptimal, the overall costs are above a green-field equilibrium and above the prices the user pays. We find examples of such a capacity shadow price manifested in biomass, coal and hydroelectric, all of which are part of the existing German power capacity mix, but evidently not all of them for any given period are “green-field optimal” based on the pure cost consideration in DIETER. Interestingly, after 2035, the sign of the capacity shadow price for hydroelectric generators reverses. This is likely due to the continuous decline in the VRE costs after 2035 tipping the power sector into a regime where hydroelectric becomes less economically competitive in DIETER, at least compared with REMIND. As a result, the standing constraint from REMIND starts to be binding on the capacity from below, relieving the resource constraint binding from above.

For DIETER, the capacity shadow price from the standing capacities also indicates the degree of disagreement between DIETER and REMIND. For most future years, REMIND standing capacity constraints are not binding in DIETER for solar, wind onshore, CCGTs and OCGTs, indicating good agreement between the models. The small amounts of shadow prices near 2060 for OCGTs and solar in Fig. 8b are likely due to the time step size change in REMIND which causes a small jump in the interest rates near these years.

Lastly, in Fig. 4, we observe a slightly higher average electricity price in REMIND than in DIETER, especially in the intermediate years. This could be due to fixed offshore wind capacities, which are never economical for endogenous investment in the parameterization used here. This generates a high capacity shadow price until around 2045–2060 that is visible in both DIETER and REMIND.

5 Scenario results under baseline and policy scenarios

In this section, we present baseline and policy scenario results for Germany using a more realistic configuration of the coupled model with electricity storage and flexible electrolyzer demand for green hydrogen production, which is then used outside the power sector (e.g., in industry or heavy trucks). We show results for a baseline scenario and a net zero by the 2045 climate policy scenario. Note that, due to REMIND's global scope, under the net-zero scenario we also assume a larger climate policy background of a 1.5 ^∘C goal for end-of-century temperature rise globally (corresponding to 500 Gt of the CO₂ emission budget until 2100) and a larger regional goal of EU-wide net-zero emission. Both scenarios consider nuclear phase-out law in Germany.

In Sect. 5.1, we present long-term power sector development. In Sect. 5.2, we present short-term power sector hourly dispatch and price results. In the following, we broadly describe how these additional features are implemented.

1. Storage: we use a simple storage implementation where DIETER makes an endogenous investment in two kinds of storage technologies:

   • 1.

     lithium-ion utility-scale batteries;

   • 2.

     on-site green hydrogen production via flexible electrolyzers, storage and combustion for power production.

   The principle of the coupling remains mostly unchanged. REMIND receives the price markups from generation technologies as in the case before without storage. However, for simplicity, the capacities of storage are not part of endogenous investment in REMIND. In REMIND, the energy loss due to storage conversion efficiency is taken as a fraction of the total demand from DIETER as a parameter and stabilized with a prefactor for each type of renewable generation (similar to the case of the curtailment rate in Sect. 3.3.2, point 4). Our battery cost development is given in Sects. S1–S2 in the Supplement.

   The reason we only allow DIETER to endogenously invest in storage technologies is that the additional intertemporal optimization offered in REMIND is relatively less important than that for the investment of generation technologies. In REMIND, intertemporality mainly accounts for two aspects in the real world: (1) implementing adjustment cost and (2) tracking standing capacity. The adjustment costs simulate system inertia to rapid capacity addition or removal. In the case of battery and other storage technologies, the ramp-up of deployment faces relatively less inertia compared to wind and solar. Compared to generation technologies such as wind and solar, the storage technologies tend to have lower total capacities, meaning their ramp-up rate is usually lower. Also, their deployment is mostly constrained by their higher cost. For utility storage technologies, they are mostly not yet deployed at scale, which means that there is very little existing capacity and the investment in storage in REMIND is mostly green-field, rendering it unnecessary to give DIETER a standing capacity of them.

2. Flexible demand: as a simple representation of flexible demand, we choose to implement a common power-to-gas (PtG) technology, i.e., the so-called “green hydrogen” electrolysis. We split the total power demand required to produce green hydrogen from REMIND from the total power demand d_y(i−1) (Sects. 3.3.1 and 3.3.3) – both demands are endogenous in REMIND. We implement the electrolysis demand as completely flexible in DIETER, i.e., no ramping cost or constraint, thereby flexibilizing part of the endogenous total power demand d_y(i−1) in REMIND. As a result, the cost minimization in DIETER automatically allocates the flexible demand to hours where electricity costs are low due to the existence of low-cost VRE. The economic value of flexible demand can be quantified by the capture price. The annual capture price of demand-side technology s_d is the annual average price of the hours when the flexible demand consumes electricity, weighted by the hourly flexible power demand by electrolyzers: ${\stackrel{\mathrm{‾}}{\mathrm{CP}}}_{s_d}=\frac{{\sum }_{h,s_d}{\stackrel{\mathrm{‾}}{d}}_{h,s_d}{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h}{{\sum }_{h,s_d}{\stackrel{\mathrm{‾}}{d}}_{h,s_d}}$.

This concept is equivalent to the market value for a variable or dispatchable generator but here for a flexible or inflexible demand source. Similarly to before, we implement a stabilization measure using a prefactor (Appendix H2, point 5).

5.1 Long-term development

This section presents scenario results of the coupled model with a long-term view of capacity and generation using either the proof-of-concept scenario or more realistic configurations.

5.1.1 Baseline scenario

In Fig. 9a, under the baseline scenario, and with available storage and flexible demand, we observe a more than 35 % increase in the total power demand from 2020 to 2045 and more than 65 % by 2080. This is due to an increase in end-use electrification. The increased electrification comes from a moderate growth in electricity use in the building sector and a more significant growth in the electric vehicle (EV) fleet. In the building sector, the final energy share of electricity is projected to increase from 28 % in 2020 to 39 % in 2045. The final energy share of electricity in the transport sector is 22 % by 2045, up from 2 % in 2020. Note that, even under no additional climate policies, based on only the increase in EVs shares in new-car sales in many world markets today, we expect higher power usage from EVs in the future. Within the energy mix, we see a slow decline in coal generation over time, which is replaced by CCGT generation and a significant increase in VREs. The VRE share reaches above 50 % by 2045, but slightly less than half of the energy mix still contains coal and gas power. In terms of capacity expansion (Fig. 9b), due to both a lower generation cost and a higher power demand, solar capacity expands by almost 5 times from today until 2045. However, the moderate VRE shares mean that the requirement for battery capacity is not high, i.e., only 12 GW of batteries by 2045. Due to the low CO₂ price, long-term electricity storage through hydrogen does not appear to be economically competitive and is not invested under the baseline.

Figure 9 DIETER–REMIND converged results of the long-term (a) generation and (b) capacity expansion for Germany's power sector in the baseline scenario, assuming a constant USD 37 / tCO₂ CO₂ price. Dashed lines represent generation before storage loss and curtailment. Storage generation is not visualized in panel (a).

By comparing the above baseline scenario (with storage and flexible demand) (Fig. 9) with the proof-of-concept baseline scenario (without storage or flexible demand) before (Figs. 5 and 6), it is clear that, while battery storage and partial demand flexibility play a role after 2040 in increasing the VRE share in Fig. 9, in the near term, the scenarios with and without available storage and demand flexibility look very similar under no additional climate policies. However, due to the technological learning effect, an even absent additional CO₂ price policy, the energy mix here has a relatively high VRE share (> 60 %) after 2050 compared to the basic case without storage and demand-side flexibilization. However, due to the low CO₂ price there is still a significant share of dispatchable technologies such as CCGTs and OCGTs, which is more economical than the implementation of long-term power storage via electrolysis and hydrogen turbines.

5.1.2 Net-zero policy scenario

In Fig. 10, under a stringent climate policy (economic-wide carbon neutrality in 2045), with available storage and partially flexibilized demand (for hydrogen production used in other sectors), the total power demand more than doubles, and the power mix is dramatically transformed. Compared to both the baseline case without storage and demand-side flexibilization (Figs. 5 and 6) and the baseline scenario with storage and flexible demand (Fig. 9), a very high VRE share in the generation mix is reached already by 2040 (> 94 %). This is mostly due to an earlier investment in VREs to drive down the cost, combined with the increased deployment of both short- and long-term storage and flexibilization of part of the demand. Capacities for storage increase significantly: lithium-ion batteries from 18 GW in 2020 to 125 GW in 2045, and 37 GW of hydrogen electrolysis and hydrogen turbine capacity (with ∼ 40 TWh of H2 storage capacity). Despite high storage capacities, due to a high VRE share, curtailment and storage loss still increase quite significantly with time, especially for solar PV. However, note that in a coupled run where interregional transmission expansion is possible connecting Germany and the rest of Europe, this loss can be reduced (see Sect. 6.3). In terms of capacity expansion (Fig. 10b), gas power plants are mostly replaced, as hydrogen turbines fill the role of peaking dispatchable plants that guarantee supply for peak demand hours. The CCGT gas turbines are equipped with CCS.

Figure 10 DIETER–REMIND results of the long-term generation and capacity expansion for Germany's power sector in the “net-zero 2045” scenario. The CO₂ price is endogenously determined based on the climate goal. It is USD 115 / tCO₂ for 2030, USD 292 / tCO₂ for 2035, USD 464 / tCO₂ for 2040, and USD 636 / tCO₂ for 2045. Dashed lines represent pre-curtailment generation. Storage generation is not visualized in panel (a).

Under the stringent climate policy scenario, dramatic changes in the end-use sectors will be under way in the form of direct electrification and substitution of fossil gas with hydrogen. In the building sector, the final energy share of electricity is projected to increase from 28 % in 2020 to 66 % in 2045. In transport, the final energy share of electricity will be 56 % by 2045. In the industry sector, the share of electricity increases from 25 % to 63 %. By 2045 there is also a notable increase in the use of green hydrogen produced from 45 GW flexible electrolyzers (at about 42 % of the average annual capacity factor), amounting to 0.5 EJ (3.5 million tons) per year in the final energy, which is primarily used in industry. For a comparison with other published German net-zero scenario results, see Sect. S4.

5.2 Short-term dispatch

In this section, results of hourly resolution are shown and discussed for a selected model year. We use established methods such as RLDCs to visualize the hourly dispatch result and show the hourly generation and dispatch time series for some typical days in summer and winter.

5.2.1 Residual load duration curve model comparison

RLDCs can be used to visualize the dispatch of energy system models. Each subsequent curve is calculated by subtracting the generation of a technology from the hourly residual demand curve and then sorting the remaining demand in descending order. On the left-hand sides of the RLDC graphs, one can easily check the amount of residual demand not met by variable wind and solar production. The topmost line in the RLDC graph is the load duration curve for inflexible demand (excluding the demand from flexible electrolysis for hydrogen production used in other sectors).

In a baseline configuration without flexibilized demand or storage, despite lacking the explicit hourly dispatch, via bidirectional soft linkage, REMIND was able to achieve a final dispatch result that replicates DIETER to a satisfactory degree (Fig. 11). This is a combined effect of a convergence of capacities (Sect. 4.2) and full-load hours at the end of the coupled run. In the peak residual demand hour (the leftmost point in the RLDC), the DIETER-coupled REMIND accounts for the requirement of dispatchable capacities via the constraint (c7), and the composition of the mix is replicated from DIETER and correctly guarantees that the peak hourly demand is met.

Figure 11 Side-by-side RLDC comparison between (a) REMIND and (b) DIETER for the simple configuration under the baseline scenario without storage or flexible demand. The DIETER RLDC (b) is constructed by subtracting hourly generation from hourly load and sorting with dispatchable generation technologies plotted in order of their annual average capacity factors. VREs are arranged such that the generation with a higher curtailment rate (i.e., solar in this case) is on the inside of the graph. To construct the REMIND RLDC (a), the dispatchable generations are sorted by their capacity factors and stacked from the bottom. The rectangles depicting dispatchable generation are made up of the width equal to the full-load hour and the height equal to the capacity. The top-most lines on either side are load duration curves (sorted hourly demand, which is entirely inflexible under this setup). For the purpose of better visualizations, solar and wind RLDCs are tilted at an angle for REMIND and plotted in the same order as the DIETER RLDC. For simplicity, in REMIND wind and solar RLDCs share the same top pivot point in the peak residual demand hour.

In a net-zero policy with storage and flexible electrolysis demands, comparing dispatch results under both scenarios (Figs. 11 and 12) for the model year 2045, it can be observed that, under a stringent emission constraint, the system allocates a significant amount of short-term storage to replace the dispatchable generation such as coal and CCGTs. Long-term storage such as hydrogen electrolysis combined with hydrogen turbines further reduces the capacity factor of the remaining OCGTs and CCGTs. Besides storage, there is also a significant amount of deployment of flexible electrolysis demand for producing hydrogen (PtG) that is not used in the power sector but is in industry or heavy-duty transport. The use of PtG technologies leverages cheap variable wind and solar energy to achieve the goal of sector coupling. By way of storage and PtG, a significant share of the curtailment can be utilized (more than 70 %), either by shifting the supply to times of low VRE production via storage or by producing hydrogen using surpluses which can be used in other sectors.

Figure 12 Side-by-side comparison between the (a) REMIND and (b) DIETER RLDCs for the “net zero by 2045” scenario with storage and flexibilized demand for Germany. The storage loading and discharging in the DIETER RLDC (b) are constructed by subtracting hourly loading or discharging from hourly inflexible load and sorting. The REMIND RLDC (a) is constructed similarly to Fig. 11. The top-most lines on either side are load duration curves for inflexible demand. For better visual comparison, in REMIND the solar RLDC starts at 80 % of the peak residual demand.

5.2.2 Hourly dispatch and power consumptions for typical days in summer and winter

To more directly inspect the results of the hourly dispatch under various scenarios, we visualize the hourly generation and demand for typical days. Due to the climate in Germany, solar potential is particularly low during winter months. Therefore, it is important to observe the periods in both summer and winter.

From the optimal hourly dispatch results of typical days from the coupled model, we observe that, compared to the baseline (Fig. 13a–b), in 2045 for a net-zero year (Fig. 13c–d), there is a significant amount of surplus solar generation in the summer during the day and some amount of surplus wind generation in the winter during nights and days. Under a net-zero scenario, the generation from fossil fuel plants in the baseline is replaced by a battery dispatch (especially in summer) and hydrogen turbines (especially in winter), and the peaker plants, which under the baseline are turned on in the summer evening, are partially replaced by solar overcapacity and batteries. A significant share of renewable surplus energy is used for the production of green hydrogen – hydrogen made from zero-carbon electricity. Due to the complete flexibility of electrolyzers, the capture price of hydrogen production is only around one-third of the average price of electricity (Sect. S2 and Fig. S1 in the Supplement).

In winter, hydrogen turbines serve as a base load for the few days when wind generation is insufficient to meet the demand. To ensure supply during longer winter periods of “renewable droughts” with little wind and solar output, e.g., over a 2–3 d period (hours 540–600 in Fig. 13d), long-term storages with hydrogen electrolysis and hydrogen turbines and some dispatchable generation (such as CCGTs with CCS and an integrated biomass gasification combined cycle) play a major role.

Figure 13 Comparison of hourly generation (positive) and consumption or storage loading (negative) for a few consecutive typical days in two seasons in Germany in 2045. (a) Summer, under the proof-of-concept baseline scenario, no storage, or flexible demand. (b) Winter, under the proof-of-concept baseline scenario, no storage, or flexible demand. (c) Summer net-zero scenario with storage and flexible demand. (d) Winter net-zero scenario with storage and flexible demand. Due to the fact that modern electrolyzers are very flexible, no ramping costs are applied to them in the models, and therefore some switching behavior between PtG electrolyzers turning on and off can be seen, but this is a minor artifact.

6 Discussion

In this section, we discuss the reasons for remaining differences between the coupled models as well as the assumptions and limitations of the soft-coupling.

6.1 Remaining discrepancies

In all our test runs, at the end of the coupling, it is always the case that the two models cannot be perfectly harmonized, and there is a slight residual difference in the convergence results (Sect. 4). The reason is two-fold.

The first reason is “legacy mismatch”, i.e., a mismatch in brown-field standing capacity constraints in the two models. The coupling method we develop here is mostly based on price information for achieving convergence. Therefore, capacity constraints that are present in the standalone long-term model but not in the standalone hourly dispatch model need to be transferred. These standing capacities are hard to evaluate purely based on economic terms, as they are ultimately a result of real-world actions and policies that might not align with the simplified economic incentives in techno-economic energy models. Therefore, the only way this information can be transferred from the brown-field model to the green-field model is by implementing a lower capacity bound in the latter. However, this bound nevertheless might not capture all the shadow prices caused by the standing capacities in REMIND. This is ultimately due to the specific generic form of the constraint we implemented: i.e., we pass on the pre-investment capacities as a lower bound regardless of the technology types. In general, hidden “legacy revenues”, which are manifested as the shadow prices of economically less competitive generators in DIETER, such as biomass, coal or hydroelectric (solid line lower bars in Fig. 8), provide incentives for brown-field models to deploy them over the long term but do not provide enough economic cases for the green-field model. This results in an observed phenomenon in the coupled run that, if these legacy capacities and their impact on the costs have not been fully transferred to the green-field model, the prices of the green-field model tend to be lower than the coupled brown-field models, causing distortion of the convergence of quantities. The effects of legacy mismatch and illustrative test run results are discussed in more detail in Supplement Sect. S3.

The second reason for the discrepancies at the end of the coupling is the actual mismatches in the Lagrangian harmonization itself, which can originate from multiple sources. It could be due to intertemporal constraints and dynamics (such as adjustment costs and brown-field constraints) not linearly reducible to single-year dynamics, resulting in misalignment between the multi-period REMIND and the single-year DIETER. It could also be due to slight numerical inaccuracies of the interest rate estimate that are not explicit in REMIND but that are derived from endogenous and intertemporal consumption. Lastly, there could be a mismatch due to a linear fitting of REMIND endogenous time series of fuel costs (biomass, oil, coal, uranium) before passing this information on to DIETER, which might result in a small amount of mismatch for fuel costs between REMIND and DIETER.

6.2 Limitations of the coupling methodology

There are limitations to our proposed methodology in terms of both convergence of two multiscale power sector models and other potential applications of model convergence. Firstly, in terms of the problem presented here – a multiscale power sector model coupling –, the method derived here is only necessary for a full convergence but may not be sufficient; i.e., a full convergence is not guaranteed. A number of additional factors could prevent a full convergence. One is the legacy mismatch and misalignment in Lagrangian mappings mentioned above in Sect. 6.1. Another factor is the role prefactors play (Sect. 3.3.2, Appendix H2). The prefactors help stabilize the coupling by turning exogenous values obtained from the last-iteration DIETER into endogenous values in REMIND, such that they can be adjusted to be in line with the optimal mix of the current iteration. However, they usually contain some small positive or negative parameters that are determined heuristically (e.g., ${\stackrel{\mathrm{‾}}{b}}_{y,s}$ in Eq. H13). These heuristic parameters usually come from rough estimates based on relations between variables in the system and generation shares, e.g., how much the market value of solar generation will decrease when the solar generation share increases by a certain percentage. In practice, while the prefactors help stabilize the run and improve the convergence speed, choosing the wrong prefactor parameters can lead to divergence or instability. Second, another limitation when it comes to modeling power market multiscale coupling is the number of products on the market. In the formulation here, both models describe the general equilibrium of a competitive market with one type of homogenous goods, i.e., electricity. However, if we introduce heat as a byproduct, such as from a combined heat and power plant, then there are two types of goods: heat and electricity. The feasibility of coupling models with more than one type of good or market has not yet been explored. Thirdly, there are multiple iterative processes that are internal to REMIND, which happens concurrently with the DIETER–REMIND coupled convergence. Among these processes, the DIETER and REMIND Nash algorithms (for interregional trading) both run between the internal REMIND Nash iterations, which means that they are external to the REMIND single-region optimization problems and therefore are soft-linked. Nevertheless, in our runs, we observe the power sector convergence to be rather swift and smooth and to happen in parallel to other iterative processes, such as the Nash algorithm and the CO₂ price path algorithm (for climate policy runs). However, systematic monitoring of the multiple internal convergence processes in REMIND during the REMIND–DIETER convergence processes under other model setups and configurations is still to be more thoroughly researched. More generally, the approach developed here – the Lagrangian mapping method for converging two multiscale optimization problems – could be useful for a general modeling of market equilibrium of multiple time resolutions. In this study, the resolution in the coupled problems is specifically only meant for temporal resolution. However, mathematically speaking, coupling models of different spatial resolutions (or both temporal and spatial resolutions) should be very similar. At least in theory, the soft-coupling approach developed here should be applicable to increasing the resolution in any arbitrary independent/orthogonal dimension of the problem of finding equilibrium market dynamics. In theory, it is also possible to build a multi-layer coupled problem architecture, where at each level the low-resolution variables can be disaggregated into finer resolution along some dimensions. However, further research is needed to explore the feasibility and convergence performance of such schemes.

6.3 Limitation of coupled results

Since the nature of this study is a proof of concept, the scenario results presented should be primarily interpreted as such. Nevertheless, it may be useful to enumerate a list of limitations for a more accurate interpretation of the results.

1. The power sector is only coupled for one single global region; i.e., information exchange only occurs for the variables of one region, Germany, while all the other regions contain the low-resolution version of the power sector of the uncoupled REMIND. The former coupled one-region result is based on a time series of VRE production today in a world of low- to medium-VRE shares and a very limited power grid expansion (in 2019). The latter results of the uncoupled regions however are parameterized based on results from a detailed PSM under a more optimistic assumption of transmission build-out, which allows VRE pooling from an expanded EU-wide power grid to smooth out regional weather variations (Pietzcker et al., 2017). Note that, in the standalone REMIND, while by default there are no annual electricity import and export imbalances between countries and regions, transmission during the year is implicitly assumed, especially for the EU region. Comparing the capacity and generation mixes of the coupled and uncoupled runs (Appendix J), we find that, in the uncoupled case, there are slightly more solar and wind capacities and generations and much less gas generation in the long term. EU-wide transmission expansion would pool both supply and demand variability, thus reducing the need for dispatchable capacity for meeting the peak demand.

2. Due to the scope of this study, we implemented a limited set of options in storage and sector-coupling technologies in this study and neglected the additional supply-side details for the German power market (such as the reserve market). Many potentially significant technological options consisting of pumped hydro storage, compressed-air energy storage, vehicle to grid and flexible heat pumps are not explicitly modeled.

3. Ramping costs for dispatchable generators are not considered, although the effect should be small (Schill et al., 2017).

4. In terms of power transmission and trading inside Germany, we assume a very simple “copperplate” spatial resolution, not explicitly modeling transmission bottlenecks inside the region. Currently, the grid capacity equation is parameterized to be proportional to pre-curtailment variable renewable generation, and the parameterization is rather optimistic based on PSM studies conducted in Pietzcker et al. (2017). As hinted at in a recent work by Frysztacki et al. (2022), the lower level of spatial detail results in an underestimation of constraints present in a real electric system, leading to an underestimation of system cost.

5. Near-term events: we have not modeled the current gas and energy crisis in Europe, which is likely to imply an overestimation of near-term gas availability in the power sector. Relatedly, we are likely to have overestimated the early retirement of coal power plants, which are capped at a maximum of 9 % yr⁻¹ of the current capacity early retirement rate in REMIND if this is uneconomical relative to cheaper sources of generation. We have included the COVID shock in the GDP projection.

6. Only one weather year (2019) is used for the DIETER input data. From the perspective of sufficient power supply under all weather conditions with few blackout events, this could introduce an underestimation of the need for reserve capacity, storage and demand-side flexibility.

7. Climate impacts under various scenarios on building sector power demand are not included in the current version of REMIND or its energy demand model for the building sector “EDGE-B” (Levesque et al., 2018). Climate extremes such as heat waves are not included in either model due to the fact that annual degree days are used, which are the results of temporal averaging. Representative weather years which maintain the temperature extremes and can represent long-term trends are also not used. However, the demand projection does change in a minor way based on SSP scenarios due to their different population projections.

8. “Perfect foresight” is assumed under REMIND's intertemporal optimization over several decades and therefore is also assumed under the coupled model. There exist many discussions related to the differences between the “ideal world” depicted in IAMs and energy system modeling on the one hand and “imperfect” but realistic real-world decision making and political economy on the other (Ellenbeck and Lilliestam, 2019; Geels et al., 2016; Keppo et al., 2021; Staub-Kaminski et al., 2014; Pahle et al., 2022). Considering that perfect foresight models such as REMIND dominate IPCC model results, it is especially important to understand the differences between the approaches with perfect foresight and those without it (the so-called “myopic models”). Such work has been carried out in studies such as Fuso Nerini et al. (2017) or Sitarz et al. (2023). If myopia is introduced in the model, the climate policy exemplified by carbon prices still follows an increasing expectation for more and more stringent climate policies, but the trajectory can be less smooth and in the near term looks more “flat”, hence inducing lock-in effects which slow the transition in the near term. These additional lock-in effects are not modeled in our work here.

9. The resulting power mix is largely due to limited options within the available energy portfolio due to Germany's energy policy and natural resources, e.g., the political decision of nuclear and coal capacity phase-out as well as limited hydro and offshore wind potential. In future research, we would like to apply the same method to all global regions.

6.4 Potential computational barriers under soft-coupling

Even though via soft-coupling an IAM can obtain hourly resolution with only a moderate computational cost increase, it nevertheless increases the complexity of the whole problem, increasing the solver time of the IAM, especially before convergence is reached under the iteration with a PSM. With additional complexity of endogenous climate policies, computational time can be long for scenarios under climate constraints (see Appendix E). This can be potentially overcome by several measures that can be topics for future research.

1. Optimize for computational costs in individual models. Individual IAMs and PSMs are usually developed incrementally, which results over time in less overall computational efficiency. However, because individually the models are not too costly to run, there are fewer incentives to manage computational costs when they are run as standalone models. However, when coupled, a computational cost may become a barrier. One of the easiest ways to reduce coupled run time is to reduce run times of the individual coupled models. Because the soft-coupling takes many iterations, a small reduction in computational time in either model will multiply to give a large reduction in iterative soft-coupled runs.

2. Other internal iterations of the IAM (if they exist) can be optimized. For example, in REMIND, most of the iterations (usually 30–50 iterations) in the coupled runs are dedicated to converging interregional trade between the 21 regions in the model, because the DIETER iteration usually converges quite quickly (5–10 iterations). By making the algorithm for the convergence of interregional trade faster, we can reduce the total coupled iterations, thereby reducing the overall computational cost. Less computational time can also be achieved if DIETER is no longer run together with REMIND after DIETER–REMIND iteration convergence is reached and when trade adjustment (or other internal adjustments in REMIND) is small enough to not have a substantial impact on the power sector results. This is especially the case if a PSM becomes more complex and its computational time exceeds far more than the single-iteration REMIND time (see also Appendix E for a comparison of the contributions to run time due to REMIND internal iteration and due to PSMs).

3. Limiting endogenous investments of capacities of certain technologies only in one model: for example, in the case of electricity transmission, more than one region (e.g., Germany with neighboring European countries) will need to be hard-coupled together in the PSM, which naturally increases the computational cost of the PSM. However, when the solutions are passed to the IAM, the regions can again be parallelized as long as an IAM does not engage in the endogenous investment of the transmission capacity. Hence the increased cost of computation due to implementing transmission is only limited to the PSM. This is also the case if within Germany the spatial resolution is increased.

4. Only include essential features in the PSM. Some PSMs are quite detailed and complicated for the purpose of studying specific technologies and the behavior of many agents or users. To couple to the IAM, the PSM should consider coarse-graining or aggregating some details while retaining the essence of the dynamics being studied. For example, to implement smart EV charging (e.g., vehicle to grid), modelers of PSMs should create a version for coupling that aggregates the many time series of charging and discharging of EVs to only one or two time series.

Faster solvers and faster supercomputers will also contribute to improving the computational efficiency of the coupled model.

7 Conclusion and outlook

In this study, we develop a new method of soft-coupling an IAM with a coarse temporal resolution and a PSM with an hourly temporal resolution. Our coupling method can be shown both mathematically and in practice to produce a convergence of the two systems to a sufficient degree. This method allows the incorporation of the temporal details of variable renewable generation explicitly into large-scope IAM modeling frameworks and increases the accuracy of power sector dynamics in long-term models. Furthermore, it allows more explicit modeling of the power sector and sector coupling as well as a vision of the energy transition where end-use demand sectors such as building, industry and transport make economic use of the generation from variable sources by

1. directly using the power at the time of production for inflexible forms of demand,

2. shifting the time of the power supply via battery and other power storage technology, and

3. transforming this into another energy carrier or product ahead of times of consumption and at times of surplus wind and solar production (e.g., PtG), without conversion back to electricity.

The fully coupled framework allows a more explicit modeling of economic competition of these options under high shares of variable renewables, finding more accurate optimal paths under long-term climate scenarios towards a net-zero power sector and the wider economy globally. In future research we plan to expand the study in the direction of demand-side management and flexibilization and later possibly in the direction of heat storage.

Coupling DIETER to the global model REMIND for the single region Germany, this study serves as a proof of concept. Our main innovation is two-fold: we derive convergence theoretically and show almost full convergence numerically. Theoretically, we derive the coupling methodology by mapping the KKT Lagrangians of the simplified versions of the two models. One key aspect of the mapping consists of iterative adjustment of the market value (i.e., the annual average revenue of one energy unit of generation) or the capture price (i.e., the annual average price of one energy unit of consumption) in the low-resolution IAM such that they take on the values like those in the high-resolution PSM. By finding the set of mathematical coupling conditions necessary for an iterative convergence as defined by the convergence of both quantities and prices, we could then design the coupling interface accordingly such that, at the end of the coupling, a joint optimal result can be found.

Numerically, we compare the converged results of the two models by examining the long-term power mix (both capacity and generation quantities) and prices of electricity and generation dispatch (via RLDCs), and we find good agreement between the two models at the end of coupled convergence despite some slight mismatches. For a proof-of-concept baseline scenario under a simple configuration without storage or flexible demand, we could achieve an energy mix with 4.4 % tolerance for any technology's absolute share difference in each time step. For a climate policy scenario under a more realistic configuration with storage and flexible demand, we could achieve 6 %–7 % tolerance. The cost breakdown and prices of power generations for both models are found to be very similar at the end of the iterative process, providing additional evidence that the quantity harmonization follows the underlying principle of price and cost harmonization. The remaining differences can be partially explained by the lack of full harmonization of the brown-field and near-term capacity constraints as well as potential mismatches due to numerical techniques aimed at enhancing performance and stability. Using the coupling methodology, we provide scenarios for power sector transition under a stringent German climate goal. Under this scenario, we observe a least-cost pathway consisting of an almost complete transformation to a wind- and solar-based power system. The results indicate an increasing role of storage and dispatchable capacity in a deep decarb scenario, consistent with the findings of previous PSM studies, but this is now transferred to the long-term models via soft-coupling.

For future works, besides expanding the research program on sector coupling into a direction containing a broader technological portfolio, we also aim to apply this framework to other world regions of interest in the REMIND model. Another important aspect would be to represent the variability-smoothing effect of transmission grids by using the same coupling framework to couple REMIND to other power sector models with more explicit modeling of transmission bottlenecks and expansion for two or more regions.

Appendix A: Comparison of model scope and specification

Because IAMs usually start out with certain assumptions for the development of macroeconomic metrics such as for GDP and population, which in turn determine the corresponding energy service levels to a larger degree prior to optimizing the energy system mix to meet demand, they are also frequently referred to as “top-down” energy system models. PSMs usually start out modeling the fine spatiotemporal detail of real-world power systems, expanding the capacity installation of power-generating plants, grid transmission and storage at minimum cost. Such models are also known as “unit commitment models” for electrical power production (Padhy, 2004). Later in model development PSMs are usually expanded to include other energy services such as heating and transportation that are electrified. In this way PSMs are also often referred to as “bottom-up” models. Reviews and intercomparisons of IAMs have been carried out recently where various IAMs are analyzed and harmonized (Weyant, 2017; Butnar et al., 2019; Keppo et al., 2021; Wilson et al., 2021; Giarola et al., 2021). For methodological reasons, we have to set the length of the model time horizon to be until 2150, which is longer than the valid model time horizon until 2100. This is because without the extra years after 2100, the model has much less time to utilize the capacities installed in the few decades before 2100, making it more difficult to justify the installation of new capacity economically. This is manifested in a model artifact where, in the last few model periods, investment in capacities decreases in general. By extending the time horizon, this “boundary” effect is pushed further into the future, so the artifact only appears after 2100. Therefore the meaningful model results for REMIND are only between 2005 and 2100, even though the years until 2150 are also modeled and coupled.

Reviews and intercomparison of typical scopes and resolutions of PSMs can be found in Supplement Sect. S5. Comparison of more PSMs can be found in Ringkjøb et al. (2018) and Prina et al. (2020).

Both models have open published source code. Partially thanks to the PSM community's advocacy of “open models”, which encompasses all steps from input data to model source code and numerical solvers (openmod – Open Energy Modelling Initiative, 2022), many research institutions have also responded to their calls to openly publish their models. For example, the IAM used in this study, REMIND, has for 2 years opened its source code on the popular hosting site GitHub.

Figure A1 Comparison of the (a) resolution and (b) scope for REMIND and a typical IAM as well as two versions of DIETER (v1.0.2 is used in this study) and a typical PSM.

Table A1 Comparison between the coupled models REMIND and DIETER.

Appendix B: Model-coupling scope

While REMIND and DIETER can both model a European-wide system with spatial subdivision (see Fig. B1 for the REMIND regional division), the soft-coupling is currently only applied to Germany, in line with the proof-of-concept nature of this study. The coupling is from 2020 to 2150 for every defined REMIND period. All common and available REMIND-generating technologies are enabled for the coupling, as shown in Fig. B2. The information for the species of technologies in REMIND is upscaled and coupled to DIETER, whereas information from DIETER is then downscaled during the feedback loop that completes the coupled iteration.

Figure B1 REMIND regional resolution used in this study (21 global regions, including detailed differentiations of EU regions). The spatial resolution of REMIND is flexible and depends on the resolution of the input data. Regional mapping is from the REMIND–EU model (Rodrigues et al., 2022).

Figure B2 Mapping of coupled technologies between REMIND and DIETER.

Appendix C: REMIND's interannual intertemporal objective function for a single region

Single-region interannual intertemporal welfare is an aggregated utility, which in turn is a logarithm function of consumption. In REMIND, the total welfare of a region is maximized and is equal to

$$W_{\mathrm{reg}}={\sum }_{y=\mathrm{2005}}^{\mathrm{2150}}{\displaystyle \frac{\mathrm{1}}{(\mathrm{1}+{\mathit{\varrho }}_{\mathrm{reg}}{)}^{y-\mathrm{2005}}}}\cdot \mathrm{\Delta }y\cdot V_{y,\mathrm{reg}}\cdot \ln \left({\displaystyle \frac{{\mathit{\chi }}_{y,\mathrm{reg}}}{{\mathrm{\Gamma }}_{y,\mathrm{reg}}}}\right),$$

where regional consumption is χ_y,reg at model time y and the weight of the consumption is determined by the pure rate of time preference ϱ_reg and population V_y,reg. The consumption χ_y,reg at time y is in turn equal to the difference between regional income (gross domestic product – GDP) minus export (which is not available for consumption) and savings (i.e., investments), subtracted by the cost of the energy system (including the power sector) and other costs in the economy. For simplicity we do not discuss several other expenditures, such as capital investment for energy services, other energy-related expenditures such as R&D and innovation, taxes, cost of pollution and land use change.

Appendix D: Deriving the soft-coupling convergence conditions

In Sect. 3.2.1, we sketch the derivation procedure and offer a short summary of the analytical results. Here we describe the derivation procedure of the coupled convergence framework in detail.

Using the Lagrangian multiplier method, based on the objective functions (Eqs. 1–2) and constraints (c1–c6) in Sect. 3.1, we can construct the KKT Lagrangians (Karush, 1939; Kuhn and Tucker, 1951; Gan et al., 2013). REMIND:

$$\begin{array}{}\text{(D1)}& \begin{array}{rl}\mathcal{L}& =\underset{\text{REMIND objective function}}{\underbrace{\sum _{y,s}\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right)}}\\ & +\underset{\text{annual electricity balance equation constraint}}{\underbrace{\sum _y{\mathit{\lambda }}_y\left[d_y-\sum _s{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})\right]}}\\ & +\underset{\text{resource constraint}}{\underbrace{\sum _{y,s}{\mathit{\omega }}_{y,s}\left(P_{y,s}-{\mathit{\psi }}_s\right)}}+\underset{\text{positive generation constraint}}{\underbrace{\sum _{y,s}{\mathit{\xi }}_{y,s}\left(-G_{y,s}\right)}}\\ & +\underset{\text{maximum generation from capacity constraint}}{\underbrace{\sum _{y,s}{\mathit{\mu }}_{y,s}\left(G_{y,s}-\mathrm{8760}\cdot {\mathit{\varphi }}_{y,s}{P}_{y,s}\right)}}\\ & +\underset{\text{standing capacity constraint}}{\underbrace{\sum _{y\le \mathrm{2020},s}{\mathit{\sigma }}_{y,s}\left(p_{y,s}-P_{y,s}\right)}}\\ & +\underset{\text{near-term ramp-up capacity constraint}}{\underbrace{\sum _{y=\mathrm{2025},s}{\mathit{\gamma }}_{y,s}\left(P_{y,s}-P_{y-\mathrm{\Delta }y,s}-q_{y,s}\right)}}.\end{array}\end{array}$$

DIETER:

$$\begin{array}{}\text{(D2)}& \begin{array}{rl}\stackrel{\mathrm{‾}}{\mathcal{L}}& =\underset{\text{DIETER objective function}}{\underbrace{\sum _s\left[{\stackrel{\mathrm{‾}}{c}}_s{\stackrel{\mathrm{‾}}{P}}_s+{\stackrel{\mathrm{‾}}{o}}_s\sum _h\left({\stackrel{\mathrm{‾}}{G}}_{h,s}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}\right)\right]}}\\ & +\underset{\text{hourly electricity balance equation constraint}}{\underbrace{\sum _h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h\left({\stackrel{\mathrm{‾}}{d}}_h-\sum _s{\stackrel{\mathrm{‾}}{G}}_{h,s}\right)}}\\ & +\underset{\text{resource constraint}}{\underbrace{\sum _s{\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s\left({\stackrel{\mathrm{‾}}{P}}_s-{\stackrel{\mathrm{‾}}{\mathit{\psi }}}_s\right)}}+\underset{\text{positive generation constraint}}{\underbrace{\sum _{h,s}{\stackrel{\mathrm{‾}}{\mathit{\xi }}}_{h,s}\left(-{\stackrel{\mathrm{‾}}{G}}_{h,s}\right)}}\\ & +\underset{\text{maximum dispatchable generation from capacity constraint}}{\underbrace{\sum _{h,\mathrm{dis}}{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{dis}}\left({\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{dis}}-{\stackrel{\mathrm{‾}}{P}}_{\mathrm{dis}}\right)}}\\ & +\underset{\text{maximum renewable generation from capacity and weather constraints}}{\underbrace{\sum _{h,\mathrm{vre}}{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{vre}}\left({\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{vre}}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}-{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,\mathrm{vre}}{\stackrel{\mathrm{‾}}{P}}_{\mathrm{vre}}\right)}}.\end{array}\end{array}$$

Comparing Lagrangians ℒ and $\stackrel{\mathrm{‾}}{\mathcal{L}}$, there are notable similarities between the terms. However, first, we can reduce the complexity by noticing that there are terms containing capacity shadow prices that are either trivial or already harmonized: resource constraint shadow prices ω are already identical for both models by design (constraint c2 in Sect. 3.1). Positive generation constraint shadow price ξ is 0 due to KKT conditions for both models (constraint c3). These constraint terms can be safely excluded from the subsequent mapping. We then note the important fact that the REMIND Lagrangian is a sum over multiple years, whereas the DIETER Lagrangian is for each year. To make a direct comparison and therefore mapping possible, we assume that the brown-field and near-term constraints are not binding. After this simplifying assumption, we realize that REMIND becomes linearly independent in terms of the temporal slices because by now the only yet-to-be-harmonized constraints left in the standalone models are (c1) and (c4), which are both constraints for each year and do not result in temporal correlations. Note that this simplifying assumption is assumed to be valid only for the derivation in this section. Later, in actual simulations, we see that these bounds generate shadow prices which are not necessarily small, impacting the degree of convergence, especially in earlier years. These constraints are also temporally localized in early periods, exerting little impact on later, more “green-field” years. In fact, when including the brown-field constraint in DIETER (c8), the model convergence is improved (Sect. 6.1).

After the aforementioned simplifications, we can construct a single-year REMIND Lagrangian ℒ_y,

$$\begin{array}{}\text{(D3)}& \begin{array}{rl}{\mathcal{L}}_y& =\underset{\text{REMIND objective function}}{\underbrace{\sum _s\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right)}}\\ & +\underset{\text{annual electricity balance equation constraint}}{\underbrace{{\mathit{\lambda }}_y\left[d_y-\sum _s{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})\right]}}\\ & +\underset{\text{maximum generation from capacity constraint}}{\underbrace{\sum _s{\mathit{\mu }}_{y,s}\left(G_{y,s}-\mathrm{8760}\cdot {\mathit{\varphi }}_{y,s}{P}_{y,s}\right)}},\end{array}\end{array}$$

and map it to the single-year DIETER Lagrangian $\stackrel{\mathrm{‾}}{\mathcal{L}}$:

$$\begin{array}{}\text{(D4)}& \begin{array}{rl}\stackrel{\mathrm{‾}}{\mathcal{L}}& =\underset{\text{DIETER objective function}}{\underbrace{\sum _s\left[{\stackrel{\mathrm{‾}}{c}}_s{\stackrel{\mathrm{‾}}{P}}_s+{\stackrel{\mathrm{‾}}{o}}_s\sum _h\left({\stackrel{\mathrm{‾}}{G}}_{h,s}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}\right)\right]}}\\ & +\underset{\text{hourly electricity balance equation constraint}}{\underbrace{\sum _h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h\left({\stackrel{\mathrm{‾}}{d}}_h-\sum _s{\stackrel{\mathrm{‾}}{G}}_{h,s}\right)}}\\ & +\underset{\text{maximum dispatchable generation from capacity constraint}}{\underbrace{\sum _{h,\mathrm{dis}}{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{dis}}\left({\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{dis}}-{\stackrel{\mathrm{‾}}{P}}_{\mathrm{dis}}\right)}}\\ & +\underset{\text{maximum renewable generation from capacity and weather constraints}}{\underbrace{\sum _{h,\mathrm{vre}}{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{vre}}\left({\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{vre}}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}-{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,\mathrm{vre}}{\stackrel{\mathrm{‾}}{P}}_{\mathrm{vre}}\right)}}.\end{array}\end{array}$$

These are the same as Eqs. (3)–(4).

Comparing ℒ_y and $\stackrel{\mathrm{‾}}{\mathcal{L}}$, we can map them by matching the following four terms in the Lagrangians individually:

• A.

  annual total power sector costs: $Z_y={\sum }_s\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right)$ and $\stackrel{\mathrm{‾}}{Z}={\sum }_s\left[{\stackrel{\mathrm{‾}}{c}}_{y,s}{\stackrel{\mathrm{‾}}{P}}_{y,s}+{\stackrel{\mathrm{‾}}{o}}_{y,s}{\sum }_h({\stackrel{\mathrm{‾}}{G}}_{y,h,s}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{y,h,\mathrm{vre}})\right]$;

• B.

  annual revenue of usable (post-curtailment) generation for each generator s: ${\mathit{\lambda }}_y{G}_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)$ and ${\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}$;

• C.

  annual payment made by the consumers: λ_yd_y and ${\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{d}}_{y,h}$; and

• D.

  maximum generation from the capacity constraint term for each generator s: ${\mathit{\mu }}_{y,s}\left(G_{y,s}-\mathrm{8760}\cdot {\mathit{\varphi }}_{y,s}{P}_{y,s}\right)$ and ${\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{y,h,s}\left({\stackrel{\mathrm{‾}}{G}}_{y,h,s}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{y,h,s}-{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,s}{\stackrel{\mathrm{‾}}{P}}_{y,s}\right)$ (here we write the two terms for VRE and dispatchable into one term for DIETER for simplicity: i.e., ${\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{y,h,\mathrm{dis}}=\mathrm{0}$ and ${\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,\mathrm{dis}}=\mathrm{1}$ for dispatchables).

The following conditions (h1–h7) can be derived from the harmonization of terms (A)–(D). Each term is harmonized by matching the values in front of decision variables at the aggregated levels, i.e., capacities and annual generations.

Term (A) can be mapped if

• h1.

  annual fixed costs are harmonized for each generator species s: $c_{y,s}={\stackrel{\mathrm{‾}}{c}}_{y,s}$; and

• h2.

  annual variable costs are harmonized for each generator species s: $o_{y,s}={\stackrel{\mathrm{‾}}{o}}_{y,s}$.

Term (B) can be mapped if,

• h3.

  for each generator species s, the annual average revenue per unit generation, i.e., the market value, is harmonized by exogenously manipulating the market value in REMIND to be the same as the last-iteration annual average market value in DIETER. We achieve this by adding a correction term, thereby modifying the REMIND original objective function Z to Z^′:

  $$Z^{\prime }=Z-{\sum }_{y,s}{\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}\left(i-\mathrm{1}\right)G_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right),$$

  where ${\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}\left(i-\mathrm{1}\right)$ is the markup for technology s in DIETER in the last iteration i−1, and i is the index of the iteration of the iterative soft-coupling. Z^′ is the modified REMIND objective function in the coupled version.

  The detailed derivation is as follows.

  Lagrangian term (B) for the models has the physical meaning of the total annual revenue of usable (post-curtailment) generation. (Annual revenue is equal to the product of usable generation and the annual market value.) We denote total annual revenue from technology s as Θ_y,s for REMIND and ${\stackrel{\mathrm{‾}}{\mathrm{\Theta }}}_{y,s}$ for DIETER. Then, for REMIND, the revenue (term B) is

  $$\begin{array}{}\text{(D5)}& {\mathrm{\Theta }}_{y,s}={\mathit{\lambda }}_y{G}_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right),\end{array}$$

  and for DIETER it is

  $$\begin{array}{}\text{(D6)}& {\stackrel{\mathrm{‾}}{\mathrm{\Theta }}}_{y,s}={\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}.\end{array}$$

  To harmonize terms Θ_y,s and ${\stackrel{\mathrm{‾}}{\mathrm{\Theta }}}_{y,s}$, our goal is to create a one-to-one mapping of the values in front of the decision variable's annually aggregated post-curtailment generation of technology s, which is $G_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)$ for REMIND and ${\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}$ for DIETER. The latter is namely a direct sum of the hourly generations. However, we notice for DIETER revenue that ${\stackrel{\mathrm{‾}}{\mathrm{\Theta }}}_{y,s}$ is a weighted sum of the hourly generation, and the direct sum cannot be separated in a straightforward way. So, first we have to rewrite ${\stackrel{\mathrm{‾}}{\mathrm{\Theta }}}_{y,s}$ (Eq. D6) by first dividing and then multiplying by the aggregated annual generation:

  $$\begin{array}{}\text{(D7)}& {\stackrel{\mathrm{‾}}{\mathrm{\Theta }}}_{y,s}={\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}}{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}.\end{array}$$

  We notice that the multiplicative term in front of the DIETER annually aggregated generation ${\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}$ is $\frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}$, which is nothing other than the market value of generation technology s (see also Eq. F24).

  We now take a look at revenue Θ_y,s on the REMIND side, which is equal to ${\mathit{\lambda }}_y{G}_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)$ (Eq. D5). To map (D5) to the DIETER revenue term ${\stackrel{\mathrm{‾}}{\mathrm{\Theta }}}_{y,s}$ (Eq. D7) in terms of the aggregated decision variables $G_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)$ and ${\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}$, we essentially would like the multiplicative term in front of the generation variable in Θ_y,s, which is λ_y, to also be $\frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}$ like in DIETER. This means that the DIETER-corrected revenue in REMIND should be

  $$\begin{array}{}\text{(D8)}& {\mathrm{\Theta }}_{y,s}^{\prime }={\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}}{G}_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right).\end{array}$$

  To harmonize Θ_y,s and ${\stackrel{\mathrm{‾}}{\mathrm{\Theta }}}_{y,s}$, we can simply add a linear correction term to compensate for the difference between them. Noticing in Eq. (D5) that the multiplicative term in front of the REMIND generation variable $G_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)$ is λ_y, which can be interpreted as the REMIND market value, we realize essentially for a linear correction term that we should add the market value difference ΔMV_y,s between the two models,

  $$\begin{array}{}\text{(D9)}& \mathrm{\Delta }{\mathrm{MV}}_{y,s}={\stackrel{\mathrm{‾}}{\mathrm{MV}}}_s-{\mathrm{MV}}_s={\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}}-{\mathit{\lambda }}_y,\end{array}$$

  to the multiplicative term λ_y in Θ_y,s, so λ_y is canceled. Note that, in Eq. (D9), as discussed before, the DIETER market value is dependent on technology index s, whereas the REMIND one is not.

  After adding the linear correction term, the modified revenue in REMIND ${\mathrm{\Theta }}_{y,s}^{\prime }$ after harmonization is

  $$\begin{array}{}\text{(D10)}& \begin{array}{rl}{\mathrm{\Theta }}_{y,s}^{\prime }& ={\mathrm{\Theta }}_{y,s}+\mathrm{\Delta }{\mathrm{MV}}_{y,s}{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})\\ & =(\mathrm{\Delta }{\mathrm{MV}}_{y,s}+{\mathit{\lambda }}_y)G_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s}).\end{array}\end{array}$$

  Plugging in Eq. (D9),

  $$\begin{array}{}\text{(D11)}& \begin{array}{rl}{\mathrm{\Theta }}_{y,s}^{\prime }& =\left({\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}}-{\mathit{\lambda }}_y+{\mathit{\lambda }}_y\right)G_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)\\ & ={\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}}{G}_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right),\end{array}\end{array}$$

  which is as desired in Eq. (D8).

  In practice, in the case of the annual shadow price λ_y in REMIND, we find that the coupling is more stable numerically if we use the annual average electricity price of DIETER instead of the last-iteration electricity price of REMIND λ_y in Eq. (D9). The equivalence between the two prices is expressed later in (h5). We can use this substitution, since we show later that (h5) can be derived from market value harmonization (h3) and demand harmonization (h4). With this substitution, the correction term, which we call ${\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}$, is in fact

  $$\begin{array}{}\text{(D12)}& \begin{array}{rl}{\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}& ={\stackrel{\mathrm{‾}}{\mathrm{MV}}}_s-\stackrel{\mathrm{‾}}{J}={\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}}\\ & -{\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{d}}_{y,h}}{{\sum }_h{\stackrel{\mathrm{‾}}{d}}_{y,h}}},\end{array}\end{array}$$

  where $\stackrel{\mathrm{‾}}{J}=\frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{d}}_{y,h}}{{\sum }_h{\stackrel{\mathrm{‾}}{d}}_{y,h}}$ is the annual average electricity price in DIETER. We calculate Eq. (D12) using the last-iteration DIETER solutions. Note that, compared to the earlier Eq. (D9), we have simply replaced the second-term REMIND annual price with the DIETER annual price.

  It is not hard to recognize ${\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}$ as the “markup” for technology s in DIETER, where markup as defined before is the difference between the market value of a technology ${\stackrel{\mathrm{‾}}{\mathrm{MV}}}_s$ and the load-weighted annual average electricity price $\stackrel{\mathrm{‾}}{J}$ (see the introduction of Sect. 3.1).

  Now we have concluded the derivation for the markup term ${\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}$ in (h3).

  Although the multiplicative terms in front of the decision variables in the two models can be harmonized via the correction term (D12), we notice that it contains endogenous values, i.e., hourly generation ${\stackrel{\mathrm{‾}}{G}}_{y,h,s}$ and hourly shadow price ${\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}$ in DIETER. Since any endogenous value can only be known ex post, this means that the Lagrangian mapping relies on endogenous values from the last iteration, i.e.,

  $$\begin{array}{rl}{\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}\left(i-\mathrm{1}\right)& ={\stackrel{\mathrm{‾}}{\mathrm{MV}}}_s\left(i-\mathrm{1}\right)-\stackrel{\mathrm{‾}}{J}\left(i-\mathrm{1}\right)\\ & ={\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}\left(i-\mathrm{1}\right){\stackrel{\mathrm{‾}}{G}}_{y,h,s}\left(i-\mathrm{1}\right)}{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}\left(i-\mathrm{1}\right)}}\\ & -{\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}\left(i-\mathrm{1}\right){\stackrel{\mathrm{‾}}{d}}_{y,h}\left(i-\mathrm{1}\right)}{{\sum }_h{\stackrel{\mathrm{‾}}{d}}_{y,h}\left(i-\mathrm{1}\right)}}.\end{array}$$

  Now, using the markup term ${\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}$, we define the linear correction term for the revenue in REMIND Θ_y,s as

  $$\mathrm{\Delta }{\mathrm{\Theta }}_{y,s}={\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}\left(i-\mathrm{1}\right)G_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right).$$

  The physical meaning of ΔΘ_y,s is the revenue difference in the two models for technology s, given that the post-curtailment generations are expressed in terms of REMIND variables.

  The coupled REMIND has a modified objective function Z^′ based on a linear correction. The correction term ΔΘ_y,s needs to be summed over s and y and subtracted – due to the negative sign in front of term (B) – from the REMIND objective function Z, since the objective term as a part of the Lagrangian can be directly manipulated:

  $$\begin{array}{rl}{Z}^{\prime }& =Z-M=Z-{\sum }_{y,s}\mathrm{\Delta }{\mathrm{\Theta }}_{y,s}\\ & =Z-{\sum }_{y,s}{\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}\left(i-\mathrm{1}\right)G_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right),\end{array}$$

  where we call the total system revenue differences M; again, these are revenues where the post-curtailment generations are expressed in terms of REMIND variables (and not DIETER variables).

  Now we have concluded the derivation for the convergence condition (h3).

  Depending on the starting point of the REMIND power system, and due to the internal iterative changes in REMIND results due to the adjustments in trade between regions in the Nash algorithm, coupled convergence usually can only be achieved over multiple iterations. Therefore the derived markup equation (Eq. D12) in general can only be expected to reflect the actual market value differences approximately in the two models. This is the reason why, in the iterative algorithm after the first iteration, we add $M\left(i\right)-M\left(i-\mathrm{1}\right)$ to the objective function Z, as the quantities and prices gradually converge between the two models. As convergence is approached, the total revenue difference between iterations $M\left(i\right)-M\left(i-\mathrm{1}\right)$ should go to zero. This is confirmed by the numerical experiments (not shown).

Term (C) can be mapped if

• h4.

  annual power demands in the two models are harmonized: $d_y={\sum }_h{\stackrel{\mathrm{‾}}{d}}_{y,h}$; and

• h5.

  annual average prices of electricity are mapped to each other: ${\mathit{\lambda }}_y=\frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{d}}_{y,h}}{{\sum }_h{\stackrel{\mathrm{‾}}{d}}_{y,h}}$ (dividing term C by h4). Because electricity price is by definition equal to the total annual system revenue divided by the total annual demand, (h5) can be shown to hold true given that technology-specific revenues are harmonized in (h3) and demands are harmonized in (h4). (If technology-specific revenues are harmonized in (h3), then the system revenues which are technology-specific revenues summed over technologies are also harmonized.) Condition (h5) can therefore be seen as a derived condition from (h3) and (h4).

Term (D) can be mapped if

• h6.

  annual average capacity factors are harmonized, i.e., ϕ_y,s in REMIND is set equal to the endogenous last-iteration DIETER result for each generation type s:

  $${\mathit{\varphi }}_{y,s}={\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,s}/\mathrm{8760},$$

  where ${\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,s}=\frac{{\stackrel{\mathrm{‾}}{G}}_{y,h,s}}{{\stackrel{\mathrm{‾}}{P}}_{y,s}}$ is the hourly capacity factor in DIETER. Without explicit manipulation of the shadow prices μ_y,s and ${\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{y,h,s}$, we show that the following claim is true: through the above capacity factor harmonization, the terms containing endogenous shadow prices will be automatically mapped. Showing this requires careful mathematical argument, which we make in detail in the case of dispatchables, and later we argue that the case is similar for renewables.

  For dispatchable generators the argument is as follows. (For simplicity, we use the generic index s.)

  We first rewrite REMIND term (D) by plugging in the harmonization condition ${\mathit{\varphi }}_{y,s}={\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,s}/\mathrm{8760}$:

  $$\begin{array}{rl}& {\mathit{\mu }}_{y,s}\left(G_{y,s}-\mathrm{8760}\cdot {\mathit{\varphi }}_{y,s}{P}_{y,s}\right)\\ & ={\sum }_y{\mathit{\mu }}_{y,s}\left(G_{y,s}-{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,s}{P}_{y,s}\right).\end{array}$$

  It should be mapped to the term ${\sum }_{y,h}{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{y,h,s}\left({\stackrel{\mathrm{‾}}{G}}_{y,h,s}-{\stackrel{\mathrm{‾}}{P}}_{y,s}\right)$ in DIETER.

  Splitting the two terms, these four terms need to be harmonized for all y and s.

  $$\begin{array}{}\text{(D13)}& {\displaystyle }{\mathit{\mu }}_{y,s}{G}_{y,s}\text{ and }{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{y,h,s}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}\text{(D14)}& {\displaystyle }{\mathit{\mu }}_{y,s}{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,s}{P}_{y,s}\text{ and }{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{y,h,s}{\stackrel{\mathrm{‾}}{P}}_{y,s}\end{array}$$

  To show that the mapping Eqs. (D13)–(D14) are automatically satisfied given (h6), we first consider two simplified power sector toy problems, Q1 and Q2, with only dispatchable technologies. Both problems have identical objective functions $\stackrel{\mathrm{̃}}{Z}={\sum }_s\left({\stackrel{\mathrm{̃}}{c}}_s{\stackrel{\mathrm{̃}}{P}}_s+{\stackrel{\mathrm{̃}}{o}}_s{\stackrel{\mathrm{̃}}{G}}_s\right)$, and the fixed and variable cost parameters ${\stackrel{\mathrm{̃}}{c}}_s$ and ${\stackrel{\mathrm{̃}}{o}}_s$ are identical. Both problems have identical hourly balance equation constraints but with two different kinds of maximum generation constraints: Q1 has an inequality constraint for each hour, and Q2 has an aggregated annual equality constraint.

  Q1: minimum Z, such that ${\stackrel{\mathrm{̃}}{G}}_{h,s}\le {\stackrel{\mathrm{̃}}{P}}_s⟂{\stackrel{\mathrm{̃}}{\mathit{\mu }}}_{h,s}$, ${\stackrel{\mathrm{̃}}{d}}_h={\sum }_s{\stackrel{\mathrm{̃}}{G}}_{h,s}⟂{\stackrel{\mathrm{̃}}{\mathit{\lambda }}}_h$

  Q2: minimum Z, such that ${\sum }_h{\stackrel{\mathrm{̃}}{G}}_{h,s}=\mathrm{8760}\cdot {\stackrel{\mathrm{̃}}{\mathit{\varphi }}}_s{\stackrel{\mathrm{̃}}{P}}_s⟂{{\stackrel{\mathrm{̃}}{\mathit{\mu }}}^{\prime }}_s$, ${\stackrel{\mathrm{̃}}{d}}_h={\sum }_s{\stackrel{\mathrm{̃}}{G}}_{h,s}⟂{{\stackrel{\mathrm{̃}}{\mathit{\lambda }}}^{\prime }}_h$

  Then the Lagrangians are the following:

  $$\begin{array}{rl}{\stackrel{\mathrm{̃}}{\mathcal{L}}}_{\mathrm{1}}& =\underset{\text{objective function}}{\underbrace{\sum _s\left({\stackrel{\mathrm{̃}}{c}}_s{\stackrel{\mathrm{̃}}{P}}_s+{\stackrel{\mathrm{̃}}{o}}_s\sum _h{\stackrel{\mathrm{̃}}{G}}_{h,s}\right)}}\\ & +\underset{\text{hourly electricity balance equation constraint}}{\underbrace{\sum _h{\stackrel{\mathrm{̃}}{\mathit{\lambda }}}_h\left(\stackrel{\mathrm{̃}}{d_h}-\sum _s{\stackrel{\mathrm{̃}}{G}}_{h,s}\right)}}\\ & +\underset{\text{maximum generation from capacity constraint}}{\underbrace{\sum _{h,s}{\stackrel{\mathrm{̃}}{\mathit{\mu }}}_{h,s}\left({\stackrel{\mathrm{̃}}{G}}_{h,s}-{\stackrel{\mathrm{̃}}{P}}_s\right)}};\end{array}$$
  $$\begin{array}{rl}{\stackrel{\mathrm{̃}}{\mathcal{L}}}_{\mathrm{2}}& =\underset{\text{objective function}}{\underbrace{\sum _s\left({\stackrel{\mathrm{̃}}{c}}_s{\stackrel{\mathrm{̃}}{P}}_s+{\stackrel{\mathrm{̃}}{o}}_s\sum _h{\stackrel{\mathrm{̃}}{G}}_{h,s}\right)}}\\ & +\underset{\text{hourly electricity balance equation constraint}}{\underbrace{\sum _h{\stackrel{\mathrm{̃}}{\mathit{\lambda }}}_h^{\prime }\left({\stackrel{\mathrm{̃}}{d}}_h-\sum _s{\stackrel{\mathrm{̃}}{G}}_{h,s}\right)}}\\ & +\underset{\text{maximum generation from capacity constraint}}{\underbrace{\sum _s{\stackrel{\mathrm{̃}}{\mathit{\mu }}}_s^{\prime }\left(\sum _h{\stackrel{\mathrm{̃}}{G}}_{h,s}-\mathrm{8760}{\stackrel{\mathrm{̃}}{\mathit{\varphi }}}_s{\stackrel{\mathrm{̃}}{P}}_s\right)}}.\end{array}$$

  The relevant KKT conditions are as follows.

  Stationarity condition for Q1:

  $$\begin{array}{}\text{(D15)}& {\displaystyle \frac{\partial {\stackrel{\mathrm{̃}}{\mathcal{L}}}_{\mathrm{1}}}{\partial {\stackrel{\mathrm{̃}}{P}}_s}}={\stackrel{\mathrm{̃}}{c}}_s-{\sum }_h{\stackrel{\mathrm{̃}}{\mathit{\mu }}}_{h,s}=\mathrm{0}.\end{array}$$

  Stationarity condition for Q2:

  $$\begin{array}{}\text{(D16)}& {\displaystyle \frac{\partial {\stackrel{\mathrm{̃}}{\mathcal{L}}}_{\mathrm{2}}}{\partial {\stackrel{\mathrm{̃}}{P}}_s}}={\stackrel{\mathrm{̃}}{c}}_s-\mathrm{8760}{\stackrel{\mathrm{̃}}{\mathit{\varphi }}}_s{{\stackrel{\mathrm{̃}}{\mathit{\mu }}}^{\prime }}_s=\mathrm{0}.\end{array}$$

  Since the fixed cost ${\stackrel{\mathrm{̃}}{c}}_s$ is equal for the two models, from Eqs. (D15) to (D16) we can derive the relation between the two shadow prices:

  $$\begin{array}{}\text{(D17)}& \mathrm{8760}\cdot {\stackrel{\mathrm{̃}}{\mathit{\varphi }}}_s{{\stackrel{\mathrm{̃}}{\mathit{\mu }}}^{\prime }}_s={\sum }_{\mathrm{h}}{\stackrel{\mathrm{̃}}{\mathit{\mu }}}_{h,s}.\end{array}$$

  Note that, for the toy models, the identical balance equation constraints do not contain capacity P, which is why the balance equation constraints do not influence the stationary conditions for P (Eqs. D15–D16).

  We now show that Eq. (D14) is automatically mapped given capacity factor harmonization (h6). We first write the equality condition for the REMIND–DIETER case, analogous to the toy model result (Eq. D17):

  $$\begin{array}{}\text{(D18)}& \mathrm{8760}\cdot {\mathit{\varphi }}_{y,s}{\mathit{\mu }}_{y,s}={\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{y,h,s}.\end{array}$$

  Note that we can apply the toy model case to the REMIND–DIETER coupling case in a rather straightforward way because, in the case of REMIND–DIETER, the objective function terms have already been harmonized by (h1)–(h2) and the balance equation constraint terms do not contain P, so they have no bearing on the generation-capacity constraint term, just like in the case of the toy models.

  Plugging (h6) ${\mathit{\varphi }}_{y,s}={\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,s}\left(i-\mathrm{1}\right)/\mathrm{8760}$ into Eq. (D18), we have derived the equality for the parameter mapping required in Eq. (D14), i.e.,

  $${\mathit{\mu }}_{y,s}{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,s}\left(i-\mathrm{1}\right)={\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{y,h,s}.$$

  To show Eq. (D13), we first use the hourly capacity factor from DIETER,

  $$\begin{array}{}\text{(D19)}& {\stackrel{\mathrm{‾}}{G}}_{y,h,s}={\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,s}{\stackrel{\mathrm{‾}}{P}}_{y,s},\end{array}$$

  as well as the primal feasibility condition from REMIND $G_{y,s}=\mathrm{8760}\cdot {\mathit{\varphi }}_{y,s}{P}_{y,s}$ (Eq. F9), to rewrite both sides of the mapping in Eq. (D13) in capacity terms. For REMIND, plugging in Eq. (F9),

  $$\begin{array}{}\text{(D20)}& {\mathit{\mu }}_{y,s}{G}_{y,s}={\mathit{\mu }}_{y,s}\cdot \mathrm{8760}\cdot {\mathit{\varphi }}_{y,s}{P}_{y,s},\end{array}$$

  and for DIETER, plugging in Eq. (D19),

  $$\begin{array}{}\text{(D21)}& {\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{y,h,s}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}={\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{y,h,s}{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,s}{\stackrel{\mathrm{‾}}{P}}_{y,s}.\end{array}$$

  Take the complementary slackness condition of DIETER ${\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,s}({\stackrel{\mathrm{‾}}{G}}_{h,s}-{\stackrel{\mathrm{‾}}{P}}_s)=\mathrm{0}$ (Eq. F16) and insert Eq. (D19) into the left-hand side. We obtain

  $${\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,s}({\stackrel{\mathrm{‾}}{G}}_{h,s}-{\stackrel{\mathrm{‾}}{P}}_s)={\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,s}({\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,s}{\stackrel{\mathrm{‾}}{P}}_{y,s}-{\stackrel{\mathrm{‾}}{P}}_s)=\mathrm{0}.$$

  Rearranging, we get

  $$\begin{array}{}\text{(D22)}& {\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{y,h,s}{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,s}{\stackrel{\mathrm{‾}}{P}}_s={\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{y,h,s}{\stackrel{\mathrm{‾}}{P}}_s\end{array}$$

  for each hour h.

  Plug Eq. (D22) and then Eq. (D18) into the right-hand side of Eq. (D21) to obtain

  $$\begin{array}{}\text{(D23)}& \begin{array}{rl}& {\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{y,h,s}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}={\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{y,h,s}{\stackrel{\mathrm{‾}}{P}}_{y,s}\\ & =\mathrm{8760}\cdot {\mathit{\varphi }}_{y,s}{\mathit{\mu }}_{y,s}{\stackrel{\mathrm{‾}}{P}}_{y,s}.\end{array}\end{array}$$

  Compare Eq. (D20) with Eq. (D23). They now have identical parameters in front of the capacity variables P_y,s and ${\stackrel{\mathrm{‾}}{P}}_{y,s}$, as desired. We concluded the proof by exogenously setting the annual capacity factor of REMIND to that of the last-iteration DIETER. We automatically harmonize the generation capacity constraint term of the Lagrangian in the case of dispatchable generators.

• h7.

  For VREs, annual curtailment rates are harmonized, $G_{y,\mathrm{vre}}{\mathit{\alpha }}_{y,\mathrm{vre}}={\sum }_h{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{y,h,\mathrm{vre}}$, i.e., by exogenously setting the curtailment rate in REMIND ${\mathit{\alpha }}_{y,\mathrm{vre}}={\sum }_h{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{y,h,\mathrm{vre}}\left(i-\mathrm{1}\right)/G_{y,\mathrm{vre}}$ and taking the endogenously determined curtailed power ${\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{y,h,\mathrm{vre}}$ from the last-iteration DIETER. This in general also harmonizes terms other than term (D), as it harmonizes the definition for the generation variable in DIETER that is post-curtailment and the REMIND definition for a generation variable that is pre-curtailment.

  For VREs the derivation is conceptually similar to the above case for dispatchables in (h6), since we can define a real capacity factor (post-curtailment) similar to the capacity factor for the dispatchable generators above:

  $${\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{y,h,\mathrm{vre}}={\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{vre}}/{\stackrel{\mathrm{‾}}{P}}_{\mathrm{vre}}.$$

  Due to the limitations of this paper, we will not present the derivation here. A detailed derivation is available upon request.

Appendix E: Coupling iteration schematics

Coupled region: Germany

Coupled REMIND time horizon: 2020–2150 (years 2010–2015 are not coupled since they are historical years and mostly have hard-fixed quantities).

Figure E1 A graphic description of the model iterative coupling. (a) Graphic illustration of the bidirectional coupling in the temporal dimension. The temporal slices of REMIND which are mapped to multiple parallel year-long DIETER problems are illustrated here. The convergence conditions are iteratively mapped at the interface. (b) Graphic illustration of the bidirectional coupling in the iteration dimension. Every ith iteration of REMIND takes the (i−1)th iteration of REMIND as a starting point for optimization and the endogenous output of the (i−1)th DIETER as exogenous input parameters. When the convergence conditions are met, i.e., REMIND satisfies its internal convergence condition, and the coupled models differ in their generation share of each technology by at most a certain percentage (e.g., 5 % for the baseline run without storage), the coupled run halts.

Under a simple configuration (no storage, no flexible demand), every REMIND run takes around 3 min and a DIETER run takes a few seconds to solve. Under more detailed configurations (with storage and flexible demand) and climate policies, every REMIND run takes around 4 min and a DIETER run takes a few minutes to solve. The entire REMIND–DIETER coupled run for a single region, Germany, under a simple configuration is around 3–4 h. It is around 6–10 h for the more detailed configurations under climate policies.

Appendix F: Derivation of the equilibrium conditions for the uncoupled REMIND and DIETER

In this Appendix, we discuss the equilibrium conditions of the uncoupled models, resulting in a rigorous formulation of the so-called “zero-profit rules” (ZPRs). We first construct the Lagrangians and compute KKT conditions and then derive the ZPRs for the standalone versions of the REMIND-reduced power sector model and the DIETER model.

Using the objective functions and constraints in Sect. 3.1, we can construct Lagrangians for the two standalone models. Using the KKT conditions derived from the Lagrangians, we can show that, if the historical and resource constraints are non-binding, i.e., shadow prices ω, σ and γ are zero, then each generator would have recovered its fixed cost, variable cost and curtailment cost through its total market revenue; i.e., each producer of electricity gets zero profit given that the profits are defined as the difference between revenue and cost. When the capacity constraints exist and are binding, we arrive at a modified version of the original ZPR, which describes the relation between cost, revenue and the capacity shadow prices.

Here we first construct the Lagrangians and derive the KKT conditions from them (Sect. F1) for both models. Then, both models' ZPRs are derived, two for each model, i.e., the technology-specific ZPR and the system ZPR (Sect. F2).

F1 Lagrangians and KKT conditions

The Lagrangians of the uncoupled model have been constructed in Appendix D (Eqs. D1–D2). From the KKT conditions for minimization, we can ascertain the following first-order conditions at stationarity for each model.

REMIND

1. Stationary conditions:

   $$\begin{array}{}\text{(F1)}& {\displaystyle }\begin{array}{rl}{\displaystyle \frac{\partial \mathcal{L}}{\partial P_{y,s}}}& =\mathrm{0}\Rightarrow c_{y,s}+{\mathit{\omega }}_{y,s}-\mathrm{8760}\cdot {\mathit{\mu }}_{y,s}{\mathit{\varphi }}_{y,s}\\ & -{\mathit{\sigma }}_{y,s}+{\mathit{\gamma }}_{y,s}=\mathrm{0},\end{array}\text{(F2)}& {\displaystyle }\begin{array}{rl}{\displaystyle \frac{\partial \mathcal{L}}{\partial G_{y,s}}}& =\mathrm{0}\Rightarrow o_{y,s}-{\mathit{\lambda }}_y\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)\\ & -{\mathit{\xi }}_{y,s}+{\mathit{\mu }}_{y,s}=\mathrm{0}.\end{array}\end{array}$$

2. Complementary slackness:

   $$\begin{array}{}\text{(F3)}& {\displaystyle }{\mathit{\omega }}_{y,s}\left(P_{y,s}-{\mathit{\psi }}_s\right)=\mathrm{0},\text{(F4)}& {\displaystyle }{\mathit{\xi }}_{y,s}{G}_{y,s}=\mathrm{0},\text{(F5)}& {\displaystyle }{\mathit{\mu }}_{y,s}(G_{y,s}-\mathrm{8760}\cdot {\mathit{\varphi }}_{y,s}{P}_{y,s})=\mathrm{0},\text{(F6)}& {\displaystyle }{\mathit{\sigma }}_{y,s}\left(p_{y,s}-P_{y,s}\right)=\mathrm{0},\left(y\le \mathrm{2020}\right),\text{(F7)}& {\displaystyle }{\mathit{\gamma }}_{y,s}\left(P_{y,s}-P_{y-\mathrm{\Delta }y,s}-q_{y,s}\right)=\mathrm{0},\left(y=\mathrm{2025}\right).\end{array}$$

3. Primal feasibility:

   $$\begin{array}{}\text{(F8)}& {\displaystyle }{d}_y-{\sum }_s{G}_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)=\mathrm{0},\text{(F9)}& {\displaystyle }{G}_{y,s}-\mathrm{8760}\cdot {\mathit{\varphi }}_{y,s}{P}_{y,s}=\mathrm{0}.\end{array}$$

4. Dual feasibility:

   $$\begin{array}{}\text{(F10)}& {\mathit{\xi }}_{y,s}\ge \mathrm{0},{\mathit{\omega }}_{y,s}\ge \mathrm{0},{\mathit{\sigma }}_{y,s}\ge \mathrm{0},{\mathit{\gamma }}_{y,s}\ge \mathrm{0}.\end{array}$$

DIETER

1. Stationary conditions:

   $$\begin{array}{}\text{(F11)}& {\displaystyle }\begin{array}{rl}& {\displaystyle \frac{\partial \mathcal{L}}{\partial {\stackrel{\mathrm{‾}}{P}}_s}}=\mathrm{0}\Rightarrow {\stackrel{\mathrm{‾}}{c}}_s+{\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s-{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,s}{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,s}=\mathrm{0},\\ & {\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,s}=\mathrm{1}\text{for dispatchables},\\ & \mathrm{0}<{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,s}<\mathrm{1}\text{ for renewables,}\end{array}\text{(F12)}& {\displaystyle \frac{\partial \stackrel{\mathrm{‾}}{\mathcal{L}}}{\partial {\stackrel{\mathrm{‾}}{G}}_{h,s}}}=\mathrm{0}\Rightarrow {\stackrel{\mathrm{‾}}{o}}_s-{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h-{\stackrel{\mathrm{‾}}{\mathit{\xi }}}_{h,s}+{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,s}=\mathrm{0},\text{(F13)}& {\displaystyle \frac{\partial \stackrel{\mathrm{‾}}{\mathcal{L}}}{\partial {\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}}}=\mathrm{0}\Rightarrow {\stackrel{\mathrm{‾}}{o}}_{\mathrm{vre}}+{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{vre}}=\mathrm{0}.\end{array}$$

2. Complementary slackness:

   $$\begin{array}{}\text{(F14)}& {\displaystyle }{\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s\left({\stackrel{\mathrm{‾}}{P}}_s-{\stackrel{\mathrm{‾}}{\mathit{\psi }}}_s\right)=\mathrm{0},\text{(F15)}& {\displaystyle }{\stackrel{\mathrm{‾}}{\mathit{\xi }}}_{h,s}{\stackrel{\mathrm{‾}}{G}}_{h,s}=\mathrm{0},\text{(F16)}& {\displaystyle }{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{dis}}({\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{dis}}-{\stackrel{\mathrm{‾}}{P}}_{\mathrm{dis}})=\mathrm{0}.\end{array}$$

3. Primal feasibility:

   $$\begin{array}{}\text{(F17)}& {\displaystyle }{\stackrel{\mathrm{‾}}{d}}_h={\sum }_s{\stackrel{\mathrm{‾}}{G}}_{h,s},\text{(F18)}& {\displaystyle }{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{vre}}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}={\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,\mathrm{vre}}{\stackrel{\mathrm{‾}}{P}}_{\mathrm{vre}}.\end{array}$$

4. Dual feasibility:

   $$\begin{array}{}\text{(F19)}& {\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s\ge \mathrm{0},{\stackrel{\mathrm{‾}}{\mathit{\xi }}}_{h,s}\ge \mathrm{0},{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{dis}}\ge \mathrm{0}.\end{array}$$

F2 Derivation of the zero-profit rules

F2.1 REMIND

The derivation of ZPRs is very similar to the one in Brown and Reichenberg (2021). Starting with the total costs for technology s for all years and applying various KKT conditions (after “|”),

$$\begin{array}{c}{\displaystyle }\begin{array}{rl}& {\sum }_y(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s})\\ & ={\sum }_y\left\{\left(-{\mathit{\omega }}_{y,s}+\mathrm{8760}\cdot {\mathit{\mu }}_{y,s}{\mathit{\varphi }}_{y,s}+{\mathit{\sigma }}_{y,s}-{\mathit{\gamma }}_{y,s}\right)P_{y,s}\right.\\ & \left.+\left[{\mathit{\lambda }}_y\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)+{\mathit{\xi }}_{y,s}-{\mathit{\mu }}_{y,s}\right]G_{y,s}\right\}|\text{(F1), (F2)}\end{array}\\ {\displaystyle }\begin{array}{rl}=& {\sum }_y\left\{\left(-{\mathit{\omega }}_{y,s}+\mathrm{8760}\cdot {\mathit{\mu }}_{y,s}{\mathit{\varphi }}_{y,s}+{\mathit{\sigma }}_{y,s}-{\mathit{\gamma }}_{y,s}\right)P_{y,s}\right.\\ & \left.+\left[{\mathit{\lambda }}_y\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)-{\mathit{\mu }}_{y,s}\right]G_{y,s}\right\}|\text{(F4)}\end{array}\\ {\displaystyle }={\sum }_y\left\{\left(-{\mathit{\omega }}_{y,s}+{\mathit{\sigma }}_{y,s}-{\mathit{\gamma }}_{y,s}\right)P_{y,s}+{\mathit{\lambda }}_y{G}_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)\right\}.|\text{(F5)}\end{array}$$

Rearranging, we arrive at the ZPR of the multi-year uncoupled REMIND for the technology cost–revenue balance.

$$\begin{array}{}\text{(F20)}& \begin{array}{rl}& \underset{{\text{generation cost}}_s}{\underbrace{\sum _y\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right)}}\\ & =-\underset{{\text{capacity shadow revenue}}_s}{\underbrace{\sum _y\left({\mathit{\omega }}_{y,s}-{\mathit{\sigma }}_{y,s}+{\mathit{\gamma }}_{y,s}\right)P_{y,s}}}\\ & +\underset{{\text{generation revenue}}_s}{\underbrace{\sum _y{\mathit{\lambda }}_y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}\end{array}\end{array}$$

Normally when there are no capacity shadow prices or when the capacity constraints are not binding, the cost exactly equals revenue. However, when capacity shadow prices are non-zero, i.e., the constraints (c2) and (c5–c6) are binding, the capacity shadow prices act as a distortion of the equality relation between costs and revenues. As an example, the shadow price ω_y,s from limited generation resources (e.g., hydroelectric power in Germany) would be positive, ${\mathit{\omega }}_{y,s}>\mathrm{0}$, when the constraint is binding and would appear as a “positive cost” or a “negative revenue” in the modeled power market. We can therefore put it either on the left-hand (cost) or right-hand (revenue) side of the equation. Here we group it together with revenues.

One observes that, from the right-hand side of Eq. (F20), there is no differentiation between the annual market values of variable and dispatchable generations such as gas and solar – they are both equal to the annual electricity price λ_y.

From Eq. (F20), we can derive a ZPR between the levelized cost of electricity (LCOE), capacity shadow price and market value (MV) for each generator type. Taking Eq. (F20), we separate the pre-curtailment LCOE from the LCOE due to curtailment and then divide it by the total post-curtailment generation ${\sum }_y{G}_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)$ for the generator type s to obtain the technology-specific ZPR:

$$\begin{array}{}\text{(F21)}& \begin{array}{rl}& \underset{{\text{pre-curtailment LCOE}}_s}{\underbrace{{\displaystyle \frac{\sum _y\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right)}{\sum _y{G}_{y,s}}}}}\\ & +\underset{{\text{curtailment LCOE}}_s}{\underbrace{{\displaystyle \frac{\sum _y\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right){\mathit{\alpha }}_{y,s}}{\sum _y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}\\ & =-\underset{{\text{capacity shadow price}}_s}{\underbrace{{\displaystyle \frac{\sum _y\left({\mathit{\omega }}_{y,s}-{\mathit{\sigma }}_{y,s}+{\mathit{\gamma }}_{y,s}\right)P_{y,s}}{\sum _y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}\\ & +\underset{{\text{market value}}_s}{\underbrace{{\displaystyle \frac{\sum _y{\mathit{\lambda }}_y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}{\sum _y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}.\end{array}\end{array}$$

The pre-curtailment LCOE is the cost of one unit of generated electricity, regardless of whether it is curtailed or being used to meet demand, whereas the curtailment LCOE is the cost of one unit of curtailed electricity. Together they add up to the post-curtailment LCOE, i.e., the cost of one unit of usable electricity.

To obtain the ZPR for the whole power system in REMIND, we first sum Eq. (F20) over all generator types s and obtain the ZPR for system cost and revenue. Then, dividing by the total post-curtailment system generation and splitting the LCOE into pre-curtailment and curtailment components, we get

$$\begin{array}{}\text{(F22)}& \begin{array}{rl}& \underset{{\text{pre-curtailment LCOE}}_{\text{system}}}{\underbrace{{\displaystyle \frac{\sum _{y,s}\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right)}{\sum _{y,s}{G}_{y,s}}}}}\\ & +\underset{{\text{curtailment LCOE}}_{\text{system}}}{\underbrace{{\displaystyle \frac{\sum _{y,s}\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right){\mathit{\alpha }}_{y,s}}{\sum _{y,s}{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}\\ & =-\underset{{\text{capacity shadow price}}_{\text{system}}}{\underbrace{{\displaystyle \frac{\sum _{y,s}\left({\mathit{\omega }}_{y,s}-{\mathit{\sigma }}_{y,s}+{\mathit{\gamma }}_{y,s}\right)P_{y,s}}{\sum _{y,s}{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}\\ & +\underset{{\text{electricity price}}_{\text{system}}}{\underbrace{{\displaystyle \frac{\sum _{y,s}{\mathit{\lambda }}_y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}{\sum _{y,s}{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}.\end{array}\end{array}$$

In other words, the LCOE of the system for usable (pre-curtailment) power, which is equal to the sum of the system LCOE for the total power generated and the curtailment cost, can be recovered by the average electricity price of the system minus the system-wide capacity constraint shadow price per energy unit.

The ZPRs of REMIND hold for the aggregate over multiple years.

From Eqs. (F21) and (F22), we learn that when a market equilibrium can be found, i.e., when the optimization problem can be successfully solved, there is an equality relation between the generation cost and market value for each generator type and similarly between the generation cost and price of electricity for the entire system. Capacity shadow prices due to various extra capacity constraints imposed on the models distort the equality relation between costs and prices by a linear term, making the prices either higher or lower than the costs at the market equilibrium.

F2.2 DIETER

Similarly to the uncoupled REMIND, from KKT conditions, at stationarity, we can obtain the cost–revenue ZPR for a single technology s for the standalone DIETER. We take the total costs for technology s for all years and, applying various KKT conditions (after “|”),

$$\begin{array}{c}{\displaystyle }\begin{array}{rl}& {\stackrel{\mathrm{‾}}{c}}_s{\stackrel{\mathrm{‾}}{P}}_s+{\sum }_h\left[{\stackrel{\mathrm{‾}}{o}}_s\left({\stackrel{\mathrm{‾}}{G}}_{h,s}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}\right)\right]\\ & =\left(-{\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s+{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,s}{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,s}\right){\stackrel{\mathrm{‾}}{P}}_s\\ & +{\sum }_h\left({\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h-{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,s}+{\stackrel{\mathrm{‾}}{\mathit{\xi }}}_{h,s}\right)\left({\stackrel{\mathrm{‾}}{G}}_{h,s}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}\right)|\text{(F11), (F12)}\\ & =-{\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s{\stackrel{\mathrm{‾}}{P}}_s+{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,\mathrm{vre}}{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{vre}}{\stackrel{\mathrm{‾}}{P}}_{\mathrm{vre}}\\ & +{\sum }_h\left({\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h-{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{vre}}+{\stackrel{\mathrm{‾}}{\mathit{\xi }}}_{h,\mathrm{vre}}\right)\left({\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{vre}}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}\right)\\ & +{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{dis}}{\stackrel{\mathrm{‾}}{P}}_{\mathrm{dis}}+{\sum }_h\left({\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h-{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{dis}}\right){\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{dis}},\\ & |\text{split}{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,s}{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,s}\text{into vre and dis and apply Eq. (F15)}\\ & \text{ for dispatchables, i.e.,}{\stackrel{\mathrm{‾}}{\mathit{\xi }}}_{h,\mathrm{dis}}{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{dis}}=\mathrm{0}\end{array}\\ {\displaystyle }\begin{array}{rl}& =-{\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s{\stackrel{\mathrm{‾}}{P}}_s+{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{h,\mathrm{vre}}{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{vre}}{\stackrel{\mathrm{‾}}{P}}_{\mathrm{vre}}\\ & +{\sum }_h\left({\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h-{\stackrel{\mathrm{‾}}{\mathit{\mu }}}_{h,\mathrm{vre}}+{\stackrel{\mathrm{‾}}{\mathit{\xi }}}_{h,\mathrm{vre}}\right)\left({\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{vre}}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}\right)\\ & +{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{dis}}|\text{(F16)}\end{array}\\ {\displaystyle }\begin{array}{rl}& =-{\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s{\stackrel{\mathrm{‾}}{P}}_s+{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{vre}}+{\sum }_h\left({\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h+{\stackrel{\mathrm{‾}}{\mathit{\xi }}}_{h,\mathrm{vre}}\right){\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}\\ & +{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{dis}}\\ & |\text{(F18), and apply (F15) for VRE, i.e.,}{\stackrel{\mathrm{‾}}{\mathit{\xi }}}_{h,\mathrm{vre}}{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{vre}}=\mathrm{0}\end{array}\\ {\displaystyle }\begin{array}{rl}& =-{\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s{\stackrel{\mathrm{‾}}{P}}_s+{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{vre}}+{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{dis}}.\\ & |\text{(F12) and (F13)}\Rightarrow {\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h+{\stackrel{\mathrm{‾}}{\mathit{\xi }}}_{h,\mathrm{vre}}=\mathrm{0}\end{array}\end{array}$$

Rearranging, we arrive at the ZPR of the single-year uncoupled DIETER for a technology-specific cost–revenue balance:

$$\begin{array}{}\text{(F23)}& \begin{array}{rl}& \underset{\stackrel{\mathrm{‾}}{{\text{annual generation cost}}_s}}{\underbrace{{\stackrel{\mathrm{‾}}{c}}_s{\stackrel{\mathrm{‾}}{P}}_s+{\stackrel{\mathrm{‾}}{o}}_s\sum _h({\stackrel{\mathrm{‾}}{G}}_{h,s}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}})}}\\ & =-\underset{\stackrel{\mathrm{‾}}{{\text{annual capacity shadow revenue}}_s}}{\underbrace{{\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s{\stackrel{\mathrm{‾}}{P}}_s}}\\ & +\underset{\stackrel{\mathrm{‾}}{{\text{annual generation revenue}}_s}}{\underbrace{\sum _h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h{\stackrel{\mathrm{‾}}{G}}_{h,s}}}.\end{array}\end{array}$$

Dividing Eq. (F23) by the annually aggregated generation of technology s, we obtain the technology-specific ZPR for DIETER:

$$\begin{array}{}\text{(F24)}& \begin{array}{rl}& \underset{\stackrel{\mathrm{‾}}{{\text{LCOE}}_s}}{\underbrace{{\displaystyle \frac{{\stackrel{\mathrm{‾}}{c}}_s{\stackrel{\mathrm{‾}}{P}}_s+{\stackrel{\mathrm{‾}}{o}}_s\sum _h({\stackrel{\mathrm{‾}}{G}}_{h,s}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}})}{\sum _h{\stackrel{\mathrm{‾}}{G}}_{h,s}}}}}\\ & =-\underset{\stackrel{\mathrm{‾}}{{\text{annual capacity shadow price}}_s}}{\underbrace{{\displaystyle \frac{{\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s{\stackrel{\mathrm{‾}}{P}}_s}{\sum _h{\stackrel{\mathrm{‾}}{G}}_{h,s}}}}}\\ & +\underset{\stackrel{\mathrm{‾}}{{\text{market value}}_s}}{\underbrace{{\displaystyle \frac{\sum _h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h{\stackrel{\mathrm{‾}}{G}}_{h,s}}{\sum _h{\stackrel{\mathrm{‾}}{G}}_{h,s}}}}}.\end{array}\end{array}$$

One observes that, from the term of ${\stackrel{\mathrm{‾}}{\text{market value}}}_s$, compared to the REMIND case (right-hand side of Eq. F21), DIETER has differentiated annual market values of gas and solar generators.

Summing Eq. (F24) over s, dividing both sides by the total annual generation ${\sum }_{h,s}{\stackrel{\mathrm{‾}}{G}}_{h,s}$ and using the identity ${\stackrel{\mathrm{‾}}{d}}_h={\sum }_s{\stackrel{\mathrm{‾}}{G}}_{h,s}$ for simplification, we obtain the ZPR for the whole power system in DIETER:

$$\begin{array}{}\text{(F25)}& \begin{array}{rl}& \underset{\stackrel{\mathrm{‾}}{{\text{LCOE}}_{\mathrm{system}}}}{\underbrace{{\displaystyle \frac{\sum _s[{\stackrel{\mathrm{‾}}{c}}_s{\stackrel{\mathrm{‾}}{P}}_s+{\stackrel{\mathrm{‾}}{o}}_s{\sum }_h({\stackrel{\mathrm{‾}}{G}}_{h,s}+{\stackrel{\mathrm{‾}}{\mathrm{\Gamma }}}_{h,\mathrm{vre}}\left)\right]}{\sum _{h,s}{\stackrel{\mathrm{‾}}{G}}_{h,s}}}}}\\ & =-\underset{\stackrel{\mathrm{‾}}{{\text{annual capacity shadow price}}_{\mathrm{system}}}}{\underbrace{{\displaystyle \frac{\sum _s{\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s\stackrel{\mathrm{‾}}{P_s}}{\sum _{h,s}{\stackrel{\mathrm{‾}}{G}}_{h,s}}}}}\\ & +\underset{\stackrel{\mathrm{‾}}{{\text{annual average electricity price}}_{\mathrm{system}}}}{\underbrace{{\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h{\stackrel{\mathrm{‾}}{d}}_h}{\sum _h{\stackrel{\mathrm{‾}}{d}}_h}}}}.\end{array}\end{array}$$

Similarly to the case of REMIND, Eqs. (F24)–(F25) show us the equality relations between the cost and value (or price) for each generator type and for the system hold (also for DIETER at its market equilibrium). Compared to REMIND, there are no brown-field or near-term capacity shadow price contributions in DIETER in the standalone versions. The DIETER ZPRs hold for 1 year instead of the aggregate of multiple years like in REMIND. For simplicity, even though it is possible to write the LCOE in pre-curtailment and curtailment terms, for DIETER it is relatively cumbersome to do, and we do not do it here.

In summary, at the REMIND and DIETER power market equilibriums, each generator exactly recovers its cost of one unit of generation through the market value and obtains zero profit in a completely competitive market over its modeling time. In the aggregate, the entire power sector obtains its cost of one unit of generation through the price of electricity that the consumer pays. Both types of relations can be distorted by the existence of capacity shadow prices.

Appendix G: Derivation of the equilibrium conditions for the coupled models

Here, in this Appendix, we gradually build up the derivation for the ZPRs of the coupled REMIND and DIETER, which will be used later to validate numerical results. The derivation consists of three steps.

1. ZPRs for the uncoupled models REMIND and DIETER

2. ZPRs for the coupled models REMIND and DIETER (simplified version only considering convergence conditions h1–h7)

3. ZPRs for the coupled models REMIND and DIETER (full version also considering c7 and c8)

Step (1) is entirely derived in Appendix F.

For step (2), based on the uncoupled ZPRs, we recognize that, from convergence conditions (h1)–(h7), the only condition which impacts the form of the ZPR is (h3) because the markup terms modify the objective function of the (simplified) coupled version of REMIND (Eq. 6). Following a similar procedure to Appendix F, we can derive the technology-specific ZPR for the coupled REMIND (simplified version) as follows:

$$\begin{array}{}\text{(G1)}& \begin{array}{rl}& \underset{{\text{pre-curtailment LCOE}}_s}{\underbrace{{\displaystyle \frac{\sum _y\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right)}{\sum _y{G}_{y,s}}}}}\\ & +\underset{{\text{curtailment cost}}_s}{\underbrace{{\displaystyle \frac{\sum _y\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right){\mathit{\alpha }}_{y,s}}{\sum _y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}\\ & =-\underset{{\text{capacity shadow price}}_s}{\underbrace{{\displaystyle \frac{\sum _y\left({\mathit{\omega }}_{y,s}-{\mathit{\sigma }}_{y,s}+{\mathit{\gamma }}_{y,s}\right)P_{y,s}}{\sum _y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}\\ & +\underset{{\text{market value}}_s}{\underbrace{{\displaystyle \frac{\sum _y({\mathit{\lambda }}_y+{\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s})G_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}{\sum _y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}.\end{array}\end{array}$$

Compared with the ZPR of the uncoupled version Eq. (F24), the only difference is that we replace the market value in the uncoupled REMIND λ_y with the DIETER markup-corrected market value ${\mathit{\lambda }}_y+{\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}$. DIETER's ZPR is unchanged at this step.

Step (3) involves two extra capacity constraints, one in each model, the first of which, (c7), is discussed in detail in Appendix H. The implementation of (c7) further modifies Eq. (G1) and results in the ZPRs of the coupled REMIND. The other constraint, (c8), will be the focus of discussion here. It only modifies the ZPRs for the coupled DIETER and not for the coupled REMIND.

Constraint (c8) is a brown-field capacity constraint implemented in DIETER to address the fact that DIETER is a green-field model that is otherwise ignorant of standing capacities in the real world. There are many ways in which we can implement this standing capacity constraint in DIETER. The most straightforward way is to implement the standing capacity at the beginning of each REMIND period, which REMIND sees before it invests additional capacities, as a lower bound on endogenous capacities in DIETER. This helps put DIETER and REMIND on an equal footing before the 5- or 10-year investment period starts, allowing us to compare their investment intentions.

• c8.

  Standing capacity constraint in DIETER: i.e., DIETER capacities at time y need to be larger than or equal to the REMIND standing capacities at the beginning of the time period.

  $${\stackrel{\mathrm{‾}}{P}}_s\ge P_{y-\mathrm{\Delta }y/\mathrm{2},s}/\left(\mathrm{1}-\mathrm{ER}\right)⟂{\stackrel{\mathrm{‾}}{\mathit{\zeta }}}_s,$$

  where the time step Δy is divided by 2 because the representative year in REMIND is in the middle of the time step. ER is the endogenous early retirement rate in REMIND.

The reason we implement the standing capacity in this way is in part because, as a proof of concept, we want to give DIETER endogenous freedom to invest in all model years, so we use only the pre-investment capacities as “soft” corridors to bound the DIETER capacities from below. If we were to transfer precisely the brown-field and near-term constraints from REMIND to DIETER, this would require a complete list of constraints for each technology and an identical implementation of all of them in DIETER. This may raise the precision of convergence in some years for some technologies, but in practice it can be more complicated to implement than a generic lower bound for all technologies.

To obtain the ZPRs of the coupled DIETER, we simply modify the capacity shadow price term of the uncoupled DIETER ZPRs (Eqs. F24–F25) by the additional capacity shadow price ${\stackrel{\mathrm{‾}}{\mathit{\zeta }}}_s$ from (c8):

$$\begin{array}{}\text{(G2)}& {\displaystyle }{\stackrel{\mathrm{‾}}{{\text{capacity shadow price}}^{\prime }}}_s={\displaystyle \frac{\left({\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s+{\stackrel{\mathrm{‾}}{\mathit{\zeta }}}_s\right){\stackrel{\mathrm{‾}}{P}}_s}{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{h,s}}},\text{(G3)}& {\displaystyle }{\stackrel{\mathrm{‾}}{{\text{capacity shadow price}}^{\prime }}}_{\mathrm{system}}={\displaystyle \frac{{\sum }_s\left({\stackrel{\mathrm{‾}}{\mathit{\omega }}}_s+{\stackrel{\mathrm{‾}}{\mathit{\zeta }}}_s\right){\stackrel{\mathrm{‾}}{P}}_s}{{\sum }_{h,s}{\stackrel{\mathrm{‾}}{G}}_{h,s}}}.\end{array}$$

Appendix H: Additional methods for numerical stability in coupled runs

Here, we introduce the two methods we employed to improve the numerical stability of the coupled runs: (1) the dispatchable capacity constraint by peak demand to avoid high markups being exchanged (Sect. H1) and (2) endogenous prefactors for all quantities from the last-iteration DIETER to the current-iteration REMIND (Sect. H2).

H1 Dispatchable capacity constraints by peak demand

H1.1 Description of the capacity constraint and price manipulation in DIETER postprocessing

The scarcity hour price can occur in a PSM run, which is the highest hourly price in a year, and it is usually equal to the annuitized fixed cost of OCGTs (capital investment cost and fixed O&M costs) (Hirth and Ueckerdt, 2013). In our simulations, the scarcity prices are usually above USD 50 per kilowatt hour. If we include the scarcity price in the markups, OCGTs will receive an annual markup usually more than 5 times higher than the annual average electricity price. The high markup results in OCGT plants receiving too high an incentive in the next-iteration REMIND, and the model overshoots (overinvests) in capacities. Over iterations, this causes oscillations in the quantity and prices in the coupled model. For better numerical stability, instead of passing on the full markups from DIETER, we only pass on the portion of the annual markups unrelated to scarcity hour prices and replace the exchange of the part of the markup due to scarcity hours from DIETER to REMIND by implementing an additional capacity constraint in REMIND for coupled runs. The two actions can later be shown to be mathematically equivalent. Generators other than OCGTs which produce at the scarcity hours also get paid in the hour at this high price. However, because they also produce at other hours with lower prices, their average market values are only moderately impacted by the scarcity hour price and do not in general lead to instability issues.

Below, we first introduce the aforementioned capacity constraint implemented on the side of REMIND and then discuss the corresponding manipulation of the markups in DIETER. Lastly, we show their mathematical equivalence and state the modified ZPR of the coupled REMIND due to these actions.

The extra capacity constraint states that the sum of all dispatchable capacities needs to be at least as large as the peak residual demand:

• c7.
  $${\sum }_{\mathrm{dis}}{P}_{y,\mathrm{dis}}>d_{y,\mathrm{residual}}⟂{\mathit{\upsilon }}_{y,\mathrm{dis}},$$

  where d_y,residual is the peak residual demand in REMIND and is semi-endogenous. d_y,residual is a function of the peak hourly residual demand in the last iteration of DIETER ${\stackrel{\mathrm{‾}}{d}}_{\mathrm{residual}}\left(y,i-\mathrm{1}\right)$. The peak hourly residual demand in DIETER is in turn defined as the maximum hourly amount of inflexible demand not met by wind, solar or hydro generations and hence must be met by dispatchable generations (under no storage conditions):

  $$\begin{array}{}\text{(H1)}& {\stackrel{\mathrm{‾}}{d}}_{\mathrm{residual}}=\underset{h}{max}\left({\stackrel{\mathrm{‾}}{d}}_h-{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{Solar}}-{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{Wind}}-{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{Hydro}}\right).\end{array}$$

  υ_y,dis is the shadow price of the capacity constraint for dispatchable technology dis.

  For the exact implementation of (c7) in the coupled run, see Sects. 3.3.2, point 2. Under storage implementation, in addition to the variable renewable contribution, the hourly storage discharge is also subtracted from the residual demand.

At the same time as implementing this capacity constraint, we remove the surplus scarcity prices in postprocessing of DIETER before passing it on to REMIND. In DIETER, we define the scarcity price as the maximum hourly price in a year,

$$\begin{array}{}\text{(H2)}& {\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h_{\mathrm{scar}}}=\underset{h}{max}\left({\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}\right),\end{array}$$

and the surplus scarcity hour price is the difference between the scarcity price and the second-highest price:

$$\begin{array}{}\text{(H3)}& \begin{array}{rl}{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,\mathrm{surplus}}& ={\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h_{\mathrm{scar}}}-max\left({\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h\mathrm{|}h\ne h_{\mathrm{scar}}}\right)\\ & =\underset{h}{max}\left({\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}\right)-max\left({\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h\mathrm{|}h\ne h_{\mathrm{scar}}}\right),\end{array}\end{array}$$

where h_scar is the scarcity hour when the scarcity price occurs, corresponding to the peak residual demand hour.

Using this, we manipulate the market value and the annual average electricity price in DIETER ex post, excluding the surplus scarcity hour price:

$$\begin{array}{}\text{(H4)}& {\displaystyle }\begin{array}{rl}& {{\stackrel{\mathrm{‾}}{\mathrm{MV}}}^{\prime }}_s=\\ & {\displaystyle \frac{{\sum }_{h\mathrm{|}h\ne h_{\mathrm{scar}}}{\stackrel{\mathrm{‾}}{G}}_{h,s}{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h+{\sum }_{h\mathrm{|}{h}_{\mathrm{scar}}}{\stackrel{\mathrm{‾}}{G}}_{h,s}\cdot max\left({\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{h\mathrm{|}h\ne h_{\mathrm{scar}}}\right)}{{\sum }_{h=\mathrm{1}}^{\mathrm{8760}}{\stackrel{\mathrm{‾}}{G}}_{h,s}}},\end{array}\text{(H5)}& {\displaystyle }{\stackrel{\mathrm{‾}}{J}}^{\prime }={\displaystyle \frac{{\sum }_{h\mathrm{|}h\ne h_{\mathrm{scar}}}{\stackrel{\mathrm{‾}}{d}}_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h+{\sum }_{h\mathrm{|}{h}_{\mathrm{scar}}}{\stackrel{\mathrm{‾}}{d}}_h\cdot max\left({\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{h\mathrm{|}h\ne h_{\mathrm{scar}}}\right)}{{\sum }_{h=\mathrm{1}}^{\mathrm{8760}}{\stackrel{\mathrm{‾}}{d}}_h}},\end{array}$$

where ${\stackrel{\mathrm{‾}}{\mathrm{MV}}}_s^{\prime }$ is the annual average market value without the surplus scarcity hour price and ${\stackrel{\mathrm{‾}}{J}}^{\prime }$ is the annual average electricity price without the surplus scarcity hour price. Thus, the corresponding modified markup term without the surplus scarcity hour price is

$$\begin{array}{}\text{(H6)}& {\stackrel{\mathrm{‾}}{\mathit{\eta }}}_s^{\prime }={\stackrel{\mathrm{‾}}{\mathrm{MV}}}_s^{\prime }-{\stackrel{\mathrm{‾}}{J}}^{\prime }.\end{array}$$

Note that, since the above manipulation is done in a postprocessing step, the LCOE in DIETER is still fully covered by MV, as the KKT conditions and ZPRs still hold by default in an optimized DIETER model.

With the implementation of (c7), the coupled ZPR (Eq. G1) is then further modified to include the new shadow price υ_y,s and the modified markup ${\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}^{\prime }$ (without the surplus scarcity price). (From now on we write υ_y,dis simply as υ_y,s.) Then, the technology-specific ZPR of the coupled REMIND is

$$\begin{array}{}\text{(H7)}& \begin{array}{rl}& \underset{{\text{pre-curtailment LCOE}}_s}{\underbrace{{\displaystyle \frac{\sum _y\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right)}{\sum _y{G}_{y,s}}}}}\\ & +\underset{{\text{curtailment LCOE}}_s}{\underbrace{{\displaystyle \frac{\sum _y\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right){\mathit{\alpha }}_{y,s}}{\sum _y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}\\ & =-\underset{{\text{capacity shadow price}}_s^{\prime }}{\underbrace{{\displaystyle \frac{\sum _y\left({\mathit{\omega }}_{y,s}-{\mathit{\sigma }}_{y,s}+{\mathit{\gamma }}_{y,s}+{\mathit{\nu }}_{y,s}\right)P_{y,s}}{\sum _y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}\\ & +\underset{{\text{market value}}_s^{\prime }}{\underbrace{{\displaystyle \frac{\sum _y({\mathit{\lambda }}_y+{\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}^{\prime })G_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}{\sum _y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}.\end{array}\end{array}$$

The system ZPR of the coupled REMIND is

$$\begin{array}{}\text{(H8)}& \begin{array}{rl}& \underset{{\text{pre-curtailment LCOE}}_{\mathrm{system}}}{\underbrace{{\displaystyle \frac{\sum _{y,s}\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right)}{\sum _{y,s}{G}_{y,s}}}}}\\ & +\underset{{\text{curtailment cost}}_{\mathrm{system}}}{\underbrace{{\displaystyle \frac{\sum _{y,s}\left(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}\right){\mathit{\alpha }}_{y,s}}{\sum _{y,s}{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}\\ & =-\underset{{\text{capacity shadow price}}_{\mathrm{system}}^{\prime }}{\underbrace{{\displaystyle \frac{\sum _{y,s}\left({\mathit{\omega }}_{y,s}-{\mathit{\sigma }}_{y,s}+{\mathit{\gamma }}_{y,s}+{\mathit{\nu }}_{y,s}\right)P_{y,s}}{\sum _{y,s}{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}\\ & +\underset{{\text{electricity price}}_{\mathrm{system}}^{\prime }}{\underbrace{{\displaystyle \frac{\sum _{y,s}({\mathit{\lambda }}_y+{\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}^{\prime })G_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}{\sum _{y,s}{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}}}.\end{array}\end{array}$$

These are the ZPRs of the coupled REMIND for the full version.

H1.2 Equivalence between the surplus scarcity price in DIETER and the capacity shadow price due to the peak residual demand in REMIND

Because of the intuitive relation between the scarcity price and the peak residual demand, i.e., that the scarcity price occurs in the hour with peak hourly residual demand due to the pricing power of the peaker gas turbines in the hour where VREs are most scarce, we can make a quantitative equivalence between the scarcity price contribution to the markup and the capacity constraint shadow price υ_y. This means that the revenue the plant receives in the scarcity hour in capacity terms (i.e., capacity credit) can be transformed directly to a revenue in energy terms (i.e., a part of the annual market value). At convergence, for any given year y, the negative shadow price, $-{\mathit{\upsilon }}_{y,\mathrm{dis}}$, when translated into annual generation terms via a capacity factor ϕ_y,s of dispatchable technology s, should be equal to the scarcity hour surplus revenue divided by annual generation by s in DIETER:

$$\begin{array}{}\text{(H9)}& {\displaystyle \frac{-{\mathit{\upsilon }}_{y,\mathrm{dis}}}{{\mathit{\varphi }}_{y,\mathrm{dis}}\cdot \mathrm{8760}}}={\displaystyle \frac{{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,\mathrm{surplus}}{\stackrel{\mathrm{‾}}{G}}_{h_{\mathrm{scar},\mathrm{dis}}}}{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,\mathrm{dis}}}}.\end{array}$$

In practice, this equivalence is confirmed by numerical results (e.g., the Fig. 8 subplot for OCGTs).

Using this equivalence, we can show as follows that, at convergence, λ_y should be equal to the DIETER power price without the surplus scarcity price ${\stackrel{\mathrm{‾}}{J}}^{\prime }$ (Eq. H5), and ${\mathit{\lambda }}_y+{{\stackrel{\mathrm{‾}}{\mathit{\eta }}}^{\prime }}_{y,s}$ should be equal to the DIETER market value without the scarcity price ${\stackrel{\mathrm{‾}}{\mathrm{MV}}}^{\prime }$ (Eq. H4).

At convergence, the annual generations have identical solutions in the two models, i.e., ${\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,h,s}=G_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)$. We plug this and the REMIND capacity factor ${\mathit{\varphi }}_{y,s}=\frac{G_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)}{P_{y,s}\cdot \mathrm{8760}}$ into Eq. (H9) to obtain

$$\begin{array}{}\text{(H10)}& {\mathit{\upsilon }}_y{P}_{y,s}={\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,\mathrm{surplus}}{\stackrel{\mathrm{‾}}{G}}_{y,h_{\mathrm{scar}},s}.\end{array}$$

Take Eq. (H7) and only consider REMIND annual revenue by multiplying generation ${\sum }_y{G}_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)$. Then, on the right-hand side, take both the revenue and the capacity shadow revenue contribution from υ_y,s for a single year, which is equal to the total single-year REMIND revenue:

$$\begin{array}{rl}{\mathrm{\Theta }}_{y,s}& =-\underset{{\text{capacity shadow revenue from c(7)}}_s}{\underbrace{{\mathit{\nu }}_{y,s}{P}_{y,s}}}\\ & +\underset{{\text{generation revenue}}_s^{\prime }}{\underbrace{({\mathit{\lambda }}_y+{\stackrel{\mathrm{‾}}{\mathit{\eta }}}_{y,s}^{\prime })G_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}.\end{array}$$

Plug in Eqs. (H10) and (H6):

$$\begin{array}{rl}{\mathrm{\Theta }}_{y,s}& =\underset{{\text{surplus scarcity revenue in scarcity hour}}_s}{\underbrace{{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,\mathrm{surplus}}{\stackrel{\mathrm{‾}}{G}}_{y,h_{\mathrm{scar}},s}}}\\ & +\underset{{\text{generation revenue}}_s^{\prime }}{\underbrace{({\stackrel{\mathrm{‾}}{\mathrm{MV}}}_{y,s}^{\prime }-{\stackrel{\mathrm{‾}}{J}}_y^{\prime }+{\mathit{\lambda }}_y)G_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})}}.\end{array}$$

Plugging in Eq. (H4),

$$\begin{array}{rl}{\mathrm{\Theta }}_{y,s}& ={\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,\mathrm{surplus}}{\stackrel{\mathrm{‾}}{G}}_{y,h_{\mathrm{scar}},s}+{\sum }_{h\ne h_{\mathrm{scar}}}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}\\ & +{\stackrel{\mathrm{‾}}{G}}_{y,h_{\mathrm{scar}},s}\cdot \max \left({\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h\mathrm{|}h\ne h_{\mathrm{scar}}}\right)\\ & -{\stackrel{\mathrm{‾}}{J}}_y^{\prime }{G}_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right)+{\mathit{\lambda }}_y{G}_{y,s}\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\right).\end{array}$$

Lastly, plug in the definition for ${\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,\mathrm{surplus}}$ (Eq. H3):

$$\begin{array}{}\text{(H11)}& \begin{array}{rl}{\mathrm{\Theta }}_{y,s}& =\sum _h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}-{\stackrel{\mathrm{‾}}{J}}_y^{\prime }{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s})\\ & +{\mathit{\lambda }}_y{G}_{y,s}(\mathrm{1}-{\mathit{\alpha }}_{y,s}).\end{array}\end{array}$$

Since the single-year revenue Θ_y,s in REMIND should be aligned with DIETER due to harmonization condition (h3) and the DIETER revenue is ${\stackrel{\mathrm{‾}}{\mathrm{\Theta }}}_{y,s}={\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_{y,h}{\stackrel{\mathrm{‾}}{G}}_{y,h,s}$, the last two terms in Eq. (H11) should sum to 0. Therefore, the REMIND electricity price λ_y should be equal to ${\stackrel{\mathrm{‾}}{J}}_y^{\prime }$.

H2 Stabilization techniques using prefactors

In this Appendix, we describe the detailed implementations of prefactors for information exchanged from DIETER to REMIND.

1. Markup prefactor

   In order to facilitate convergence in REMIND, we implement an endogenous prefactor $f_{y,s}^{\mathit{\eta }}$ for MV in the REMIND markup equation Eq. (5):

   $$\begin{array}{}\text{(H12)}& {\mathit{\eta }}_{y,s}\left(i\right)=f_{y,s}^{\mathit{\eta }}\left(i\right)\cdot {\stackrel{\mathrm{‾}}{\mathrm{MV}}}_{y,s}^{\prime }\left(i-\mathrm{1}\right)-{\stackrel{\mathrm{‾}}{J}}_y^{\prime }\left(i-\mathrm{1}\right).\end{array}$$

   The endogenous prefactor $f_{y,s}^{\mathit{\eta }}$ is dependent on the difference between the in-iteration endogenous generation share and the last-iteration DIETER generation share:

   $$\begin{array}{}\text{(H13)}& f_{y,s}^{\mathit{\eta }}\left(i\right)=\mathrm{1}-{\stackrel{\mathrm{‾}}{b}}_{y,s}\left(i-\mathrm{1}\right)\mathrm{\Delta }{S}_{y,s},\end{array}$$

   where ${\stackrel{\mathrm{‾}}{b}}_{y,s}$ is a positive parameter equal to the ratio between market values and the average price, depending on their relationship in the last-iteration DIETER,

   $$\begin{array}{c}{\displaystyle }{\stackrel{\mathrm{‾}}{b}}_{y,s}={\displaystyle \frac{{\stackrel{\mathrm{‾}}{\mathrm{MV}}}_{y,s}^{\prime }}{{\stackrel{\mathrm{‾}}{J}}_y^{\prime }}}\text{if}{\stackrel{\mathrm{‾}}{\mathrm{MV}}}_{y,s}^{\prime }>{\stackrel{\mathrm{‾}}{J}}_y^{\prime },\\ {\displaystyle }{\stackrel{\mathrm{‾}}{b}}_{y,s}={\displaystyle \frac{{\stackrel{\mathrm{‾}}{J}}_y^{\prime }}{{\stackrel{\mathrm{‾}}{\mathrm{MV}}}_{y,s}^{\prime }}}\text{if}{\stackrel{\mathrm{‾}}{\mathrm{MV}}}_{y,s}^{\prime }<{\stackrel{\mathrm{‾}}{J}}_y^{\prime },\end{array}$$

   and where the generation share difference across the models and consecutive iteration ΔS_y,s is

   $$\begin{array}{rl}\mathrm{\Delta }{S}_{y,s}& ={\displaystyle \frac{G_{y,s}\left(i\right)\left(\mathrm{1}-{\mathit{\alpha }}_{y,s}\left(i\right)\right)}{{\sum }_s\left[G_{y,s}\right(i\left)\right(\mathrm{1}-{\mathit{\alpha }}_{y,s}\left(i\right)\left)\right]}}\\ & -{\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{y,s}\left(i-\mathrm{1}\right)}{{\sum }_{h,s}{\stackrel{\mathrm{‾}}{G}}_{y,s}\left(i-\mathrm{1}\right)}}.\end{array}$$

   The values of ${\stackrel{\mathrm{‾}}{b}}_{y,s}$ are heuristically determined (see Sect. 6.2).

   When the in-iteration REMIND solar generation share increases due to the price signal from the last-iteration DIETER market value, such that the REMIND share is larger than in the last DIETER iteration, the formula Eq. (H13) results in a prefactor smaller than 1, decreasing the in-iteration markup η_y,s(i).

2. Peak demand prefactor

   The peak demand in REMIND d_residual,y depends on the last-iteration DIETER peak hourly residual demand ${\stackrel{\mathrm{‾}}{d}}_{\mathrm{residual}}\left(y,i-\mathrm{1}\right)$. Implementing it in constraint (c7),

   $${\sum }_{\mathrm{dis}}{P}_{y,\mathrm{dis}}<d_{\mathrm{residual},y}\cdot f_y^{d_{\mathrm{residual}}}\left(i\right),$$

   for iteration i, we use $f_y^{d_{\mathrm{residual}}}\left(i\right)$ as a prefactor for stabilization:

   $$f_y^{d_{\mathrm{residual}}}\left(i\right)=\mathrm{1}-b_{y,\mathrm{peak}}\cdot \mathrm{\Delta }{S}_{y,\mathrm{wind}}.$$

   b_y,peak is a heuristic constant dependent on y, and ΔS_y,wind is the wind generation share. We use the wind generation share in the current iteration of REMIND for stabilization because, in the peak residual demand hour, there usually is some wind production for the historical year we chose (but no solar). In general, b_y,peak is 0.5 for earlier years and increases to 1 for later years under a baseline scenario. For climate scenarios, b_y,peak is around 1.5 for less stringent scenarios, and for more stringent scenarios, it is 0.5 for earlier years and increases to 3 for later years.

3. Capacity factor prefactor

   We set the REMIND capacity factor ϕ_y,dis to be equal to the DIETER annual average capacity factor from the last iteration multiplied by a prefactor:

   $${\mathit{\varphi }}_{y,\mathrm{dis}}\left(i\right)={\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{\mathrm{dis}}\left(y,i-\mathrm{1}\right)\cdot f_{y,s}^{{\mathit{\varphi }}_{\mathrm{dis}}}\left(i\right),$$

   where the DIETER annual average capacity factor is ${\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{\mathrm{dis}}=\frac{{\sum }_h{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{dis}}}{{\stackrel{\mathrm{‾}}{P}}_{\mathrm{dis}}\cdot \mathrm{8760}}$ for each year y. In order to facilitate convergence, a similar prefactor $f_{y,s}^{{\mathit{\varphi }}_{\mathrm{dis}}}$ to Eq. (H13) is implemented.

   $f_{y,s}^{{\mathit{\varphi }}_{\mathrm{dis}}}\left(i\right)=\mathrm{1}-\mathrm{0.5}\mathrm{\Delta }{S}_{y,s}$ if ${\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{\mathrm{dis}}\left(y,i-\mathrm{1}\right)<\mathrm{0.5}$ (i.e., the plant is a “peaker” or “mid-load” type in the last iteration).

   $f_{y,s}^{{\mathit{\varphi }}_{\mathrm{dis}}}\left(i\right)=\mathrm{1}+\mathrm{0.5}\mathrm{\Delta }{S}_{y,s}$ if ${\stackrel{\mathrm{‾}}{\mathit{\varphi }}}_{\mathrm{dis}}\left(y,i-\mathrm{1}\right)\ge \mathrm{0.5}$ (i.e., the plant is a “base-load” type in the last iteration), where 0.5 is a heuristic factor.

   The sign in the prefactor formula is determined based on the observation that, under a system with variable renewable generations, for generator plants that have relatively high running costs and low investment costs, i.e., most economically operated as “peaker” plants or as “mid-load” plants of a lower capacity factor, when their generation share incrementally increases, their capacity factor decreases. Conversely, for generators with relatively low running costs and high investment costs, i.e., most economically operated as “base-load” plants, when their generation share incrementally increases, their capacity factor increases.

4. Curtailment prefactor

   The curtailment ratio in REMIND α_y,vre is equal to the last-iteration DIETER curtailment ratio multiplied by the prefactor $f_{y,\mathrm{vre}}^{\mathit{\alpha }}$:

   $${\mathit{\alpha }}_{y,\mathrm{vre}}\left(i\right)={\displaystyle \frac{{\sum }_h{\stackrel{\mathrm{‾}}{\mathit{\gamma }}}_{h,\mathrm{vre}}\left(y,i-\mathrm{1}\right)}{{\sum }_{h,s}{\stackrel{\mathrm{‾}}{G}}_{h,\mathrm{vre}}\left(y,i-\mathrm{1}\right)}}\cdot f_{y,\mathrm{vre}}^{\mathit{\alpha }}\left(i\right),$$

   where the prefactor is $f_{y,\mathrm{vre}}^{\mathit{\alpha }}\left(i\right)=\mathrm{1}+\mathrm{\Delta }{S}_{y,\mathrm{vre}}$.

5. Capture price prefactor

   Similar to the case of markup from the demand side, the markup for any demand-side technology given to REMIND is

   $${\mathit{\eta }}_{y,s_d}\left(i\right)=f_{y,s_d}^{\mathit{\eta }}\left(i\right)\cdot {\stackrel{\mathrm{‾}}{\mathrm{CP}}}_{y,s_d}\left(i-\mathrm{1}\right)-{\stackrel{\mathrm{‾}}{J}}_y\left(i-\mathrm{1}\right),$$

   where ${\stackrel{\mathrm{‾}}{J}}_y$ is the annual average electricity price of all demand types s_d for period y,

   $$\stackrel{\mathrm{‾}}{J}={\displaystyle \frac{{\sum }_h\left({\sum }_{s_d}{\stackrel{\mathrm{‾}}{d}}_{h,s_d}\right)\cdot {\stackrel{\mathrm{‾}}{\mathit{\lambda }}}_h}{{\sum }_{h,s_d}{\stackrel{\mathrm{‾}}{d}}_{h,s_d}}},$$

   and $f_{y,s_d}^{\mathit{\eta }}\left(i\right)$ is an endogenous stabilization prefactor for the flexible-demand markup based on shares of demand by s_d in the total demand for each year.

Appendix I: Derivation for the equilibrium condition for REMIND in the case of an additional adjustment cost

Adjustment cost, an additional linear term in the objective function, acts as an inertia against fast or slow capacity additions or retirement. The implementation of positive adjustment costs mimics the challenges of scaling up the supply chains and training new workers to do installation and construction. Adjustment costs are applied to all model time periods, so it is by nature intertemporal. The objective function for the power sector including the adjustment cost Ξ_y,s is

$$Z={\sum }_{y,s}(c_{y,s}{P}_{y,s}+o_{y,s}{G}_{y,s}+{\mathrm{\Xi }}_{y,s}),$$

where Ξ_y,s is a quadratic function of the difference between capacity additions of subsequent time periods y−Δy and y:

$${\mathrm{\Xi }}_{y,s}=c_{y,s}{k}_s{\left({\displaystyle \frac{\mathrm{\Delta }{P}_{y,s}-\mathrm{\Delta }{P}_{y-\mathrm{\Delta }y,s}}{\mathrm{\Delta }{y}^{\mathrm{2}}}}\right)}^{\mathrm{2}}/\left({\displaystyle \frac{\mathrm{\Delta }{P}_{y-\mathrm{\Delta }y,s}}{\mathrm{\Delta }y}}+{\mathit{\beta }}_{y,s}\right),$$

where ΔP_y,s is as before the capacity addition during time period y of technology s, β_y,s is an offset parameter to offset additions in initial time periods, k_s is a regional technological coefficient, and c_y,s is the capital expenditure cost per capacity unit as before. Because the adjustment cost is a quadratic function of the endogenous variable P_y,s, it turns the power sector cost minimization in REMIND into a nonlinear problem.

Similar to the case without adjustment costs in Sect. 3.2.3, the first stationary condition becomes

$$\begin{array}{rl}& {\displaystyle \frac{\partial \mathcal{L}}{\partial P_{y,s}}}=\mathrm{0},\Rightarrow c_{y,s}+{\mathit{\omega }}_{y,s}-{\mathit{\mu }}_{y,s}{\mathit{\varphi }}_{y,s}-{\mathit{\sigma }}_{y,s}+{\mathit{\gamma }}_{y,s}\\ & +\mathrm{2}{c}_{y,s}{k}_s{\displaystyle \frac{\mathrm{\Delta }{P}_{y,s}-\mathrm{\Delta }{P}_{y-\mathrm{\Delta }y,s}}{\left(\mathrm{\Delta }{P}_{y-\mathrm{\Delta }y,s}+{\mathit{\beta }}_{y,s}\right)\mathrm{\Delta }{y}^{\mathrm{2}}}}=\mathrm{0},\end{array}$$

simplifying

$$c_{y,s}=-{\mathit{\omega }}_{y,s}+{\mathit{\mu }}_{y,s}{\mathit{\varphi }}_{y,s}+{\mathit{\sigma }}_{y,s}-{\mathit{\gamma }}_{y,s}-a_{y,s}{c}_{y,s},$$

where $a_{y,s}=\mathrm{2}{k}_s\frac{\mathrm{\Delta }{P}_{y,s}-\mathrm{\Delta }{P}_{y-\mathrm{\Delta }y,s}}{\left(\mathrm{\Delta }{P}_{y-\mathrm{\Delta }y,s}+{\mathit{\beta }}_{y,s}\right)\mathrm{\Delta }{y}^{\mathrm{2}}}$ is the endogenous adjustment factor of investment and is a function of capacity.

The new ZPR including the adjustment cost in terms of cost and revenue for technology s can be derived as

$$\begin{array}{rl}& {\sum }_y\left[\left(c_{y,s}+a_{y,s}{c}_{y,s}\right)P_{y,s}+o_{y,s}{G}_{y,s}\right.\\ & \left.+{\mathit{\lambda }}_y{\mathit{\alpha }}_{y,s}{G}_{y,s}+\left({\mathit{\omega }}_{y,s}-{\mathit{\sigma }}_{y,s}+{\mathit{\gamma }}_{y,s}\right)P_{y,s}\right]\\ & ={\sum }_y\left({\mathit{\lambda }}_y{G}_{y,s}\right).\end{array}$$

The adjustment cost $a_{y,s}{c}_{y,s}$ can act as a disincentive or an incentive for capacity additions. If the capacity addition in the current period is higher than in the last period $\mathrm{\Delta }{P}_{y,s}>\mathrm{\Delta }{P}_{y-\mathrm{\Delta }y,s}$, i.e., a ramp-up case of capacity addition, the adjustment cost is positive and acts as a disincentive, and so the ramp-up speed is slower. When added capacities decrease with time, i.e., a ramp-down case of capacity addition, the adjustment cost is negative and acts as an incentive; as a result, the ramp-down speed is slower.

In the coupled run we see only a moderate adjustment cost which drops down quickly as a function of time (see, e.g., Fig. 6).

Appendix J: Comparing the coupled and uncoupled runs

Figure J1 Under the 2 ^∘C global scenario (no German net-zero goal), we compare (a) the capacity mix and (b) the generation mix of Germany for the DIETER-coupled version of REMIND with endogenous storage (dashed bar) and for the uncoupled version of REMIND with parameterized storage (solid bar). In panel (a), triangle dots indicate the peak residual demand of the year as determined in DIETER.

Appendix K: Complete list of mathematical symbols

The units used in the two models are usually different. Here we uniformly use megawatt hour for energy units and megawatt for capacity units. In the main text, overline $\stackrel{\mathrm{‾}}{.}$ is used to denote DIETER parameters and variables. An apostrophe is used to indicate a modified version of the variable. An asterisk is used to indicate the values of variables at the optimum of objective functions.

Table K1 Complete list of mathematical symbols. For simplicity, in general, we only list the symbols, not their indices or in which model they are used.

Appendix L: Complete list of abbreviations

Table L1 Complete list of abbreviations.