Integrated assessment models (IAMs) are a central tool for the
quantitative analysis of climate change mitigation strategies. However, due
to their global, cross-sectoral and centennial scope, IAMs cannot explicitly
represent the temporal and spatial details required to properly analyze the
key role of variable renewable energy (VRE) in decarbonizing the power
sector and enabling emission reductions through end-use electrification. In
contrast, power sector models (PSMs) can incorporate high spatiotemporal
resolutions but tend to have narrower sectoral and geographic scopes and
shorter time horizons. To overcome these limitations, here we present a
novel methodology: an iterative and fully automated soft-coupling framework
that combines the strengths of a long-term IAM and a detailed PSM. The key
innovation is that the framework uses the market values of power generations
and the capture prices of demand flexibilities in the PSM as price signals that change the capacity and power mix of the IAM. Hence, both
models make endogenous investment decisions, leading to a joint solution. We
apply the method to Germany in a proof-of-concept study using the IAM REgional Model of INvestments and Development (REMIND) v3.0.0 and the PSM Dispatch and Investment Evaluation Tool with Endogenous Renewables (DIETER) v1.0.2 and confirm the theoretical prediction of
almost-full convergence in terms of both decision variables and (shadow)
prices. At the end of the iterative process, the absolute model difference
between the generation shares of any generator type for any year is

Thanks to decade-long policy support in many regions of the world and technological learning, the costs of both wind power and solar photovoltaics have plummeted (IEA, 2021; Lazard, 2021). These types of variable electricity generation are now highly cost-competitive against other alternatives, such that their deployment is increasingly driven by market forces instead of climate policies. Among the newly added renewable generations in 2020, nearly two-thirds were cheaper than the cheapest new fossil fuel (IRENA, 2020). Due to both cost declines and pressing concerns over climate change, investing in these clean and abundant resources has become a crucial part of national and regional strategies to decarbonize the power sector (The White House, 2021; Cherp et al., 2021; National long-term strategies, 2022; Rechsteiner, 2021; ICCSD Tsinghua University, 2022).

Given this dramatic development in the power sector over the past 2 decades, a universal consensus has emerged among energy transition scholars and policy makers: emissions in the power sector are relatively “easy-to-abate” (Luderer et al., 2018; Azevedo et al., 2021; Clarke et al., 2022). Compared with other primarily non-electrified end-use sectors such as buildings, transport and industry, the technologies required to transform the power sector are low-cost, mature and readily available. This trend has in recent years led to a second emerging consensus: the power sector will be the fundamental basis for a future low-cost, efficient and climate-neutral energy system (Brown et al., 2018b; Ram et al., 2018; Ramsebner et al., 2021; Luderer et al., 2022a). In addition to direct electrification, which requires end-use transformations of currently non-electrified demand, emerging technological developments in hydrogen and e-fuels produced from renewable electricity have also contributed to the broadening of potential technology portfolios for the “hard-to-abate” sectors, such as high-temperature heat and chemical productions (Parra et al., 2019; Bhaskar et al., 2020; Griffiths et al., 2021). Together, direct and indirect electrification supports a broad concept of “sector coupling”, which facilitates decarbonization by powering end-use demand with variable renewable energy sources (Ramsebner et al., 2021).

Due to the pivotal role of electrification and sector coupling in mitigation
scenarios, there is an increasing demand on the scope and level of detail of
energy–economy models used to guide energy transition and climate policies. The models would ideally encompass a global, multi-decadal and
multi-sectoral scope such that the scenarios are relevant for international
and regional climate policies while simultaneously incorporating a high level of spatiotemporal detail. The latter is important to account for the
specifics of variable renewable electricity generation as well as its
physical and economic interplay with the electrification of energy demand
(Li and Pye, 2018; Brunner et al., 2020; Prol and Schill, 2021; Böttger
and Härtel, 2022; Ruhnau, 2022). This need for improved modeling methods
or frameworks, which has to overcome the trade-off between scope and detail,
is a substantial methodological challenge. It entails realizing two main
objectives.

Objective (1). Accurately model the power sector transformation over long time
horizons in terms of investment and dispatch, especially at high shares of
variable renewable energy (VRE) sources. Long-term pathways for the
following power sector quantities and prices should accurately incorporate
short-term hourly details:

capacity and generation mix of the power sector;

market values (annual average revenues per power generation unit) for variable and dispatchable plants;

capacity factors of the dispatchable plants and the curtailment rates of variable renewables; and

storage capacity and dispatch.

Objective (2). Accurately model direct electrification of end-use sectors as well as indirect electrification technologies such as green hydrogen production, where existing and emerging sources of power demand can in part be flexibilized.

Current energy system models broadly fall into two distinct categories carried out by two research communities with little institutional overlap: integrated assessment models (IAMs) and power sector models (PSMs), each with their own strengths and weaknesses. IAMs are comprehensive models of a global scale and span multiple decades, linking macroeconomics, energy systems, land use and environmental impacts (Stehfest et al., 2014; Calvin et al., 2017; Huppmann et al., 2019; Baumstark et al., 2021; Keppo et al., 2021; Guivarch et al., 2022), thereby providing an “integrated assessment” of multiple factors (Rotmans and van Asselt, 2001). IAMs substantially shape the IPCC assessments of long-term climate mitigation scenarios and play an important role in policy making (Rogelj et al., 2018; UNEP, 2019; NGFS, 2022; IPCC, 2022). In comparison to IAMs, PSMs typically have narrower spatial and sectoral scopes and shorter time horizons but provide higher resolutions and increased technological detail (Palzer and Henning, 2014; Zerrahn and Schill, 2017; Brown et al., 2018a; Ram et al., 2018; Sepulveda et al., 2018; Blanford and Weissbart, 2019; Böttger and Härtel, 2022; Ringkjøb et al., 2018; Prina et al. 2020). (See also Sect. S5 in the Supplement for a comparison of model specifications of a few selected PSMs.) This allows PSMs to more accurately model the power sector under high-VRE shares (Bistline, 2021; Chang et al., 2021). Note that we use the term “power sector model” here to represent all general smaller-scope models than IAMs (usually by geographical or time horizon measures), even though many of them have sector-coupling aspects and do not only contain the traditional power sector.

IAMs and PSMs are therefore limited by a lack of spatiotemporal detail and a lack of scope, respectively. IAMs usually have a temporal resolution no shorter than a year (Keppo et al., 2021) and therefore include simplified representations of hourly power sector variability that mimic the real-world dynamics to varying degrees of success (Pietzcker et al., 2017). In general, a lack of high temporal resolutions can lead to difficulties when estimating the optimal level of variable renewable generation, often either overestimating or underestimating the market value of solar or wind generation, the challenges of variable renewable integration, the peak hourly residual demand, and the need for energy storage and base load (Pina et al., 2011; Haydt et al., 2011; Ludig et al., 2011; Kannan and Turton, 2013; Welsch et al., 2014; Luderer et al., 2017; Pietzcker et al., 2017; Bistline, 2021). While approximate methods such as parameterization via residual load duration curves (RLDCs) are able to capture the supply-side dynamics of VREs, they remain methodologically limited for representing the flexible demand-side dynamics (Ueckerdt et al., 2015, 2017; Creutzig et al., 2017). Besides limited temporal resolutions, IAMs also usually have coarse spatial resolutions, which can lead to underestimation or overestimation of transmission grid bottlenecks, geographical variability of wind and solar resources, and the flexibility requirements for balancing supply and demand (Aryanpur et al., 2021; Frysztacki et al., 2021; Martínez-Gordón et al., 2021). PSMs, on the other hand, usually lack the global and sectoral scopes required for addressing global climate mitigation, in part because of limited availability of detailed data and due to computational challenges. Furthermore, PSMs with a short-term horizon may lack the vintage tracking of standing capacities, capacity evolution over time and long-term perfect foresight, which can help policy makers and companies to look ahead beyond the short-term business cycles, to invest early and to actively drive technical progress. In contrast, in IAMs such as REMIND, proactive early investment is a built-in feature, because the optimization is done from a long-term social planner's perspective. In IAMs, investing early in the technological learning phase results in lower costs of energy expenditure later, avoiding the severity of punishment to economic growth later in time in the form of lower consumption, which raises the welfare which the model optimizes.

IAMs and PSMs differ in scope and resolution across the three main modeling dimensions: temporal, spatial and technological. A soft-coupling approach can tap into these complementarities and combine their strengths at potentially only a moderate increase in computational cost. The main challenge of the soft-coupling approach is to show that the two models can converge under coupling, which leads to a joint equilibrium that maximizes regional interannual intertemporal welfare in the IAM and minimizes total power system costs in the PSM. Ideally, the converged model offers the “best of both worlds”: it has both the broad scope required to assess global long-term energy transitions and the technical resolution required to capture the interplay between VREs, storage and newly electrified demand on a much shorter timescale.

Approaches aiming to bridge the “temporal resolution gap” between long-term energy system models and hourly PSMs have been proposed in the past (Deane et al., 2012; Sullivan et al., 2013; Alimou et al., 2020; Brinkerink et al., 2020; Seljom et al., 2020; Guo et al., 2022; Younis et al., 2022; Brinkerink et al., 2022; Mowers et al., 2023). While these achieved some aspects of Objective (1) with adequate results, none attempted to incorporate and achieve Objective (2). In addition, there is a methodological gap in the previous attempts at a full harmonization of the multiscale models. By a full harmonization, we mean a comprehensive coupling of the power sector dynamics and an eventual model convergence in capacities, generation and prices. In only a few previous studies, price information has been fed back into the long-term models from the short-term models: in one study only partial price information has been exchanged (Seljom et al., 2020) and in another study some subset of price information is exchanged, but they are not fully endogenized (instead, they are parameterized) and the exchange is also unidirectional (Mowers et al., 2023). Without a feedback mechanism through prices, the investment in the coupled model will very likely be suboptimal due to two effects. (1) Because of the misalignment in prices in the two models, there is a mismatch in investment incentives, resulting in a mismatch for optimal capacities if both models are completely endogenous. (2) In all previous studies, the capacities are fixed in the PSM, and only the long-term model is allowed to invest in new capacities. This implementation can further propagate and sustain the price mismatch due to (1) nontrivial shadow prices from these capacity bounds, in turn creating price distortions in the PSM that can be passed on to the IAM. Therefore, the methodological gap in previous work prevented a comprehensive convergence of the coupled models of both quantities and prices. As we show later in this study, without a comprehensive coupling of price information, no system-wide convergence can be achieved. However, with price coupling as our method proposes, we could achieve all aspects of Objective (1) and Objective (2) for one type of flexible demand with adequate numerical results, thereby representing a first step in bridging the previous methodological gap.

Compared to previous studies, our approach features three main innovations. (1) The coupling is achieved by linking market values and not hard-fixing quantities, allowing both models to invest “as endogenously as possible”. (2) The market values of all power sector technologies are coupled, not just the electricity price of the system or the market value of a particular technology, allowing models to achieve close to full convergence. (3) Under idealized coupling assumptions and for a simplified “proof-of-concept” model without storage, we can mathematically derive the necessary conditions under which comprehensive model convergence can be reached, which puts multiscale coupling on a firm theoretical footing. Our coupling approach is bidirectional, iterative and fully automated.

One should note that our methodology bears certain mathematical similarities to Benders decomposition from the discipline of operation research (Conejo et al., 2006), which is used in the long-term energy system model PRIMES (Price-Induced Market Equilibrium System) to obtain hourly detail (E3Mlab, 2018). There are however crucial differences. For example, the optimization in our work is carried out iteratively outside solver time, whereas the Benders decomposition is carried out iteratively during solver time. In addition, our approach can function even when the objective function is convex, whereas the Benders decomposition cannot, allowing our approach to be applied in more general cases. Mathematically, the subproblems in Benders decomposition have fixed capacities obtained from master problems and therefore are not endogenous, but the shadow prices of these constraints are iteratively passed on to the master problems, ensuring mathematical convergence. The exact ways in which our methodology is connected to Benders decomposition or other similar methods are yet to be fully explored.

To showcase such a framework and its ability to achieve iterative convergence, we couple the PSM Dispatch and Investment Evaluation Tool with Endogenous Renewables (DIETER), which has an hourly resolution (8760 h in a year), with the IAM REMIND for a single region (Germany). Germany is a well-suited case study for exploring high VRE shares in the power sector. The country is expected to meet stringent climate targets despite its high level of residential and industrial power demand, relatively small geographical size, and lack of solar endowment during winter seasons. Nevertheless, the German government has set very ambitious targets for the expansion and use of variable renewable energy sources (Schill et al., 2022). A viable zero-carbon power mix in Germany must include an adequate amount of storage and transmission for renewable generation as well as “clean firm generation” such as geothermal, biomass or gas with carbon capture and storage (CCS) (Sepulveda et al., 2018).

The models used in this study are well-documented open-source models (REMIND is an open-source model but requires proprietary input data to run). A side-by-side comparison of the scope, resolution and other specifications of the two models can be found in Appendix A. The coupling scope can be found in Appendix B. Details on model input data can be found in Sect. S1.

REMIND is a process-based IAM that describes complex global energy–economy–climate interactions (Baumstark et al., 2021). REMIND has frequently been used in long-term planning of decarbonization scenarios, most notably in the IPCC (IPCC, 2014; Rogelj et al., 2018; IPCC, 2022). The REMIND model links different modules that describe the global economy, energy, land and climate systems to a relatively detailed representation of the energy sector compared to non-process-based IAMs. The model is formulated as an interannual intertemporal optimization problem. Due to the computational complexity of nonlinear optimization, the model simulates a time span from 2005 to 2100 with a temporal resolution of either 5 years (between 2005 and 2060) or 10 years (between 2070 and 2100). The years in REMIND are representative years of the surrounding 5- or 10-year period; e.g., year “2030” represents the 5-year period 2028 to 2032. Spatially, the model represents the world composed of aggregated global regions (Fig. B1). For each region, using a nested constant elasticity of substitution (CES) production function, the model maximizes interannual intertemporal welfare as a function of labor, capital and energy use (Baumstark et al., 2021). The macroeconomic projections of REMIND come from various established global socioeconomic scenarios jointly used by social scientists and economists – the so-called shared socioeconomic pathways (SSPs) (Bauer et al., 2017).

By default, REMIND runs in a regionally decentralized iterative “Nash
mode” where all regions are run in parallel and the interannual intertemporal welfare is maximized for each region for each internal
“Nash” iteration. Trade flows between the regions are determined between
the Nash iterations. During the Nash algorithm, REMIND regions share partial
information with each other, i.e., trade variables in primary energy products and goods. The Nash algorithm is said to converge when all markets
are cleared and no region has an incentive to change its behavior
regarding its trade decisions; i.e., no resources can be reallocated to
make one region better off without making at least one region worse off. A
successfully converged run of standalone REMIND in Nash mode usually
consists of 30 to 70 iterations of single-region models in parallel. Each
parallel single-region model usually takes 3–6 min to solve. A typical
REMIND run in Nash mode lasts 2.5–6 h, depending on the level of sectoral details included. The latest version of REMIND (v3.0.0) is published
as an open-source version on GitHub (release REMIND v3.0.0

DIETER is an open-source power sector model developed for Germany and Europe. In a long-running equilibrium setting (i.e., a competitive benchmark), the model minimizes the overall system costs of the power sector for 1 year. DIETER determines the least-cost investment and hourly dispatch of various power-generation, storage and demand-side flexibility technologies. In the previous literature, different versions of the model have been used to explore scenarios with high-VRE shares, where storage (Zerrahn et al., 2018; Zerrahn and Schill, 2017; Schill and Zerrahn, 2018), hydrogen (Stöckl et al., 2021), power to heat (Schill and Zerrahn, 2020) or solar prosumage (Say et al., 2020; Günther et al., 2021) are evaluated with a high degree of technological detail. DIETER recently also contributed to model comparison exercises that focused on power sector flexibility for VRE integration and sector coupling (Gils et al., 2022b, a; van Ouwerkerk et al., 2022).

As a first step in building a model-coupling infrastructure, we implemented an earlier and simpler version of DIETER (v1.0.2) that is purely based on the General Algebraic Modeling System (GAMS). It has limited features in ramping constraints, flexible demand and storage. The model minimizes the total investment and dispatch cost of a power system for a single region considering all consecutive hours of 1 full year. The technology portfolio contains conventional generators such as coal and gas power plants, nuclear power as well as renewable sources such as hydroelectric power, solar photovoltaics (PV) and wind turbines. Endogenous storage investment and dispatch as well as demand flexibilizations are offered as additional features that can be turned on or off. DIETER, like many PSMs, is a linear program (LP). A typical standalone run (with essential features) lasts from several seconds to several minutes for a single region. See Zerrahn and Schill (2017) for detailed documentation of the initial model, which was implemented purely in GAMS. Later, DIETER's GAMS core was embedded in a Python wrapper for enhanced scenario analysis and postprocessing, but the model can still be run in GAMS-only mode (Gaete-Morales et al., 2021).

It is central to our approach that the price-based variables, such as the market values of electricity generation, are exchanged between the models. This approach ensures full convergence – including both quantity convergence and price convergence in the market equilibrium. Here, we first introduce the intuition behind this approach and then conduct a deep dive into the economic theory behind energy system modeling.

Economic concepts such as market values or capture prices (Böttger and Härtel, 2022), as key variables in our coupling, translate the physical characteristics of variable power generation or flexible consumption into economic ones. For example, generation technologies differ with respect to physical features and constraints – solar and wind generation depends on current weather conditions as well as diurnal and seasonal patterns, whereas this is less the case for dispatchable power plants such as coal, gas, biomass, nuclear or storage (López Prol and Schill, 2021). One consequence of this is that, for example, prices in hours where PV does not produce will essentially be set by other and usually more expensive forms of generation. In cost-minimizing PSMs, the shadow prices of the energy balance are interpreted as wholesale market prices (Brown and Reichenberg, 2021; López Prol and Schill, 2021). Therefore, in general, hourly resolution PSMs are well equipped to translate such physical constraints of generation into (wholesale) power market price time series. By providing such prices generated by PSMs (among other variables of the power sector dynamics) to IAMs, the latter can be indirectly informed about power market dynamics happening on much shorter timescales, even if they lack hourly resolutions. Over iterations, the prices from PSMs act as “price signals” to induce investment decision changes in IAMs, which can in turn provide feedback to the PSMs until the two models converge.

One innovation of our method is that the prices used for the model coupling can be symmetrically applied on the power supply side and on the demand side. On the supply side, the coupling method mainly utilizes the concept of a market value (i.e., the annual average revenue per energy unit of a generator) in a competitive market at equilibrium. Generally speaking, market values of generation usually convey the degree of variability intrinsic to a given source of power supply and reflect the generator's ability to meet an inflexible hourly demand, especially given the lower cost of variable generation compared to dispatchable technologies. Mirroring the concept of the market value, on the demand side, there is the concept of the “capture price” of electricity demand, which conveys the degree of demand-side flexibility. Note that there may be multiple terminologies for demand-side electricity prices; we use “capture price” to be consistent with one example of the literature on this topic. The capture price is the average electricity price that a flexible demand technology pays over a year. For example, flexible demand technologies such as heat pumps, electrolyzers or electric vehicles (EVs) can take advantage of electricity at hours when the generation is cheap in order to obtain a lower capture price, whereas inflexible demand has to pay a higher price on average. Price information given from a PSM to an IAM from both the supply and demand sides can change the IAM's inherent investment and dispatch decisions of power generation as well as inflexible and flexible demand-side technologies.

For an intuitive understanding of our innovative coupling scheme, we take
the supply side as an example and use a toy model to visualize the approach of coupling via market values. The market values of electricity-generating
technologies have been studied in depth (Sensfuß, 2007; Sensfuß et
al., 2008; Hirth, 2013; Mills and Wiser, 2015; Hildmann et al., 2015;
Koutstaal and va. Hout, 2017; Figueiredo and Silva, 2018; Hirth, 2018; Brown
and Reichenberg, 2021). The general idea of the coupling is illustrated in
Fig. 1 for a simplified case of only two types of generators – dispatchable
gas turbines and solar photovoltaics with variable output. Note that we
assume the system to be at a solar share of

Schematic illustration of the coupling approach for a simple power
system in an IAM with coarse temporal resolution, consisting of only gas and
solar generators (no storage).

There are several advantages to this new coupling approach centered on linking prices. First, instead of simply prescribing quantities such as yearly generation and capacities, the approach allows endogenous investment decisions to be made by both models as they converge towards a joint solution. This gives maximal freedom to the coupled models while minimizing unnecessary distortions from one model to the other when some necessary quantities are being prescribed. Second, our coupling scheme provides an elegant treatment of both supply- and demand-side technologies using the concepts of market values on the one hand and capture prices on the other. Third, from a theoretical point of view, transferring the market values of all the generation types in a system alongside mappings of other relevant system parameters can lead to a convergence of the solutions of the two models under idealized coupling circumstances. It can be rigorously shown that our method contains an exhaustive list of interfacing parameters and variables for full model convergence of both quantities and prices. To the best of the authors' knowledge, the last point has not been explored or shown in any previous work.

In certain IAMs, VRE integration cost parameterization has been implemented to mimic the economic consequences of variability of VREs, especially when the models have a lower temporal resolution. Such VRE integration costs are contained in the uncoupled default REMIND power sector modeling. However, the exact parameterization always depends on a particular set of technological costs and parameters which might be subject to changes (Pietzcker et al., 2017), and the parameterization often needs to be carried out anew under new assumptions and scenarios. In contrast, the model-coupling approach is more general, and no such bespoke parameterization is needed.

Inspired by the theoretical framework based on the Karush–Kuhn–Tucker (KKT) conditions for power sector optimization problems (Brown and Reichenberg, 2021), we develop the theoretical basis for the coupling method in this section, which we use for validating convergence in numerical coupling in later sections. In Sect. 3.1, we analytically formulate the fundamental economic theory of the coupling approach. We first introduce the power sector formulations in the two uncoupled models (Sect. 3.1). Then we carry out a derivation of the convergence conditions and criteria, where we map the Lagrangians of the two power sector problems at different time resolutions and derive the equilibrium condition for the coupled models (Sect. 3.2). In Sect. 3.3, we introduce the iterative coupling interface which contains all the previously derived convergence conditions. For REMIND information being passed on to DIETER (Sect. 3.3.1) and DIETER information being passed on to REMIND (Sect. 3.3.2), we list and define the variables and parameters being exchanged at the interface as well as additional constraints and implementations which serve to improve the coupling.

Complete lists of the mathematical symbols and abbreviations can be found in the Appendices.

In the following sections, we first formulate the two uncoupled models and then move on to discussing coupled models. The theoretical tools we develop here are the foundation for the numerical implementation of coupling and serve to validate and assess the model convergence in the Results section.

REMIND and DIETER are both optimization models. REMIND maximizes interannual
global welfare from 2005 to 2150, whereas DIETER minimizes the power sector
system cost for a single year and a single region. For a given REMIND
Nash iteration (see Sect. 2.1), the single-region economy is in
long-term equilibrium after the optimization problem is solved. Since, given
a fixed national income, lower energy system costs mean higher consumption,
which leads to increased welfare (see Appendix C for details), maximizing
welfare can be assumed to correspond to minimizing energy system costs, a
part of which is power sector costs. Therefore, to reduce the complexity of
our analysis, we formulate an uncoupled REMIND model based solely on the
power sector cost minimization and not the total welfare maximization. For the
standalone REMIND, the multi-year power system cost for a single region
equals the sum of all variable and fixed costs of generation,

Under the simplifying assumptions made for the derivation in this paper, the only independent decision variables are capacities, generations and curtailments. Small letters denote either exogenously given parameters or endogenous shadow prices.

For standalone DIETER, which has a year-long time horizon, the power system
cost is

The cost minimization of the total power sector cost

Without coupling, and under a baseline scenario, there are several
constraints for each model. In the following equations we denote the shadow
price (i.e., the Lagrangian multiplier) of a constraint by the symbol
following

Constraint on generation for meeting demand, a.k.a. “supply–demand
balance equation” or “balance equation” for short:

Constraint on maximum capacity by the available annual potential

Constraint on generation being non-negative:

Constraint on maximum generation from capacity:

“Historical” constraints on capacities in REMIND. This makes REMIND a
so-called “brown-field model”, i.e., a model accounting for the standing
capacities in the real world. Past capacities (

Near-term upscaling constraint on VRE capacity expansion, represented by
an upper bound on near-term capacity addition in model period

Note that, regarding the simplification of REMIND above, to the best of the authors' knowledge, there is no theoretical or empirical concept that addresses the validity of drawing equivalence between welfare maximization and energy system cost minimization in IAMs. Naively, given that the gross domestic product (GDP) is unchanged, decreasing energy system costs raises consumption and therefore welfare. However, this is only valid under the assumption that energy is a substitute for (and not a complement to) capital and labor; i.e., one usually cannot raise economic output (GDP) simply by spending more on higher energy expenditure (while satisfying the same level of energy demand). Nevertheless, this is likely a necessary condition and not a sufficient one for proving the equivalence. More theoretical research will be needed to draw a precise and rigorous equivalence. However, in practice, we see that during our numerical calculation the model is well behaved according to this reduced theory, which means that the parameters in the models are in a regime where such an assumption is valid, at least in the case of IAM REMIND.

In the last section we discussed the standalone uncoupled power sector formulations in REMIND and DIETER. In this section we discuss the coupled models and their convergence. Under simplified assumptions, we first derive the mapping between the models that is necessary for a convergence (Sect. 3.2.1 and 3.2.2), and then we derive theoretical relations that are later used to validate the numerical results of the coupled run (Sect. 3.2.3).

Our aim is to develop a method under which comprehensive convergence can be reached for soft-coupled multiscale models. We achieve this by deriving a mapping of the two problems such that their decision variables have identical optimal solutions and the endogenous shadow prices are also equal across the models. The convergence conditions of the coupled REMIND–DIETER model for the power sector are the result of such a mapping. Below, we first define what is meant by a “comprehensive model convergence” and then sketch the workflow of the derivation of a coupling framework which would result in a comprehensive model convergence of both decision variables and shadow prices. The detailed derivation is in Appendix D.

Here, we derive the conditions under which the endogenous decision variables
are identical at each model's optimum, i.e.,

Our coupling approach fundamentally relies on mapping the parameterization
of the Lagrangians for both optimization problems. It is trivial to show
that, as long as the KKT Lagrangians are identical with respect to the
decision variables, the solutions of the problem are identical, e.g.,
if an optimization problem A has Lagrangian

The schematics of the Lagrangian-based derivation procedure for a simplified version of REMIND–DIETER iterative convergence. After simplifying assumptions, we can construct the Lagrangians of the reduced REMIND model and the full DIETER model for a single year (Eqs. 3–4). Comparing and mapping terms in the Lagrangians (a key step in bold), we discover that iterative exchange of a broad range of information is needed for a fully harmonized parameterization of the Lagrangians. Under the harmonization specified in the seven convergence conditions (color-coded for directions of information flow), the coupled models can give rise to identical optimal solutions of the models' respective (annually aggregated) decision variables and hence a full quantity convergence. The necessary shadow price convergence is shown in the detailed derivation of the harmonization conditions (h1–h7) in Appendix D.

The analytical derivation workflow, as shown in Fig. 2, is described in detail as follows. First, we apply simplifying assumptions to reduce the complexity of the uncoupled models (before the key step in blue in Fig. 2). Assumptions have to be made to justify reducing the scope of the REMIND model, such that for the purpose of the analysis it is on equal footing to DIETER. We achieve this by reducing the global REMIND model to a single sector (the power sector), single year and single region. To reduce the REMIND model from a macroeconomic–energy model to a power-sector-only model, we make similar assumptions to before when formulating the uncoupled REMIND power sector (see Sect. 3.1). To reduce the REMIND model further to a single year, we assume that the models only contain constraints in the power sector that are not intertemporal; i.e., we ignore the brown-field and near-term constraints for now. Since for each iteration of the REMIND model in Nash mode interregional trading happens between the iterations, the single-iteration optimization model is already for a single region and therefore does not require simplification. After these simplifying steps, in this part of the derivation, we can treat REMIND's power sector as “separate” from the rest of the model and treat the dynamics of a single year in REMIND as independent of the dynamics of other years. Later, the numerical results of the convergence can confirm to a large degree the validity of these assumptions, especially in the green-field temporal ranges, i.e., where the intertemporal brown-field constraints have little influence on the dynamics. Note that, with the inclusion of these intertemporal constraints in the derivation, the mapping becomes more complicated, especially for the near-term range, i.e., before 2035. So, in practice, this derivation of the coupling interface is only an approximation of what is needed for a full convergence of DIETER and REMIND, since it deliberately ignores such constraints. See also Sect. 6.1.

After the necessary simplification assumptions, we construct the Lagrangians
for the simplified model REMIND and for DIETER (after the blue block in Fig. 2) (Gan et al., 2013). For a single-year reduced REMIND power sector model,
the Lagrangian is

This Lagrangian-mapping-based derivation can theoretically show that our approach (in its simplest form) necessarily leads to model convergence and has the advantage of being mathematically straightforward and rigorous. The necessary information from the power sector dynamics is all contained in the list of conditions derived from such a mapping. If the coupling contains less information, a convergence is not possible; at the same time, for a model convergence, one does not need to pass on any additional information beyond what is contained in this list of conditions. The list of information derived here is therefore complete and exhaustive for a coupled convergence.

The convergence conditions (h1–h7), which are derived in detail in Appendix D following the procedure in Sect. 3.2.1, are summarized here.

Annual fixed costs are harmonized:

Annual variable costs are harmonized:

Annual average market values for each generation type

Using markup Eq. (5), we modify the original objective function

Annual power demands are harmonized:

Annual average prices of electricity are harmonized:

Annual average capacity factors for each generation type

Annual curtailments are harmonized:

In mapping the Lagrangians (Eqs. 3–4), except for the objective function, the rest of the parameterization contains endogenous shadow prices and endogenous quantities. Since endogenous values can only be known ex post, this imposes a strict requirement on the coupling that it must be iterative, with the endogenous part of the parameterization coming from previous iteration optimization results – usually from the other model. The mapping of the endogenous information requires careful argument in each case (i.e., the derivation of h3–h7). In the case of the balance equation constraint Lagrangian term (corresponding to c1), the shadow prices of the constraint in the current-iteration REMIND model are exogenously corrected by a set of technology-specific markups (see the introduction in Sect. 3.1), such that the new “corrected” market value in REMIND is manipulated to match the market value of the previous iteration of DIETER. This is the heart of our coupling approach using markups as the price signals. In the case of the constraint on maximum generation from capacity (corresponding to c4), the endogenous shadow prices in the current iteration REMIND can be shown to be automatically mapped to those in the previous iteration of DIETER given that the annual average capacity factors in the constraints are harmonized (h6–h7).

In actual implementation, most of the above mappings are modified for numerical stability (Sect. 3.3.2, Appendix H).

Here we first state the convergence criteria, which are mathematical relations that are being satisfied under model convergence. Then we also discuss equilibrium conditions of the coupled models that alongside the convergence criteria can be used to check numeric results to validate and assess the convergence outcome.

Under a theoretical full convergence of the coupled model,

annual average electricity prices,

capacities and

(post- or pre-curtailment) generations

Besides using convergence criteria (v1–v3), we also use a type of equilibrium condition – the so-called “zero-profit rules” (ZPRs) – to validate the numerical model convergence. ZPRs are mathematical relations which state that, under market equilibrium, prices are equal to the costs for electricity. This is not always the case, especially in situations where there are extra constraints on the model that distort this equality. ZPRs contain model parameters and decision variables at market equilibrium, and they can be derived from the KKT conditions of the model (Appendix F). ZPRs are therefore reliable tools in ascertaining the sources of market values or the price of electricity of the power sector because, according to the ZPRs, one can always decompose the prices into the cost components, i.e., so-called levelized costs of electricity (LCOEs). The decomposition of prices into cost components is important because the prices of electricity in the power market are overdetermined by the energy mix, so it is possible that two different power mixes correspond to the same electricity price. In numerical results, a slight mismatch of the energy mix at the end of the coupling is unavoidable, so, alongside comparing the prices, it is often helpful to compare the makeup of the LCOEs across the models, such that they also appear harmonized at the end of the iterative convergence. Overall, ZPRs are a helpful tool for visualizing and understanding the power market dynamics, both from the point of view of each generator type and from the point of view of the entire electricity system. It is worth noting that the ZPRs, which are mathematical conditions derived from an idealized modeling of the power sector as fully competitive, are only an approximation of the real-world markets, where firm profits exist. ZPRs in their technical definition simply mean that, at model equilibrium, cost equals revenue. Given that the profits are defined as the difference between revenue and cost, the profits are zero in this situation. The name “zero-profit rule” therefore should not be overinterpreted beyond its technical contents, and one should be aware of its theoretical origin and the assumptions under which it is valid.

The ZPRs of the coupled model can be derived based on (1) the uncoupled models, (2) the modification made to the model due to the coupling interface (h1–h7) and (3) any additional modifications made to the model during our numerical implementation. In the last category, for a complete numerical implementation of the coupling, we add one additional capacity constraint, i.e., (c7) and (c8) for each model. The first capacity constraint (c7) is created in REMIND to circumvent the issue of extremely high markups from peaker gas plants in the scarcity hour of the year in the DIETER model, which otherwise causes instability during the iterative coupling. The second constraint (c8) is a simple brown-field constraint implemented in DIETER to address the fact that DIETER is a green-field model that is otherwise ignorant of standing capacities in the real world. For simplicity, (c7) and (c8) are not included in the convergence condition derivations in Sect. 3.2.1. The derivations of the ZPRs outlined by the above three steps have been carried out in Appendix F (uncoupled models), Appendix G (coupled REMIND only including the coupling interface, coupled DIETER including constraint c8) and Appendix H (coupled REMIND, including constraint c7).

In summary, the ZPRs for both coupled models are as follows.

Coupled REMIND

Technology-specific ZPR:

System ZPR:

Coupled DIETER

Technology-specific ZPR:

System ZPR:

The prime sign indicates that the term has been modified from the uncoupled
versions due to implementation in the coupling.

Due to the fact that several sources of shadow prices cannot be incorporated
during the derivation for convergence (Sect. 3.2.1), in numerical
experiments of the coupled run it is appropriate to compare the following two
types of prices across the two models for price convergence:

electricity price convergence, not including any capacity shadow prices;

sum of electricity prices and all respective capacity shadow prices converging.

In this section we list parameters and endogenous variables that are exchanged between REMIND and DIETER. This already satisfies most convergence conditions, while the remaining condition (h5) is checked in Sect. 4 as part of the convergence criteria (v1–v3). An overview of the model coupling and the flow of information under convergence conditions is shown in Fig. 3.

The schematics of the REMIND–DIETER iterative soft-coupling. The power sector module of IAM REMIND, which is between the layer of primary-to-secondary energy transformation, is hard-coupled with other modules inside REMIND such as macro-economy, industry and transport. In PSM DIETER, the power market with generators of various types is modeled with hourly resolution, with options for storage and flexible demand. The information exchanged between the models (block arrows) is determined via the convergence conditions (h1–h7) derived before (Sect. 3.2.1). In order to improve performance and facilitate convergence, additional constraints (c7 and c8) are included in the coupling interface. The coupling interface for REMIND to DIETER is programmed as part of modified DIETER code and vice versa. Both interfaces are written in GAMS. For a single region, the scheduling of coupled iterations is illustrated in Fig. E1 in Appendix E. Sixteen DIETER optimization problems are solved for each representative year of REMIND in parallel, scheduled after each internal REMIND Nash iteration (see Sect. 2.1 for a description of the iterative Nash algorithm).

During the coupling, the following exchanges of parameters and variables take place iteratively in both directions via the interface.

The following information flow is from REMIND to DIETER.

Technology fixed costs (convergence condition h1)

Annualized capital investment cost: this is calculated from the endogenously determined overnight investment cost, plant lifetime and the endogenously determined interest rate. The overnight investment cost is determined from floor cost, learning rate and the endogenous global accumulated deployment. Note that investment costs decrease according to the endogenous learning rate. The interest rate is about 5 % on average but is endogenous and time-dependent in REMIND.

Annualized operation and maintenance (“O&M”) fixed costs (OMF): they are a fixed share of the capital costs.

Adjustment cost: this is technology-specific and is proportional to the capital investment cost. See Appendix I for its implementation.

Technology variable costs (convergence condition h2)

Primary energy fuel costs: they are endogenously determined as the shadow prices of the primary fuel balance equations in REMIND. Import prices, domestic prices of extraction, the amount of regional reserves and the amount of fuel demand can all influence the fuel cost. The relevant fuel costs include coal, gas, biomass and uranium. The fuel costs can have interannual intertemporal oscillatory components which can cause instability during iteration if coupled directly. We mitigate this by conducting a linear fit to the time series before passing them on to DIETER.

Conversion efficiency of each generation technology

O&M variable costs (OMV)

CO

Grid cost: in REMIND, the stylized grid capacity equation is proportional to the amount of pre-curtailment VRE generation. So, the grid cost is effectively a variable cost. Note that, in future work, grid costs can be modeled in more detail either in DIETER or in another PSM. Here, we use the parameterized grid costs which are implemented in the default REMIND as an approximation to the necessary grid cost.

Power demand (convergence condition h4). REMIND informs DIETER of the
total power demand

Pre-investment capacities

Total regional renewable resources for wind, solar and hydro (constraint c2), such that DIETER capacities are constrained by the same total available resources as in REMIND

Annual average theoretical capacity factors of VREs and hydroelectric ones in
REMIND (convergence condition h6). We denote the pre-curtailment utilization
rates of VRE capacity as “theoretical capacity factors”, as these can be
achieved in theory if there is no curtailment. They are usually determined
by meteorological factors such as wind and solar potential as well as the
efficiency of the turbines or solar photovoltaic modules. In contrast, the
post-curtailment utilization rates of VREs are “real capacity factors”, as
these are the real utilization rates after optimal endogenous dispatch. The
time series of theoretical utilization rates of VRE generations of 1
historical year in DIETER are scaled up such that the annual average
theoretical capacity factors in DIETER equal the exogenous parameters in
REMIND:

The following information is passed from the last-iteration DIETER to REMIND.

Market values are

The peak hourly residual power demand

Annual capacity factors of dispatchable plants

Annual solar and wind curtailment ratio: curtailment as a fraction of total
annual post-curtailment generation

The prefactors usually depend on the differences between generation shares
in the two models: for example, the prefactor for markup is a linear function of the
difference between the current-iteration REMIND endogenous generation share
and the last-iteration DIETER generation share. We illustrate the mechanism of
prefactors using the markup for solar as an example: a lower market value for
solar is consistent with a higher solar share according to the well-known
self-cannibalization effect of a decreasing VRE market value as the VRE share
increases (Hirth, 2018). Therefore, we can introduce an automatic
stabilization measure through a negative feedback loop: if the REMIND
endogenous share is larger than in the last DIETER iteration, in which case
the in-iteration market value should be lower than the last-iteration DIETER
market value, the multiplicative prefactor for the market value should be
constructed such that it is smaller than 1. This lowers the market value
for solar and decreases the in-iteration REMIND markup

In this section, we check the convergence behavior for prices and quantities (capacity and generation) in coupled model runs using the convergence validation criteria from the last section. Comparing the numerical results with the theoretical prediction, we can confirm that REMIND–DIETER soft-coupling indeed produces almost full convergence.

Throughout this section, we only use one scenario – a proof-of-concept baseline scenario. Under the proof-of-concept scenario of the coupled run, we disable storage (i.e., batteries and hydrogen) and flexible demand (i.e., electrolyzers) in both models, as this allows us to use the theoretically derived convergence criteria from Sect. 3, which would become overly complex in a model with storage and flexible demand. The coupled run is under a baseline scenario; i.e., there is no additional climate policy implementation. Since this is a configuration created only for comparison with the theoretical prediction, it is not meant to be a policy-relevant configuration. In more policy-relevant coupled runs, we turn on storage and flexible demand (see Sect. 5). For schematics and computational run times of the coupled iterations, see Appendix E.

For the coupled runs, we define a baseline scenario for the single region
Germany under SSP2 assumptions corresponding to the “middle-of-the-road”
scenario (for a definition of the SSPs, see Koch and Leimbach, 2023).
Specifically, this means that REMIND runs for all global regions in
parallel but that DIETER only runs for Germany. Only information in the German
power sector is exchanged for the two models. We use a low CO

Due to the particular implementation of offshore wind in REMIND, DIETER wind offshore capacities are fixed to those of REMIND to avoid too much distortion. Since in our scenarios offshore wind capacity in Germany is relatively small compared to other generators, this fixing represents only a minor distortion of the coupling. Hydroelectric generation in REMIND is assumed to have an average annual capacity factor of around 25 %. This capacity factor is implemented as a bound in DIETER. For simplicity, instead of a time series profile for hydroelectric generation, we allow the hourly capacity factor to be no higher than 90 %, meaning hydro is close to being dispatchable in all our scenarios. In the German context, hydro usually means run-of-the-river hydroelectricity, which has a variable output. Nevertheless, we find the 90 % maximum hourly capacity factor a reasonable assumption to make, since in our runs we do not yet consider pumped hydro as a technology in this study, so a more dispatchable quality of hydro can be assumed. Results presented in this section belong to the same coupled run under the proof-of-concept scenario.

According to theoretical convergence criteria (under simplifying assumptions, Sect. 3.2.1–3.2.3), at numerical convergence, the electricity price of REMIND should be equal to the price of DIETER. However, REMIND is interannual intertemporal, whereas DIETER is only year-long, so we compare the differences over time as well as the interannual average of the price differences (Fig. 4).

Annual average electricity price convergence behavior of a coupled
run for Germany under a “proof-of-concept” baseline scenario.

In Fig. 4a, the price difference oscillates from period to period. As the coupling starts, the REMIND price is much higher than DIETER, especially in the earlier years. After around the 10th iteration, the difference in the early years starts to reverse: DIETER's price becomes higher than REMIND. Around 2040–2060, REMIND has a higher average price than DIETER due to the VRE market values being higher than their LCOEs. This is discussed later in Sect. 4.3.2.

In Fig. 4b, we calculate the difference between two time series – the time-averaged power prices in the two models. We observe that the difference between them decreases over the iterations, showing a clear converging trend, and stabilizes at around 3 % of the REMIND price. There are two observations regarding the price convergence of the coupled run. First, the convergence happens rather quickly within 10 iterations. Second, the converged value of the price difference is not exactly 0 but is slightly above 0, a few percent of the full price (a few USD per megawatt hour). Under ideal convergence conditions, according to (v1), the two prices should be equal at full convergence for every coupled year. However, in practice, the average prices do not match perfectly, as there are several sources of distortions from capacity shadow prices. The capacity shadow prices come from many sources in both models: extra constraints such as (c7)–(c8) that are not part of the analysis leading to (v1), constraints that are in REMIND but not in DIETER (c5–c6), and exogenous wind offshore capacity in DIETER. Some of these capacity shadow prices in both models can be more or less consistent with each other (such as the standing capacity constraint in DIETER and brown-field constraints in REMIND), but others are not and can distort two models in different ways, causing some degrees of misalignment in prices. As discussed before, prices can be overdetermined by the energy mix (Sect. 3.2.3). Therefore, some of the capacity shadow prices – even though not aligned between the two models – can nevertheless cancel each other out (especially when averaged over time), potentially causing the price differences to be moderate. To examine exactly how well the prices at the end of the coupling match, we need to check the cost decomposition of the prices. This is discussed later in Sect. 4.3.

Also note that Fig. 4b presents a time-averaged price comparison, and on average the difference between the prices in the two models is small at the end of the coupling. However, when one compares the maximal deviation for any single year at the end of the coupling, it can be as high as USD 10 per megawatt hour, e.g., around 2050 (Fig. 4a). This is much larger than the 3 % averaged deviation in Fig. 4b. However, compared to default REMIND prices (which we cannot show due to limited space), we are fairly confident that the oscillation of the coupled REMIND results from internal dynamics that are also visible in the default uncoupled version. So, a time-averaged treatment is adequate in displaying the total price convergence here.

Besides price convergence, the capacity and generation decision variables must also converge within a certain tolerance at the end of the coupling. This is reflected in the generation mix (Fig. 5) and the capacity mix (Fig. 6) at the end of the coupled run. Due to the existence of several sources of mismatch between the two models already mentioned in the last section, which is already manifested in the mismatch in electricity prices of the two models, a certain degree of mismatch in quantities is also to be expected. Nevertheless, the agreement between the two endogenous sets of decision variables is satisfactory. For this coupled run, the differences of the generation share of any single technology between the two models are smaller than 4.4 % for each year until 2100. Figure 5b highlights some subtle model differences in generation. For example, after 2040, REMIND favors solar and coal, whereas DIETER tends to have more combined-cycle gas turbines (CCGTs) and wind onshore. Due to the low capacity factor of open-cycle gas turbines (OCGTs) and solar compared to the capacity factors of the other generators, the capacity mix differences between models are amplified for these two technologies (Fig. 6). However, overall, the generation mixes and the capacity portfolios at the end of the coupled run are generally similar.

Annual electricity generation convergence at the final iteration
of a coupled run for Germany under the proof-of-concept baseline
scenario.

Capacity convergence at the final iteration of a coupled run for
Germany under the proof-of-concept baseline scenario.

For periods that are policy-relevant in the short to medium term (i.e., before 2070), the convergence for quantities is generally slightly worse in the near term, i.e., in the 2020s and 2030s, likely due to the capacity bound mismatch in the near term (such as the capacity bounds c5–c6 in REMIND not being completely replicated by the standing capacity constraint c8 in DIETER). If DIETER does not contain identical bounds to REMIND, then its endogenous decision will have more of a green-field rationale than REMIND does, the latter of which is more constrained in the near term. In case an improvement in near-term convergence is desired, these bounds could be implemented more carefully and be more technology-specific. Due to the limited scope, we only apply a generic standing capacity constraint (c8) in DIETER to represent the basket of various constraints. The convergence of quantities is also not perfect in the green-field periods, such as after 2040, when both models are less constrained by near-term dynamics. The reason for this is likely the fact that, in DIETER, hydroelectric generation is not economically competitive against other, cheaper forms of generation such as solar and wind. However, in REMIND it is economically competitive, likely due to the long lifetime of the plants. Semi-exogenous wind offshore capacities in both models could also play a role. This is discussed in more detail in Sect. 6.1.

As our analytical discussion showed before in Sect. 3.2.3, model equilibria in the form of ZPRs are useful for validating convergence in a more detailed way by decomposing prices into cost components as well as any perturbation from capacity shadow prices. In this section, we first compare the system LCOEs, price and capacity shadow prices of the two models for ZPRs on the system level, and then we show the technology-specific ZPRs. Using this validating step, we can visually ascertain that the cost components and prices or market values in the two models are remarkably similar on the system level as well as on the technological level, demonstrating that the underlying principle behind the coupled convergence holds to a good degree.

At the convergence of the soft-coupled model, we expect ZPRs to be satisfied
for the two systems individually (Eq. 11 for REMIND and Eq. 13 for
DIETER); i.e., each price time series also matches the LCOE time series to a
good degree, barring distortions from the capacity shadow prices. This is to
say, under full convergence, that the time series of system LCOEs and the sum of
the time series of the electricity prices and time series for capacity
shadow prices for both models should overlap one another within numerical
tolerance. The costs and prices at the last iteration of the coupled run are
summarized in Fig. 7. The electricity prices derived from the shadow prices
of the balance equations are shown in dark grey: (a) REMIND electricity
price

From Fig. 7, we can conclude that the ZPR for DIETER is satisfied to very good accuracy for every year (the blue line: the sum of the electricity price and the capacity shadow price has exactly the same value as the sum of the LCOE bars). For REMIND, the ZPR is satisfied year on year to a lesser degree but on average to a good degree given the interannual fluctuations. The prices in the coupled REMIND become very erratic for the early years (2020–2025), likely due to the interaction between the historical or near-term bounds in REMIND and the exchanged information from DIETER for those years. The LCOE component structures match well across the models for most years, which serve as additional visual support for the price convergence shown in Fig. 4; i.e., the cost structures behind the prices are harmonized as well at the end of the coupling. The origins of the differences between LCOEs and prices as well as the degree with which capacity shadow prices account for them can be found when one examines the LCOE and market values of specific technologies, which are analyzed next.

Cost components of the system LCOEs (bars), electricity prices
(grey lines), and the sum of electricity prices and capacity shadow prices
for

After validating ZPRs on the system level, we further dive into each technology and check the ZPRs for each technology in both models at the last iteration of the coupled run (Fig. 8).

Technology-specific costs and market values for

In Fig. 8b, DIETER LCOE and market values for the eight types of generators are shown. As expected from the ZPR, the LCOE always matches the sum of the market value and capacity shadow prices for each technology and for each year (Eq. 12). The differences between the dashed and solid lines are largely the generation capacity shadow prices. It is worth noting that, at the end of convergence, the sizes of the shadow prices are in general small for the main generator types, e.g., solar, wind onshore, CCGTs and OCGTs. This indicates the fact that, for these technologies for most periods, the optimal DIETER generation mix is close to that of a green-field model. That is, DIETER hardly faces any exogenous constraints (except resource constraints that are aligned with those of REMIND) and can make fully endogenous investment and dispatch decisions based on cost information alone. On the whole, DIETER at the coupled convergence experiences only a small amount of distortion from the brown-field model REMIND, especially concerning the “model-suboptimal” real-world standing capacities from biomass, hydro and coal.

In Fig. 8a, we show the REMIND LCOE and market values for the same generation technologies. Due to the intertemporal nature of REMIND, the sum of the market value and the capacity shadow price for each technology and for each year generally matches the LCOE slightly less well than DIETER. This means that for REMIND the ZPR (Eq. 10) for each generator type is also satisfied to a good degree for the main generator types, e.g., solar, wind onshore, coal, CCGTs and OCGTs. The mismatch in biomass and hydro might come from the shadow price from historical capacities.

Since the differences between market values and costs are accounted for by the capacity shadow price to a large degree, it is worth interpreting physically the sources of these “hidden” costs and revenues. For REMIND, the capacity shadow prices consist of those in (c2), (c5) and (c6) as well as the “peak residual demand constraint” from DIETER (c7). Constraint (c7) is created to circumvent high markups, especially from peaker gas plants (Appendix H1), because peaker gas plants generate power mostly only at hours with high prices (especially the scarcity hour price) and therefore have very high market values compared to the annual average electricity price. The high market values of OCGT – usually more than 5 times the average annual electricity prices – act as a large incentive in the next-iteration REMIND and lead to over-investment in capacities. Over iterations, this causes oscillations in the quantities and prices in the coupled model and prevents model convergence. To circumvent the issue of high markup, we implement (c7) as an equivalent peak residual demand constraint. As can be shown mathematically (Appendix H), (c7) generates essentially the scarcity hour price, and it is very easy to validate this for OCGT in Fig. 8a. The capacity shadow price derived from this peak residual demand constraint, when translated to energy terms and added to the market value, correctly recovers the LCOE for OCGT, recovering the original ZPR (Appendix H1.2). This indicates that, under multiscale model coupling, an extra constraint is an effective way of circumventing potential issues of numerical divergence due to the large impact from short-term dynamics, such as the large market value of peaker gas plants.

For DIETER, the two sources of the capacity shadow price are the total renewable potential limit (constraint c2 in Sect. 3.1) and the standing capacity constraint from REMIND (constraint c8 in Sect. 3.2.3). For the first type, the resulting capacity shadow price is a hidden “positive cost” from the perspective of the power user since endogenously DIETER would like to invest more but is limited by the natural resources available. An example of this first type is hydroelectric power between 2020 and 2035 due to the limited resource (run-of-the-river) in Germany. It is worth noting that, from the generator's perspective, the capacity shadow price from the resource constraint can be interpreted as an extra resource rent. The second type of capacity constraint originates from the standing capacity, and the latter is received by DIETER from REMIND as a lower bound. This constraint usually results in a hidden “negative cost” from the perspective of a power user; i.e., a part of the cost (LCOE) does not get passed on to the electricity price, so the users get part of the capacity “for free”. (This can also be interpreted as subsidies for generators to sustain these unprofitable capacities.) This is because, based on green-field cost optimization, DIETER endogenously would invest less in certain technologies. However, since the standing capacities account for the existing generation assets in the real world, which can be model-suboptimal, the overall costs are above a green-field equilibrium and above the prices the user pays. We find examples of such a capacity shadow price manifested in biomass, coal and hydroelectric, all of which are part of the existing German power capacity mix, but evidently not all of them for any given period are “green-field optimal” based on the pure cost consideration in DIETER. Interestingly, after 2035, the sign of the capacity shadow price for hydroelectric generators reverses. This is likely due to the continuous decline in the VRE costs after 2035 tipping the power sector into a regime where hydroelectric becomes less economically competitive in DIETER, at least compared with REMIND. As a result, the standing constraint from REMIND starts to be binding on the capacity from below, relieving the resource constraint binding from above.

For DIETER, the capacity shadow price from the standing capacities also indicates the degree of disagreement between DIETER and REMIND. For most future years, REMIND standing capacity constraints are not binding in DIETER for solar, wind onshore, CCGTs and OCGTs, indicating good agreement between the models. The small amounts of shadow prices near 2060 for OCGTs and solar in Fig. 8b are likely due to the time step size change in REMIND which causes a small jump in the interest rates near these years.

Lastly, in Fig. 4, we observe a slightly higher average electricity price in REMIND than in DIETER, especially in the intermediate years. This could be due to fixed offshore wind capacities, which are never economical for endogenous investment in the parameterization used here. This generates a high capacity shadow price until around 2045–2060 that is visible in both DIETER and REMIND.

In this section, we present baseline and policy scenario results for
Germany using a more realistic configuration of the coupled model with
electricity storage and flexible electrolyzer demand for green hydrogen
production, which is then used outside the power sector (e.g., in industry or
heavy trucks). We show results for a baseline scenario and a net zero by
the 2045 climate policy scenario. Note that, due to REMIND's global scope, under
the net-zero scenario we also assume a larger climate policy background of
a 1.5

In Sect. 5.1, we present long-term power sector development. In Sect. 5.2,
we present short-term power sector hourly dispatch and price results. In the
following, we broadly describe how these additional features are
implemented.

Storage: we use a simple storage implementation where DIETER makes an
endogenous investment in two kinds of storage technologies:

lithium-ion utility-scale batteries;

on-site green hydrogen production via flexible electrolyzers, storage and combustion for power production.

The reason we only allow DIETER to endogenously invest in storage technologies is that the additional intertemporal optimization offered in REMIND is relatively less important than that for the investment of generation technologies. In REMIND, intertemporality mainly accounts for two aspects in the real world: (1) implementing adjustment cost and (2) tracking standing capacity. The adjustment costs simulate system inertia to rapid capacity addition or removal. In the case of battery and other storage technologies, the ramp-up of deployment faces relatively less inertia compared to wind and solar. Compared to generation technologies such as wind and solar, the storage technologies tend to have lower total capacities, meaning their ramp-up rate is usually lower. Also, their deployment is mostly constrained by their higher cost. For utility storage technologies, they are mostly not yet deployed at scale, which means that there is very little existing capacity and the investment in storage in REMIND is mostly green-field, rendering it unnecessary to give DIETER a standing capacity of them.

Flexible demand: as a simple representation of flexible demand, we choose to
implement a common power-to-gas (PtG) technology, i.e., the so-called
“green hydrogen” electrolysis. We split the total power demand required to
produce green hydrogen from REMIND from the total power demand

This section presents scenario results of the coupled model with a long-term view of capacity and generation using either the proof-of-concept scenario or more realistic configurations.

In Fig. 9a, under the baseline scenario, and with available storage and
flexible demand, we observe a more than 35 % increase in the total power
demand from 2020 to 2045 and more than 65 % by 2080. This is due to an
increase in end-use electrification. The increased electrification comes
from a moderate growth in electricity use in the building sector and a more
significant growth in the electric vehicle (EV) fleet. In the building sector, the final energy
share of electricity is projected to increase from 28 % in 2020 to 39 %
in 2045. The final energy share of electricity in the transport sector is
22 % by 2045, up from 2 % in 2020. Note that, even under no additional
climate policies, based on only the increase in EVs shares in new-car sales
in many world markets today, we expect higher power usage from EVs in the
future. Within the energy mix, we see a slow decline in coal generation over
time, which is replaced by CCGT generation and a significant increase in
VREs. The VRE share reaches above 50 % by 2045, but slightly less than half of
the energy mix still contains coal and gas power. In terms of capacity
expansion (Fig. 9b), due to both a lower generation cost and a higher power
demand, solar capacity expands by almost 5 times from today until 2045.
However, the moderate VRE shares mean that the requirement for battery
capacity is not high, i.e., only 12 GW of batteries by 2045. Due to the low
CO

DIETER–REMIND converged results of the long-term

By comparing the above baseline scenario (with storage and flexible demand)
(Fig. 9) with the proof-of-concept baseline scenario (without storage or
flexible demand) before (Figs. 5 and 6), it is clear that, while battery
storage and partial demand flexibility play a role after 2040 in increasing
the VRE share in Fig. 9, in the near term, the scenarios with and without
available storage and demand flexibility look very similar under no
additional climate policies. However, due to the technological learning effect,
an even absent additional CO

In Fig. 10, under a stringent climate policy (economic-wide carbon neutrality
in 2045), with available storage and partially flexibilized demand (for
hydrogen production used in other sectors), the total power demand more than
doubles, and the power mix is dramatically transformed. Compared to both the
baseline case without storage and demand-side flexibilization (Figs. 5 and 6)
and the baseline scenario with storage and flexible demand (Fig. 9), a very
high VRE share in the generation mix is reached already by 2040
(

DIETER–REMIND results of the long-term generation and capacity
expansion for Germany's power sector in the “net-zero 2045” scenario. The CO

Under the stringent climate policy scenario, dramatic changes in the end-use sectors will be under way in the form of direct electrification and substitution of fossil gas with hydrogen. In the building sector, the final energy share of electricity is projected to increase from 28 % in 2020 to 66 % in 2045. In transport, the final energy share of electricity will be 56 % by 2045. In the industry sector, the share of electricity increases from 25 % to 63 %. By 2045 there is also a notable increase in the use of green hydrogen produced from 45 GW flexible electrolyzers (at about 42 % of the average annual capacity factor), amounting to 0.5 EJ (3.5 million tons) per year in the final energy, which is primarily used in industry. For a comparison with other published German net-zero scenario results, see Sect. S4.

In this section, results of hourly resolution are shown and discussed for a selected model year. We use established methods such as RLDCs to visualize the hourly dispatch result and show the hourly generation and dispatch time series for some typical days in summer and winter.

RLDCs can be used to visualize the dispatch of energy system models. Each subsequent curve is calculated by subtracting the generation of a technology from the hourly residual demand curve and then sorting the remaining demand in descending order. On the left-hand sides of the RLDC graphs, one can easily check the amount of residual demand not met by variable wind and solar production. The topmost line in the RLDC graph is the load duration curve for inflexible demand (excluding the demand from flexible electrolysis for hydrogen production used in other sectors).

In a baseline configuration without flexibilized demand or storage, despite lacking the explicit hourly dispatch, via bidirectional soft linkage, REMIND was able to achieve a final dispatch result that replicates DIETER to a satisfactory degree (Fig. 11). This is a combined effect of a convergence of capacities (Sect. 4.2) and full-load hours at the end of the coupled run. In the peak residual demand hour (the leftmost point in the RLDC), the DIETER-coupled REMIND accounts for the requirement of dispatchable capacities via the constraint (c7), and the composition of the mix is replicated from DIETER and correctly guarantees that the peak hourly demand is met.

Side-by-side RLDC comparison between

In a net-zero policy with storage and flexible electrolysis demands, comparing dispatch results under both scenarios (Figs. 11 and 12) for the model year 2045, it can be observed that, under a stringent emission constraint, the system allocates a significant amount of short-term storage to replace the dispatchable generation such as coal and CCGTs. Long-term storage such as hydrogen electrolysis combined with hydrogen turbines further reduces the capacity factor of the remaining OCGTs and CCGTs. Besides storage, there is also a significant amount of deployment of flexible electrolysis demand for producing hydrogen (PtG) that is not used in the power sector but is in industry or heavy-duty transport. The use of PtG technologies leverages cheap variable wind and solar energy to achieve the goal of sector coupling. By way of storage and PtG, a significant share of the curtailment can be utilized (more than 70 %), either by shifting the supply to times of low VRE production via storage or by producing hydrogen using surpluses which can be used in other sectors.

Side-by-side comparison between the

To more directly inspect the results of the hourly dispatch under various scenarios, we visualize the hourly generation and demand for typical days. Due to the climate in Germany, solar potential is particularly low during winter months. Therefore, it is important to observe the periods in both summer and winter.

From the optimal hourly dispatch results of typical days from the coupled model, we observe that, compared to the baseline (Fig. 13a–b), in 2045 for a net-zero year (Fig. 13c–d), there is a significant amount of surplus solar generation in the summer during the day and some amount of surplus wind generation in the winter during nights and days. Under a net-zero scenario, the generation from fossil fuel plants in the baseline is replaced by a battery dispatch (especially in summer) and hydrogen turbines (especially in winter), and the peaker plants, which under the baseline are turned on in the summer evening, are partially replaced by solar overcapacity and batteries. A significant share of renewable surplus energy is used for the production of green hydrogen – hydrogen made from zero-carbon electricity. Due to the complete flexibility of electrolyzers, the capture price of hydrogen production is only around one-third of the average price of electricity (Sect. S2 and Fig. S1 in the Supplement).

In winter, hydrogen turbines serve as a base load for the few days when wind generation is insufficient to meet the demand. To ensure supply during longer winter periods of “renewable droughts” with little wind and solar output, e.g., over a 2–3 d period (hours 540–600 in Fig. 13d), long-term storages with hydrogen electrolysis and hydrogen turbines and some dispatchable generation (such as CCGTs with CCS and an integrated biomass gasification combined cycle) play a major role.

Comparison of hourly generation (positive) and
consumption or storage loading (negative) for a few consecutive typical days in
two seasons in Germany in 2045.

In this section, we discuss the reasons for remaining differences between the coupled models as well as the assumptions and limitations of the soft-coupling.

In all our test runs, at the end of the coupling, it is always the case that the two models cannot be perfectly harmonized, and there is a slight residual difference in the convergence results (Sect. 4). The reason is two-fold.

The first reason is “legacy mismatch”, i.e., a mismatch in brown-field standing capacity constraints in the two models. The coupling method we develop here is mostly based on price information for achieving convergence. Therefore, capacity constraints that are present in the standalone long-term model but not in the standalone hourly dispatch model need to be transferred. These standing capacities are hard to evaluate purely based on economic terms, as they are ultimately a result of real-world actions and policies that might not align with the simplified economic incentives in techno-economic energy models. Therefore, the only way this information can be transferred from the brown-field model to the green-field model is by implementing a lower capacity bound in the latter. However, this bound nevertheless might not capture all the shadow prices caused by the standing capacities in REMIND. This is ultimately due to the specific generic form of the constraint we implemented: i.e., we pass on the pre-investment capacities as a lower bound regardless of the technology types. In general, hidden “legacy revenues”, which are manifested as the shadow prices of economically less competitive generators in DIETER, such as biomass, coal or hydroelectric (solid line lower bars in Fig. 8), provide incentives for brown-field models to deploy them over the long term but do not provide enough economic cases for the green-field model. This results in an observed phenomenon in the coupled run that, if these legacy capacities and their impact on the costs have not been fully transferred to the green-field model, the prices of the green-field model tend to be lower than the coupled brown-field models, causing distortion of the convergence of quantities. The effects of legacy mismatch and illustrative test run results are discussed in more detail in Supplement Sect. S3.

The second reason for the discrepancies at the end of the coupling is the actual mismatches in the Lagrangian harmonization itself, which can originate from multiple sources. It could be due to intertemporal constraints and dynamics (such as adjustment costs and brown-field constraints) not linearly reducible to single-year dynamics, resulting in misalignment between the multi-period REMIND and the single-year DIETER. It could also be due to slight numerical inaccuracies of the interest rate estimate that are not explicit in REMIND but that are derived from endogenous and intertemporal consumption. Lastly, there could be a mismatch due to a linear fitting of REMIND endogenous time series of fuel costs (biomass, oil, coal, uranium) before passing this information on to DIETER, which might result in a small amount of mismatch for fuel costs between REMIND and DIETER.

There are limitations to our proposed methodology in terms of both convergence of two multiscale power sector models and other potential applications of model convergence. Firstly, in terms of the problem
presented here – a multiscale power sector model coupling –, the method
derived here is only necessary for a full convergence but may not be
sufficient; i.e., a full convergence is not guaranteed. A number of
additional factors could prevent a full convergence. One is the legacy
mismatch and misalignment in Lagrangian mappings mentioned above in Sect. 6.1. Another factor is the role prefactors play (Sect. 3.3.2, Appendix H2).
The prefactors help stabilize the coupling by turning exogenous values
obtained from the last-iteration DIETER into endogenous values in REMIND, such
that they can be adjusted to be in line with the optimal mix of the current
iteration. However, they usually contain some small positive or negative
parameters that are determined heuristically (e.g.,

Since the nature of this study is a proof of concept, the scenario results
presented should be primarily interpreted as such. Nevertheless, it may be
useful to enumerate a list of limitations for a more accurate interpretation
of the results.

The power sector is only coupled for one single global region; i.e., information exchange only occurs for the variables of one region, Germany, while all the other regions contain the low-resolution version of the power sector of the uncoupled REMIND. The former coupled one-region result is based on a time series of VRE production today in a world of low- to medium-VRE shares and a very limited power grid expansion (in 2019). The latter results of the uncoupled regions however are parameterized based on results from a detailed PSM under a more optimistic assumption of transmission build-out, which allows VRE pooling from an expanded EU-wide power grid to smooth out regional weather variations (Pietzcker et al., 2017). Note that, in the standalone REMIND, while by default there are no annual electricity import and export imbalances between countries and regions, transmission during the year is implicitly assumed, especially for the EU region. Comparing the capacity and generation mixes of the coupled and uncoupled runs (Appendix J), we find that, in the uncoupled case, there are slightly more solar and wind capacities and generations and much less gas generation in the long term. EU-wide transmission expansion would pool both supply and demand variability, thus reducing the need for dispatchable capacity for meeting the peak demand.

Due to the scope of this study, we implemented a limited set of options in storage and sector-coupling technologies in this study and neglected the additional supply-side details for the German power market (such as the reserve market). Many potentially significant technological options consisting of pumped hydro storage, compressed-air energy storage, vehicle to grid and flexible heat pumps are not explicitly modeled.

Ramping costs for dispatchable generators are not considered, although the effect should be small (Schill et al., 2017).

In terms of power transmission and trading inside Germany, we assume a very simple “copperplate” spatial resolution, not explicitly modeling transmission bottlenecks inside the region. Currently, the grid capacity equation is parameterized to be proportional to pre-curtailment variable renewable generation, and the parameterization is rather optimistic based on PSM studies conducted in Pietzcker et al. (2017). As hinted at in a recent work by Frysztacki et al. (2022), the lower level of spatial detail results in an underestimation of constraints present in a real electric system, leading to an underestimation of system cost.

Near-term events: we have not modeled the current gas and energy crisis in
Europe, which is likely to imply an overestimation of near-term gas
availability in the power sector. Relatedly, we are likely to have
overestimated the early retirement of coal power plants, which are capped at a maximum of 9 % yr

Only one weather year (2019) is used for the DIETER input data. From the perspective of sufficient power supply under all weather conditions with few blackout events, this could introduce an underestimation of the need for reserve capacity, storage and demand-side flexibility.

Climate impacts under various scenarios on building sector power demand are not included in the current version of REMIND or its energy demand model for the building sector “EDGE-B” (Levesque et al., 2018). Climate extremes such as heat waves are not included in either model due to the fact that annual degree days are used, which are the results of temporal averaging. Representative weather years which maintain the temperature extremes and can represent long-term trends are also not used. However, the demand projection does change in a minor way based on SSP scenarios due to their different population projections.

“Perfect foresight” is assumed under REMIND's intertemporal optimization over several decades and therefore is also assumed under the coupled model. There exist many discussions related to the differences between the “ideal world” depicted in IAMs and energy system modeling on the one hand and “imperfect” but realistic real-world decision making and political economy on the other (Ellenbeck and Lilliestam, 2019; Geels et al., 2016; Keppo et al., 2021; Staub-Kaminski et al., 2014; Pahle et al., 2022). Considering that perfect foresight models such as REMIND dominate IPCC model results, it is especially important to understand the differences between the approaches with perfect foresight and those without it (the so-called “myopic models”). Such work has been carried out in studies such as Fuso Nerini et al. (2017) or Sitarz et al. (2023). If myopia is introduced in the model, the climate policy exemplified by carbon prices still follows an increasing expectation for more and more stringent climate policies, but the trajectory can be less smooth and in the near term looks more “flat”, hence inducing lock-in effects which slow the transition in the near term. These additional lock-in effects are not modeled in our work here.

The resulting power mix is largely due to limited options within the available energy portfolio due to Germany's energy policy and natural resources, e.g., the political decision of nuclear and coal capacity phase-out as well as limited hydro and offshore wind potential. In future research, we would like to apply the same method to all global regions.

Even though via soft-coupling an IAM can obtain hourly resolution with only a
moderate computational cost increase, it nevertheless increases the
complexity of the whole problem, increasing the solver time of the IAM,
especially before convergence is reached under the iteration with a PSM.
With additional complexity of endogenous climate policies, computational
time can be long for scenarios under climate constraints (see Appendix E).
This can be potentially overcome by several measures that can be topics for future research.

Optimize for computational costs in individual models. Individual IAMs and PSMs are usually developed incrementally, which results over time in less overall computational efficiency. However, because individually the models are not too costly to run, there are fewer incentives to manage computational costs when they are run as standalone models. However, when coupled, a computational cost may become a barrier. One of the easiest ways to reduce coupled run time is to reduce run times of the individual coupled models. Because the soft-coupling takes many iterations, a small reduction in computational time in either model will multiply to give a large reduction in iterative soft-coupled runs.

Other internal iterations of the IAM (if they exist) can be optimized. For example, in REMIND, most of the iterations (usually 30–50 iterations) in the coupled runs are dedicated to converging interregional trade between the 21 regions in the model, because the DIETER iteration usually converges quite quickly (5–10 iterations). By making the algorithm for the convergence of interregional trade faster, we can reduce the total coupled iterations, thereby reducing the overall computational cost. Less computational time can also be achieved if DIETER is no longer run together with REMIND after DIETER–REMIND iteration convergence is reached and when trade adjustment (or other internal adjustments in REMIND) is small enough to not have a substantial impact on the power sector results. This is especially the case if a PSM becomes more complex and its computational time exceeds far more than the single-iteration REMIND time (see also Appendix E for a comparison of the contributions to run time due to REMIND internal iteration and due to PSMs).

Limiting endogenous investments of capacities of certain technologies only in one model: for example, in the case of electricity transmission, more than one region (e.g., Germany with neighboring European countries) will need to be hard-coupled together in the PSM, which naturally increases the computational cost of the PSM. However, when the solutions are passed to the IAM, the regions can again be parallelized as long as an IAM does not engage in the endogenous investment of the transmission capacity. Hence the increased cost of computation due to implementing transmission is only limited to the PSM. This is also the case if within Germany the spatial resolution is increased.

Only include essential features in the PSM. Some PSMs are quite detailed and complicated for the purpose of studying specific technologies and the behavior of many agents or users. To couple to the IAM, the PSM should consider coarse-graining or aggregating some details while retaining the essence of the dynamics being studied. For example, to implement smart EV charging (e.g., vehicle to grid), modelers of PSMs should create a version for coupling that aggregates the many time series of charging and discharging of EVs to only one or two time series.

In this study, we develop a new method of soft-coupling an IAM with a coarse
temporal resolution and a PSM with an hourly temporal resolution. Our
coupling method can be shown both mathematically and in practice to produce
a convergence of the two systems to a sufficient degree. This method allows
the incorporation of the temporal details of variable renewable generation
explicitly into large-scope IAM modeling frameworks and increases the accuracy of power sector dynamics in long-term models. Furthermore, it
allows more explicit modeling of the power sector and sector coupling as well as a
vision of the energy transition where end-use demand sectors such as
building, industry and transport make economic use of the generation from
variable sources by

directly using the power at the time of production for inflexible forms of demand,

shifting the time of the power supply via battery and other power storage technology, and

transforming this into another energy carrier or product ahead of times of consumption and at times of surplus wind and solar production (e.g., PtG), without conversion back to electricity.

Coupling DIETER to the global model REMIND for the single region Germany, this study serves as a proof of concept. Our main innovation is two-fold: we derive convergence theoretically and show almost full convergence numerically. Theoretically, we derive the coupling methodology by mapping the KKT Lagrangians of the simplified versions of the two models. One key aspect of the mapping consists of iterative adjustment of the market value (i.e., the annual average revenue of one energy unit of generation) or the capture price (i.e., the annual average price of one energy unit of consumption) in the low-resolution IAM such that they take on the values like those in the high-resolution PSM. By finding the set of mathematical coupling conditions necessary for an iterative convergence as defined by the convergence of both quantities and prices, we could then design the coupling interface accordingly such that, at the end of the coupling, a joint optimal result can be found.

Numerically, we compare the converged results of the two models by examining the long-term power mix (both capacity and generation quantities) and prices of electricity and generation dispatch (via RLDCs), and we find good agreement between the two models at the end of coupled convergence despite some slight mismatches. For a proof-of-concept baseline scenario under a simple configuration without storage or flexible demand, we could achieve an energy mix with 4.4 % tolerance for any technology's absolute share difference in each time step. For a climate policy scenario under a more realistic configuration with storage and flexible demand, we could achieve 6 %–7 % tolerance. The cost breakdown and prices of power generations for both models are found to be very similar at the end of the iterative process, providing additional evidence that the quantity harmonization follows the underlying principle of price and cost harmonization. The remaining differences can be partially explained by the lack of full harmonization of the brown-field and near-term capacity constraints as well as potential mismatches due to numerical techniques aimed at enhancing performance and stability. Using the coupling methodology, we provide scenarios for power sector transition under a stringent German climate goal. Under this scenario, we observe a least-cost pathway consisting of an almost complete transformation to a wind- and solar-based power system. The results indicate an increasing role of storage and dispatchable capacity in a deep decarb scenario, consistent with the findings of previous PSM studies, but this is now transferred to the long-term models via soft-coupling.

For future works, besides expanding the research program on sector coupling into a direction containing a broader technological portfolio, we also aim to apply this framework to other world regions of interest in the REMIND model. Another important aspect would be to represent the variability-smoothing effect of transmission grids by using the same coupling framework to couple REMIND to other power sector models with more explicit modeling of transmission bottlenecks and expansion for two or more regions.

Because IAMs usually start out with certain assumptions for the development of macroeconomic metrics such as for GDP and population, which in turn determine the corresponding energy service levels to a larger degree prior to optimizing the energy system mix to meet demand, they are also frequently referred to as “top-down” energy system models. PSMs usually start out modeling the fine spatiotemporal detail of real-world power systems, expanding the capacity installation of power-generating plants, grid transmission and storage at minimum cost. Such models are also known as “unit commitment models” for electrical power production (Padhy, 2004). Later in model development PSMs are usually expanded to include other energy services such as heating and transportation that are electrified. In this way PSMs are also often referred to as “bottom-up” models. Reviews and intercomparisons of IAMs have been carried out recently where various IAMs are analyzed and harmonized (Weyant, 2017; Butnar et al., 2019; Keppo et al., 2021; Wilson et al., 2021; Giarola et al., 2021). For methodological reasons, we have to set the length of the model time horizon to be until 2150, which is longer than the valid model time horizon until 2100. This is because without the extra years after 2100, the model has much less time to utilize the capacities installed in the few decades before 2100, making it more difficult to justify the installation of new capacity economically. This is manifested in a model artifact where, in the last few model periods, investment in capacities decreases in general. By extending the time horizon, this “boundary” effect is pushed further into the future, so the artifact only appears after 2100. Therefore the meaningful model results for REMIND are only between 2005 and 2100, even though the years until 2150 are also modeled and coupled.

Reviews and intercomparison of typical scopes and resolutions of PSMs can be found in Supplement Sect. S5. Comparison of more PSMs can be found in Ringkjøb et al. (2018) and Prina et al. (2020).

Both models have open published source code. Partially thanks to the PSM community's advocacy of “open models”, which encompasses all steps from input data to model source code and numerical solvers (openmod – Open Energy Modelling Initiative, 2022), many research institutions have also responded to their calls to openly publish their models. For example, the IAM used in this study, REMIND, has for 2 years opened its source code on the popular hosting site GitHub.

Comparison of the

Comparison between the coupled models REMIND and DIETER.

While REMIND and DIETER can both model a European-wide system with spatial subdivision (see Fig. B1 for the REMIND regional division), the soft-coupling is currently only applied to Germany, in line with the proof-of-concept nature of this study. The coupling is from 2020 to 2150 for every defined REMIND period. All common and available REMIND-generating technologies are enabled for the coupling, as shown in Fig. B2. The information for the species of technologies in REMIND is upscaled and coupled to DIETER, whereas information from DIETER is then downscaled during the feedback loop that completes the coupled iteration.

REMIND regional resolution used in this study (21 global regions, including detailed differentiations of EU regions). The spatial resolution of REMIND is flexible and depends on the resolution of the input data. Regional mapping is from the REMIND–EU model (Rodrigues et al., 2022).

Mapping of coupled technologies between REMIND and DIETER.

Single-region interannual intertemporal welfare is an aggregated utility,
which in turn is a logarithm function of consumption. In REMIND, the total
welfare of a region is maximized and is equal to

In Sect. 3.2.1, we sketch the derivation procedure and offer a short summary of the analytical results. Here we describe the derivation procedure of the coupled convergence framework in detail.

Using the Lagrangian multiplier method, based on the objective functions
(Eqs. 1–2) and constraints (c1–c6) in Sect. 3.1, we can construct the KKT
Lagrangians (Karush, 1939; Kuhn and Tucker, 1951; Gan et al., 2013).
REMIND:

DIETER:

Comparing Lagrangians

After the aforementioned simplifications, we can construct a single-year
REMIND Lagrangian

These are the same as Eqs. (3)–(4).

Comparing

annual total power sector costs:

annual revenue of usable (post-curtailment) generation for each generator

annual payment made by the consumers:

maximum generation from the capacity constraint term for each generator

Term (A) can be mapped if

annual fixed costs are harmonized for each generator species

annual variable costs are harmonized for each generator species

for each generator species

The detailed derivation is as follows.

Lagrangian term (B) for the models has the physical meaning of the total annual
revenue of usable (post-curtailment) generation. (Annual revenue is equal to
the product of usable generation and the annual market value.) We denote total
annual revenue from technology

We now take a look at revenue

After adding the linear correction term, the modified revenue in REMIND

In practice, in the case of the annual shadow price

It is not hard to recognize

Now we have concluded the derivation for the markup term

Although the multiplicative terms in front of the decision variables in the two
models can be harmonized via the correction term (D12), we notice that it
contains endogenous values, i.e., hourly generation

The coupled REMIND has a modified objective function

Now we have concluded the derivation for the convergence condition (h3).

Depending on the starting point of the REMIND power system, and due to the
internal iterative changes in REMIND results due to the adjustments in trade
between regions in the Nash algorithm, coupled convergence usually can only be achieved over multiple iterations. Therefore the derived markup
equation (Eq. D12) in general can only be expected to reflect the actual
market value differences approximately in the two models. This is the reason
why, in the iterative algorithm after the first iteration, we add

annual power demands in the two models are harmonized:

annual average prices of electricity are mapped to each other:

annual average capacity factors are harmonized, i.e.,

For dispatchable generators the argument is as follows. (For simplicity, we
use the generic index

We first rewrite REMIND term (D) by plugging in the harmonization condition

Splitting the two terms, these four terms need to be harmonized for all

To show that the mapping Eqs. (D13)–(D14) are automatically satisfied given (h6), we
first consider two simplified power sector toy problems, Q1 and Q2, with
only dispatchable technologies. Both problems have identical objective
functions

Q1: minimum

Q2: minimum

Then the Lagrangians are the following:

The relevant KKT conditions are as follows.

Stationarity condition for Q1:

We now show that Eq. (D14) is automatically mapped given capacity factor
harmonization (h6). We first write the equality condition for the
REMIND–DIETER case, analogous to the toy model result (Eq. D17):

Plugging (h6)

Plug Eq. (D22) and then Eq. (D18) into the right-hand side of Eq. (D21) to obtain

For VREs, annual curtailment rates are harmonized,

For VREs the derivation is conceptually similar to the above case for
dispatchables in (h6), since we can define a real capacity factor (post-curtailment) similar to the capacity factor for the dispatchable
generators above:

Coupled region: Germany

Coupled REMIND time horizon: 2020–2150 (years 2010–2015 are not coupled since they are historical years and mostly have hard-fixed quantities).

A graphic description of the model iterative coupling.

Under a simple configuration (no storage, no flexible demand), every REMIND run takes around 3 min and a DIETER run takes a few seconds to solve. Under more detailed configurations (with storage and flexible demand) and climate policies, every REMIND run takes around 4 min and a DIETER run takes a few minutes to solve. The entire REMIND–DIETER coupled run for a single region, Germany, under a simple configuration is around 3–4 h. It is around 6–10 h for the more detailed configurations under climate policies.

In this Appendix, we discuss the equilibrium conditions of the uncoupled models, resulting in a rigorous formulation of the so-called “zero-profit rules” (ZPRs). We first construct the Lagrangians and compute KKT conditions and then derive the ZPRs for the standalone versions of the REMIND-reduced power sector model and the DIETER model.

Using the objective functions and constraints in Sect. 3.1, we can construct
Lagrangians for the two standalone models. Using the KKT conditions derived
from the Lagrangians, we can show that, if the historical and resource
constraints are non-binding, i.e., shadow prices

Here we first construct the Lagrangians and derive the KKT conditions from them (Sect. F1) for both models. Then, both models' ZPRs are derived, two for each model, i.e., the technology-specific ZPR and the system ZPR (Sect. F2).

The Lagrangians of the uncoupled model have been constructed in Appendix D (Eqs. D1–D2). From the KKT conditions for minimization, we can ascertain the following first-order conditions at stationarity for each model.

REMIND

Stationary conditions:

Complementary slackness:

Primal feasibility:

Dual feasibility:

Stationary conditions:

Complementary slackness:

Primal feasibility:

Dual feasibility:

The derivation of ZPRs is very similar to the one in Brown and Reichenberg (2021). Starting with the total costs for technology

Normally when there are no capacity shadow prices or when the capacity
constraints are not binding, the cost exactly equals revenue. However, when
capacity shadow prices are non-zero, i.e., the constraints (c2) and (c5–c6)
are binding, the capacity shadow prices act as a distortion of the equality
relation between costs and revenues. As an example, the shadow price

One observes that, from the right-hand side of Eq. (F20), there is no
differentiation between the annual market values of variable and
dispatchable generations such as gas and solar – they are both equal to the
annual electricity price

From Eq. (F20), we can derive a ZPR between the levelized cost of electricity
(LCOE), capacity shadow price and market value (MV) for each generator
type. Taking Eq. (F20), we separate the pre-curtailment LCOE from the LCOE
due to curtailment and then divide it by the total post-curtailment generation

The pre-curtailment LCOE is the cost of one unit of generated electricity, regardless of whether it is curtailed or being used to meet demand, whereas the curtailment LCOE is the cost of one unit of curtailed electricity. Together they add up to the post-curtailment LCOE, i.e., the cost of one unit of usable electricity.

To obtain the ZPR for the whole power system in REMIND, we first sum Eq. (F20) over all generator types

The ZPRs of REMIND hold for the aggregate over multiple years.

From Eqs. (F21) and (F22), we learn that when a market equilibrium can be found, i.e., when the optimization problem can be successfully solved, there is an equality relation between the generation cost and market value for each generator type and similarly between the generation cost and price of electricity for the entire system. Capacity shadow prices due to various extra capacity constraints imposed on the models distort the equality relation between costs and prices by a linear term, making the prices either higher or lower than the costs at the market equilibrium.

Similarly to the uncoupled REMIND, from KKT conditions, at stationarity, we can
obtain the cost–revenue ZPR for a single technology

Rearranging, we arrive at the ZPR of the single-year uncoupled DIETER for a
technology-specific cost–revenue balance:

Summing Eq. (F24) over

Similarly to the case of REMIND, Eqs. (F24)–(F25) show us the equality relations between the cost and value (or price) for each generator type and for the system hold (also for DIETER at its market equilibrium). Compared to REMIND, there are no brown-field or near-term capacity shadow price contributions in DIETER in the standalone versions. The DIETER ZPRs hold for 1 year instead of the aggregate of multiple years like in REMIND. For simplicity, even though it is possible to write the LCOE in pre-curtailment and curtailment terms, for DIETER it is relatively cumbersome to do, and we do not do it here.

In summary, at the REMIND and DIETER power market equilibriums, each generator exactly recovers its cost of one unit of generation through the market value and obtains zero profit in a completely competitive market over its modeling time. In the aggregate, the entire power sector obtains its cost of one unit of generation through the price of electricity that the consumer pays. Both types of relations can be distorted by the existence of capacity shadow prices.

Here, in this Appendix, we gradually build up the derivation for the ZPRs of
the coupled REMIND and DIETER, which will be used later to validate numerical results. The derivation consists of three steps.

ZPRs for the uncoupled models REMIND and DIETER

ZPRs for the coupled models REMIND and DIETER (simplified version only considering convergence conditions h1–h7)

ZPRs for the coupled models REMIND and DIETER (full version also considering c7 and c8)

For step (2), based on the uncoupled ZPRs, we recognize that, from
convergence conditions (h1)–(h7), the only condition which impacts the form of
the ZPR is (h3) because the markup terms modify the objective function of
the (simplified) coupled version of REMIND (Eq. 6). Following a similar
procedure to Appendix F, we can derive the technology-specific ZPR for
the coupled REMIND (simplified version) as follows:

Step (3) involves two extra capacity constraints, one in each model, the first of which, (c7), is discussed in detail in Appendix H. The implementation of (c7) further modifies Eq. (G1) and results in the ZPRs of the coupled REMIND. The other constraint, (c8), will be the focus of discussion here. It only modifies the ZPRs for the coupled DIETER and not for the coupled REMIND.

Constraint (c8) is a brown-field capacity constraint implemented in DIETER
to address the fact that DIETER is a green-field model that is otherwise
ignorant of standing capacities in the real world. There are many ways in which we can implement this standing capacity constraint in DIETER. The most
straightforward way is to implement the standing capacity at the beginning of each REMIND period, which REMIND sees before it invests
additional capacities, as a lower bound on endogenous capacities in DIETER.
This helps put DIETER and REMIND on an equal footing before the 5- or 10-year
investment period starts, allowing us to compare their investment
intentions.

Standing capacity constraint in DIETER: i.e., DIETER capacities at
time

To obtain the ZPRs of the coupled DIETER, we simply modify the capacity shadow
price term of the uncoupled DIETER ZPRs (Eqs. F24–F25) by the additional
capacity shadow price

Here, we introduce the two methods we employed to improve the numerical stability of the coupled runs: (1) the dispatchable capacity constraint by peak demand to avoid high markups being exchanged (Sect. H1) and (2) endogenous prefactors for all quantities from the last-iteration DIETER to the current-iteration REMIND (Sect. H2).

The scarcity hour price can occur in a PSM run, which is the highest hourly price in a year, and it is usually equal to the annuitized fixed cost of OCGTs (capital investment cost and fixed O&M costs) (Hirth and Ueckerdt, 2013). In our simulations, the scarcity prices are usually above USD 50 per kilowatt hour. If we include the scarcity price in the markups, OCGTs will receive an annual markup usually more than 5 times higher than the annual average electricity price. The high markup results in OCGT plants receiving too high an incentive in the next-iteration REMIND, and the model overshoots (overinvests) in capacities. Over iterations, this causes oscillations in the quantity and prices in the coupled model. For better numerical stability, instead of passing on the full markups from DIETER, we only pass on the portion of the annual markups unrelated to scarcity hour prices and replace the exchange of the part of the markup due to scarcity hours from DIETER to REMIND by implementing an additional capacity constraint in REMIND for coupled runs. The two actions can later be shown to be mathematically equivalent. Generators other than OCGTs which produce at the scarcity hours also get paid in the hour at this high price. However, because they also produce at other hours with lower prices, their average market values are only moderately impacted by the scarcity hour price and do not in general lead to instability issues.

Below, we first introduce the aforementioned capacity constraint implemented on the side of REMIND and then discuss the corresponding manipulation of the markups in DIETER. Lastly, we show their mathematical equivalence and state the modified ZPR of the coupled REMIND due to these actions.

The extra capacity constraint states that the sum of all dispatchable
capacities needs to be at least as large as the peak residual demand:

For the exact implementation of (c7) in the coupled run, see Sects. 3.3.2, point 2. Under storage implementation, in addition to the variable renewable contribution, the hourly storage discharge is also subtracted from the residual demand.

At the same time as implementing this capacity constraint, we remove the surplus scarcity prices in postprocessing of DIETER before passing it on to REMIND. In DIETER, we define the scarcity price as the maximum hourly price in a year,Using this, we manipulate the market value and the annual average electricity
price in DIETER ex post, excluding the surplus scarcity hour price:

With the implementation of (c7), the coupled ZPR (Eq. G1) is then further
modified to include the new shadow price

These are the ZPRs of the coupled REMIND for the full version.

Because of the intuitive relation between the scarcity price and the peak
residual demand, i.e., that the scarcity price occurs in the hour with peak
hourly residual demand due to the pricing power of the peaker gas turbines
in the hour where VREs are most scarce, we can make a quantitative equivalence
between the scarcity price contribution to the markup and the capacity
constraint shadow price

Using this equivalence, we can show as follows that, at convergence,

At convergence, the annual generations have identical solutions in the two
models, i.e.,

Since the single-year revenue

In this Appendix, we describe the detailed implementations of prefactors for
information exchanged from DIETER to REMIND.

Markup prefactor

In order to facilitate convergence in REMIND, we implement an endogenous
prefactor

When the in-iteration REMIND solar generation share increases due to the price
signal from the last-iteration DIETER market value, such that the REMIND
share is larger than in the last DIETER iteration, the formula Eq. (H13)
results in a prefactor smaller than 1, decreasing the in-iteration markup

Peak demand prefactor

The peak demand in REMIND

Capacity factor prefactor

We set the REMIND capacity factor

The sign in the prefactor formula is determined based on the observation that, under a system with variable renewable generations, for generator plants that have relatively high running costs and low investment costs, i.e., most economically operated as “peaker” plants or as “mid-load” plants of a lower capacity factor, when their generation share incrementally increases, their capacity factor decreases. Conversely, for generators with relatively low running costs and high investment costs, i.e., most economically operated as “base-load” plants, when their generation share incrementally increases, their capacity factor increases.

Curtailment prefactor

The curtailment ratio in REMIND

Capture price prefactor

Similar to the case of markup from the demand side, the markup for any
demand-side technology given to REMIND is

Adjustment cost, an additional linear term in the objective function, acts
as an inertia against fast or slow capacity additions or retirement. The
implementation of positive adjustment costs mimics the challenges of scaling
up the supply chains and training new workers to do installation and
construction. Adjustment costs are applied to all model time periods, so it
is by nature intertemporal. The objective function for the power sector
including the adjustment cost

Similar to the case without adjustment costs in Sect. 3.2.3, the first
stationary condition becomes

The new ZPR including the adjustment cost in terms of cost and revenue for
technology

In the coupled run we see only a moderate adjustment cost which drops down quickly as a function of time (see, e.g., Fig. 6).

Under the 2

The units used in the two models are usually different. Here we uniformly
use megawatt hour for energy units and megawatt for capacity units. In the main text,
overline

Complete list of mathematical symbols. For simplicity, in general, we only list the symbols, not their indices or in which model they are used.

Complete list of abbreviations.

The coupled and uncoupled REMIND codes are
implemented in GAMS, and the code and data management is done using R. The
coupled and uncoupled DIETER models are entirely implemented in GAMS. The
default uncoupled REMIND v3.0.0 code is available from the GitHub
website at

The supplement related to this article is available online at:

Methodology development was done by CG, FU and RP. CG designed and carried out the numerical implementation and performed the theoretical analysis of the methodology. The methodology was first conceptualized by GL. Supervision and funding acquisition were carried out by FU and GL. OA participated in the development of model postprocessing and the overall structuring of the manuscript. MK and WPS performed theoretical and conceptual validation of the manuscript. CG prepared the manuscript with contributions from all the co-authors.

At least one of the (co-)authors is a member of the editorial board of

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The authors thank Tom Brown at the Technical University of Berlin as well as Marian Leimbach, Renato Rodrigues, Nico Bauer, and Christof Schoetz at the Potsdam Institute for Climate Impact Research for discussion.

This research has been supported by the Bundesministerium für Bildung und Forschung (grant nos. 03SFK5N0, 03SFK5A and 01LP1928A) and the Deutsche Bundesstiftung Umwelt (PhD scholarship).The article processing charges for this open-access publication were covered by the Potsdam Institute for Climate Impact Research (PIK).

This paper was edited by Sam Rabin and reviewed by two anonymous referees.