The origins of meteorological observation can be traced back to ancient civilizations, where people began to notice patterns in the weather and celestial phenomena. However, it was not until the 17th century that systematic weather observations began in earnest with the development of instruments such as the mercury barometer and the thermometer by scientists Evangelista Torricelli and Daniel Gabriel Fahrenheit . An early example of this interest in observing the atmosphere using instruments is the Central England Temperature (CET) record , which is one of the longest instrumental temperature records in the world, dating back to 1659. It provides a continuous monthly temperature series for the central England region and is often used as a proxy for temperature variations in western Europe. Other examples of early weather monitoring date back to the 18th century, such as the Paris–Montsouris observations in France , the Zentralanstalt für Meteorologie und Geodynamik in Austria (Vienna) , the Uppsala University observations in Sweden , and the earliest observations recorded for the Iberian Peninsula like those starting in Seville (Spain) in 1780 . Since these first observations began, the number of surface meteorological observatories worldwide has increased significantly, as shown in Fig. a.

The critical role played by surface meteorological stations in climate monitoring and research is emphasized by the Intergovernmental Panel on Climate Change (IPCC) in all its assessments and reports . One of the aspects addressed is the need to maintain high-quality and consistent data following high standards of quality assurance and control . These kind of procedures are essential to ensure that the data collected are homogeneous, accurate, and reliable. Errors or inconsistencies in the data can lead to erroneous climate assessments and predictions . Another important fact mentioned is the need for a dense network of surface meteorological stations around the globe to provide comprehensive coverage of different regions and climates. Dense monitoring networks are less common in remote or less densely populated regions or where the environmental conditions are too harsh to operate and maintain the instruments .

It has been commonly accepted that surface meteorological stations are still the best way to identify long-term trends and variability in climate. They have also proven to be critical for validating and calibrating other atmospheric databases such as those obtained from satellites or remote sensing instruments . They are also a key element for the development and validation of gridded databases obtained by numerical models such as reanalyses . More recently, meteorological measurements have become an essential element of machine learning methods applied to atmospheric modeling.

Due to the important role played by surface observations in climate assessing and weather forecasting, several countries established and expanded their surface meteorological observatories during the 20th century, trying to cover as much territory as possible , from several thousand surface stations at the end of the 19th century to several tens of thousands at the end of the 20th century (Fig. a). However, as mentioned before, the results are very uneven around the world (Fig. c), with important areas of the world still under-monitored.

Contrary to what might be expected, the number of surface meteorological stations has not increased in recent decades at a rate that would fill the gaps in the documented under-monitored areas. Rather, the ratio of operational stations has slowed down and decreased since the 1970s (Fig. a). As can be seen, the number of decommissioned stations has increased in recent decades in many regions of the globe, disrupting some historical climate time series.

One potential explanation for this decline in the number of surface stations is the advent of satellites as a novel method for observing the weather and climate. Following the launch of the world's first weather satellite, TIROS-1, in 1960, satellite weather observations became prevalent and began to offer a number of advantages over on-site weather observations, as cited in . For instance, they permit global coverage and cost-effectiveness since they do not necessitate an extensive network of ground-based weather stations to cover vast areas. However, satellite weather observations also have limitations. For example, they have difficulties accurately measuring conditions at the Earth's surface, their data availability is highly dependent on cloud cover, they often exhibit long-term instrument drift, and they have calibration issues in remote areas where surface observations are still unavailable.

Another potential factor contributing to this decline is the increasing use of model reanalysis data to conduct climate research . Model reanalyses employ a combination of observational data sources, including in situ surface weather observations, satellite data, and others to generate a gridded and consistent dataset of weather and climate information from the past. The resulting datasets are comprehensive, are homogeneous, and have strong climatological consistency. They cover global areas, enabling analysis where in situ data are not accessible. In many cases, reanalyses are used instead of in situ measurements for climate studies. They are certainly useful for studying broad climate patterns and long-term climate trends, and they could theoretically be used to fill gaps or extend the temporal and spatial coverage of observations. However, they suffer significant losses with regards to temporal and spatial resolution, as well as information relating to local phenomena. Global reanalyses have inherent difficulties in providing fine-scale details that are often missed in the physics of the models or are meaningless at the low resolution considered.

Although it may seem that global weather data are fully accessible through the more precise reanalyses available nowadays, there may be a hidden loss of information about local phenomena that only surface weather stations are able to capture. When historical meteorological data are not continued indefinitely or interrupted, many of the resources invested over decades are lost. In addition to these interrupted time series, there are also numerous surface meteorological observation series of good quality around the world as a result of short-term campaigns or very recent initiatives. These time series also provide a wealth of information on local processes, but their short duration is still insufficient for climatological analyses .

We have global reanalyses that span the entire 20th century and part of the 21st century, with low resolution and limited phenomena. However, these datasets are sufficient to consider the main drivers that force weather and climate at the surface. In contrast, we have sets of interrupted or recent surface measurements that capture local phenomena, but these are too short to conduct climatological analyses. It thus appears feasible to downscale reanalysis data in order to obtain pseudo-observations that can provide the best of both worlds.

Downscaling has been performed since the inception of reanalysis. Two general approaches to downscaling are dynamical downscaling (DD) and empirical statistical downscaling (ESD). ESD relies on observational data to establish empirical relationships between the large-scale fields provided by the reanalysis and the local phenomena seen in the observations . These are grouped into model output statistics, perfect prognosis, and weather generators (in which analog models are used). On the other hand, DD is achieved by using higher-resolution physical models that account for lower-scale phenomena nested within the reanalysis fields .

Numerous papers scrutinize the advantages and disadvantages of the various methods . As a rule of thumb, empirical techniques are generally less computationally intensive than physical downscaling and may yield better results at a lower cost. However, empirical downscaling is only feasible when a sufficiently long and uniformly collected dataset of observations is available. Assuming the hypothesis that there is a connection between the large-scale phenomena shown by the reanalysis and the local phenomena captured by the observations is also essential.

Regardless of the chosen downscaling method, combining reanalysis and surface observations to create long and homogeneous time series requires a significant amount of effort. Setting up a dynamic regionalization system can be expensive in terms of both computation and human resources, but even a relatively simple statistically based regionalization method entails a learning curve that may discourage or slow down certain climate studies.

This work introduces and explains RASCAL v1.0 (Reconstruction by AnalogS of ClimatologicAL time series), an open-source tool for climatological time series reconstruction and extension using ESD. The primary goal of RASCAL is to promote and accelerate rigorous climate research in regions where surface meteorological observations are insufficient for climate analysis and where relevant regional and local meteorological processes can only be captured through in situ observations. RASCAL could prove highly beneficial for mountain climate research and other areas with unique microclimates, such as river valleys, forests, caves, or canyons.

This study is organized as follows: Sect. 2 provides a detailed description of the implemented method, while Sect. 3 describes the model structure and implementation. In Sect. 4, we evaluate the performance of the package by downscaling the daily maximum and minimum temperatures and precipitation of four stations near a mountainous region in central Spain. We draw final conclusions and important remarks in Sect. 5.

RASCAL is based on an ESD type known as an analog model or weather generator. This is a widely used technique in climate research . It is based on the premise that large-scale atmospheric conditions tend to produce comparable local weather patterns, allowing the prediction of local conditions for a day without real-time observations. This is done by identifying an analog day from general circulation models (GCMs), such as reanalyses, and assigning its local conditions. This approach allows the study of climate variability over an extended time frame, providing valuable perspectives on long-term patterns and connections between different geographic locations, while also incorporating important local factors in the analysis .

The analog method is a nonlinear technique that relies on the identification of strong statistical relationships between two fields: the predictor variable extracted from GCM products and the predictand variable obtained from local historical observations. To predict an atmospheric feature (the predictand) for a given day, this method searches for the day with the most similar predictor field in the historical record and uses its atmospheric features to make a prediction, allowing the reconstruction of missing data points .

To incorporate the relationship between large-scale meteorological patterns and local weather, the analog method is often combined with principal component analysis (PCA). PCA reduces the high dimensionality of the atmospheric phase space by generating an orthogonal basis of vectors that represent the main directions of variability. As a result, only a limited set of m coefficients, called principal components (PCs), are required to represent the atmospheric state . The resulting set of m coefficients of the PCs at time t are considered the predictor X(t)=(X1(t),X2(t),…,Xm(t)) for the local predictand Y(t). For a given day t, the objective is to identify N historical days t1,t2,t3,…,tN such that the predictor X(t) is similar to the predictors X(ti). The similarity between the predictors in the historical record X(ti) and the predictor of the day to reconstruct X(t) is measured with the Euclidean distance.

1dX(t,ti)=‖X(t)-X(ti)‖

The N days with the smallest distance dX(t,ti) constitute an analog pool. Various similarity methods can be used to select the best analog Y^(t) or group of analogs from the pool. The most straightforward method is to choose the day ti that has the smallest distance dX(t,ti) and that is the closest day in the PC space. However, similar synoptic patterns can sometimes produce different local weather if the role of other variables or more complex phenomena is not taken into account. Therefore, assigning the day with the most similar predictor field pattern as the analog day may not always be accurate. To avoid making the reconstruction method too complex, one solution is to select the N closest days from the pool of analogs instead of just one. Then, perform a weighted average of the predictand Y(ti) by the square of the distance in space of the PCs of those days: 2Y^(t)=1N∑i=1NwiY(ti), where the weight wi is 3wi=1dX(t,ti)2∑j=1N1dX(t,tj)2.

This way, the days with a more similar synoptic pattern have more presence in the average, while also considering possible phenomena that have not occurred on the closest day but on the other similar days.

Averaging can impact the distribution of the variable by smoothing the data and removing extreme values. To preserve extremes while still accounting for possible phenomena beyond the similarity of synoptic patterns, bias reduction methods such as quantile mapping can be used. This technique employs a “mapping variable” Z(t) and examines the quantile of the day to reconstruct Q(Z(t)) in the distribution of this variable in the analog pool (Z(t1),Z(t2),…,Z(tN))). The method first examines the distribution of the predictand variable in the analog pool (Y(t1),Y(t2),…,Y(tN))) and selects the day ti that occupies the same quantile as the best analog.

4Y^(t)=Y(ti),whenQ(Y(ti))=Q(Z(t))

This approach improves the representation of extreme events. However, it is crucial to note that the mapping variable must have a strong correlation with the predictand variable. One possible solution is to use the reanalysis predictand variable as the mapping variable.

The analog method fundamentally involves the reorganization of observed time series data, aiming to maintain the statistical characteristics of the original dataset. The efficacy of this method relies on ensuring that the downscaling and training periods exhibit a comparable climatic context . The reconstruction capability of the analog method is constrained by the temporal extent and accuracy of historical observations. This means that it cannot replicate unobserved events and, therefore, cannot reproduce new record values in the context of climate change. However, its utility becomes pronounced in scenarios where external climate forcing induces shifts in the frequency of observed phenomena. In essence, it serves as a valuable tool for discerning alterations in the occurrence patterns of documented events.

Empirical downscaling techniques involve laborious steps that must be carefully addressed to ensure the quality of the local climate series reconstructions, as pointed out in . RASCAL is a Python library that implements the analog method in a clear and simple way. It is an object-oriented library with four main blocks or classes: Station, Predictor, Analogs, and RSkill. This library is a valuable complement to other empirical downscaling libraries, such as pyESD from , which is based on machine learning downscaling methods that focus on generating monthly time series. RASCAL is based on classical statistical methods, which produce results that are easier to interpret physically, and additionally, it is more focused on daily-resolution reconstructions rather than monthly, which allows for the calculation of relevant daily climate indices. This section describes these components and their implementation workflow, as illustrated in Fig. .

To test RASCAL performance, we tested the skill of the reconstructions of maximum and minimum temperature and daily precipitation at four different surface stations in the vicinity of the Central System of the Iberian Peninsula (Spain), as shown in Fig. . This mountain range is of vital importance as it is the main contributor to the hydrological resources of central Spain due to the high levels of rainfall and snow runoff in spring. This area has been the subject of several studies by authors in recent years . The reader can refer to these previous works in order to understand the importance of having long time series in this area.

We have confirmed that a decline of in situ observations is a noteworthy concern as it may result in the loss of crucial information in areas where local effects are relevant to climatology. While the reanalysis provides a homogeneous and comprehensive dataset, its applicability to studying climatologies with highly localized effects, particularly in regions with intricate orography, has been called into question.

In order to mitigate this possible loss of meteorological information based on surface observations, RASCAL has been developed. This is an open-source Python tool designed to fill gaps in observational data, enabling climate studies in regions with limited long-term data. This tool proved to be particularly useful for the test sites, especially in the mountainous areas. It is expected to also be useful in other areas with important local effects or distinctive locations like river valleys, forests, caves, or canyons.

The package presented here utilizes an object-oriented programming (OOP) approach, treating weather stations, predictors, and reconstructions as objects with multiple functional attributes that encompass all necessary functionalities. This has allowed for the execution of all modeling steps with just a few lines of code. The core methodology is based on linking large-scale circulation patterns with local atmospheric features. This linkage is established through the analog method and principal component analysis and has been shown to be more effective than reanalysis in conveying climatic characteristics. It is a faster and less computationally expensive alternative to dynamical downscaling methods and an easier-to-interpret method than machine learning statistical downscaling methods.

The package was evaluated at four stations in Spain, including three near a mountainous area in central Spain and one in a highly urbanized area. The results were compared to the products of the reanalysis ERA20C and ERA20CM. RASCAL outperformed the reanalysis in terms of R2, standard deviation, and bias. This improvement was particularly noticeable in the reconstruction of monthly total precipitation, with correlation values reaching 0.8. The reconstructed maximum and minimum temperatures show a slight improvement over the reanalysis in terms of standard deviation and correlation, reaching very high values of correlation of over 0.99. Additionally, the biases present in the reanalysis are significantly corrected by the reconstructions. This is also evident when examining the distributions of daily data. RASCAL is proficient at generating series that closely resemble the observations, unlike the reanalysis, which exhibits skewness towards low precipitation and biases in maximum and minimum temperatures.

The various methods for selecting the best analog have exhibited diverse behaviors when examining the different characteristics of the series. Therefore, it is recommended not to designate a single method as the best possible but to choose it based on the scientific objectives.

Seasonality also demonstrates a marked enhancement compared to the reanalysis. RASCAL produces reconstructions with an annual cycle closely resembling the observations. While the precipitation annual cycle exhibits some disparities during unstable months, such as November and March, the cycles of maximum and minimum temperatures are nearly identical to the observations in every month when using the quantile map method. This method better represents the monthly variability for both precipitation and temperatures.

In climate studies, the use of indices is a common practice to condense key climatic features of a study area. RASCAL has demonstrated its capacity to reproduce indices like days of precipitation above 1 mm, icing days, and frost days well for a station situated in the core of a mountain range. This achievement is particularly noteworthy given the difficult conditions posed by the strong dry and warm biases of the reanalysis in this region, which would otherwise hinder the accurate computation of these indices. The reconstructed data showcase high correlation coefficients with observations, ranging from 0.8 to 0.96. Additionally, consistently low values of MBE and RMSE were observed. These outcomes highlight the significant potential of RASCAL to facilitate climate studies in regions with complex climatic dynamics. These results confirm RASCAL's effectiveness in capturing and reproducing important climatic features for reliable climate research, highlighting its potential in regions with limited long-term weather data.

However, it is important to acknowledge instances where this methodology may have limitations. This approach requires sufficiently long series, as it cannot create reconstructions with data that have not been observed. Additionally, land use changes or urbanization processes can disrupt the intricate relationship between large and small scales, affecting the relationship between predictor and predictand and, ultimately, the quality of the reconstruction.

The implementation of this package has yielded positive results, providing opportunities to conduct comprehensive climate variability analyses within the study area. In a short time, it is expected that we can use RASCAL in the analysis of the climate variability and climate change in the mountainous area of the Central System (Spain). On the other hand, improvements to be implemented in this methodology will be studied once it has been applied to different cases, scenarios, and regions. Finally, whether this package can be extended as a downscaling tool for short- and medium-term numerical forecast, as well as for seasonal prediction and even climate change scenarios, will be analyzed.