For our study we utilize Level 3 (L3) sea surface temperature (SST) satellite observation products. L3 level of product refers to the satellite product where spatially sparse and irregular point observations of the ocean surface are gridded into a fixed grid across space and/or time. Such products may combine multiple satellite overpasses or even multiple sensors for the same observed quantity.

Specifically we consider the following three datasets corresponding to three different geographic regions:

Central Mediterranean: The SST_MED_SST_L3S_NRT_OBSERVATIONS_010_ 012_a dataset contains daily near-real-time (NRT) SST measurements over the Mediterranean sea from 1 January 2008 to 31 December 2021. The dataset is provided on a remapped grid with a spatial resolution of 0.0625° × 0.0625°.

These regions were chosen due to their oceanographic variety. The Adriatic is an elongated semi-enclosed basin with correspondingly poor satellite coverage, and the Central Mediterranean exhibits a wide variety of oceanographic regimes (from regions of freshwater influence in the northern Adriatic to a much deeper Ionian where Levantine and Adriatic water masses communicate), while the Atlantic region is essentially an open ocean region, very different from the Adriatic. These regions should demonstrate generalization abilities of CRITER under a variety of oceanographic conditions. The geographic areas of the three datasets are shown in Fig. . It is worth noting that two different satellite products are used in this study, a near-real-time (NRT) and a multi-year (MY) reprocessed dataset. This was done to show that like DINCAE2, CRITER also generalizes well across various datasets of SST. Furthermore, multi-year reprocessed datasets come at a higher resolution and span significantly longer periods of time, which gives access to a larger train and, more importantly, test set.

The Coarse Reconstruction Module (CRM, Fig. ) follows the ViT encoder–decoder architecture , similar to spatio-temporal MAE . The input observation fields Xm=[xt-1,xt,xt+1]∈R3×1×W×H are first fed to a tokenization process. To encode information about the yearly temperature cycle, each observation field xt is concatenated channel-wise with a day-of-the-year auxiliary tensor at=[sin⁡(dt2π365),cos⁡(dt2π365)]∈R2×W×H, where the two channels contain constants, and dt is the numerical day of year index (between 1 and 365). The resulting fields are split into non-overlapping 3×8×8 patches, which are then flattened and linearly projected into tokens of shape 1×Dt, where Dt is the dimension of tokens used in ViT blocks, thus creating the list of tokens T={Tr,Tc}. Tokens Tr correspond to patches in xt with at least one unobserved pixel and thus have to be reconstructed. Tokens Tc are the remaining tokens, and they are used as a context for reconstruction. To encode the extent of missing values in a token, all tokens in xt are summed with their corresponding mask tokens. These are obtained by splitting the binary mask indicating missing pixels Mt∈{0,1}W×H into 8×8 non-overlapping patches, which are then flattened and projected into mask tokens of shape 1×Dt. To maintain the necessary spatio-temporal location of each token, all tokens in T are summed with a spatio-temporal positional embedding as in .

After obtaining tokens T, the context tokens Tc are encoded by a ViT encoder EViT into TcE (Fig. ). Then, the list of tokens Tr requiring reconstruction is concatenated with the list of the encoded tokens TcE. The set of all tokens is again summed with the spatio-temporal positional embedding and passed through a ViT decoder DViT, producing the decoded tokens TD. The decoded tokens not corresponding to the central observation xt are removed from TD, resulting in TtD. Tokens in TtD are then linearly projected into 1×82 vectors and reshaped into 1×8×8 patches. Finally, the patches are reassembled into a grid to form the coarse reconstruction x^t. All pixel values corresponding to land areas are set to zero using the land mask Ml∈{0,1}W×H that accompanies the data.

To improve the reconstruction accuracy, the coarse reconstruction x^t is refined by an iterative refinement module (IRM, Fig. ) through a sequence of residual improvements, producing the final reconstruction x̃ and the corresponding uncertainty characterized by the variance σ2. Per pixel j, we model the reconstructed SST as a Gaussian distribution parameterized by predicted mean x̃(j) and standard deviation σ(j), following . Note that σ2 emerges from training the model to minimize Eq. (), which penalizes over- and underestimation of the error variance σ2.

Let x̃(i) and σ2(i) be the reconstruction of the observation field xt and its estimated uncertainty at ith refinement iteration. An iteration of IRM proceeds as follows. The input observation fields Xm=[xt-1,xt,xt+1]∈R3×W×H and the refined estimates [x̃(i),σ2(i)]∈R2×W×H from the previous iteration are concatenated channel-wise and passed to a residual estimation network REN⁽ⁱ⁾ (detailed in Sect. “Residual estimation network (REN)”) alongside the tokens TtD produced by CRM, to produce a two-channel output Y(i)=[Y1(i),Y2(i)]∈R2×W×H. Following the formulation of , Y(i)=[Y1(i),Y2(i)] are decoded into reconstruction δx(i) and uncertainty δσ2(i) residuals: 1δσ2(i)=1max(exp(min(Y1(i),θ1)),θ2),2δx(i)=Y2(i)⊙δσ2(i), where ⊙ denotes element-wise tensor multiplication (the Hadamard product), while θ1 and θ2, θ1>θ2>0 are hyperparameters ensuring training stability. The reconstruction and uncertainty estimates at iteration i=0 are initialized with the coarse reconstruction x̃(0)=x^t and a zero σ2(0)=0. The reconstruction and uncertainty estimated at the (i+1)th refinement iteration are thus x̃(i+1)=x̃(i)+δx(i) and σ2(i+1)=σ2(i)+δσ2(i), respectively. IRM runs for NIRM iterations, with each REN⁽ⁱ⁾ having its own set of trained parameters, allowing each to specialize to its respective residual estimation, finally producing the refined reconstruction x̃ and uncertainty σ2.

CRITER is trained in two stages to train both CRM and IRM. First, supervised learning with automatically generated targets is used to train CRM. In this setup, part of the input signal is deleted, and the network is trained to reconstruct the entire input signal. The input training samples are created by sampling triples of consecutive observations [xt-1,xt,xt+1] and deleting parts of the central observation xt, resulting in [xt-1,xt⊙Mm,xt+1], where Mm∈{0,1}W×H is a generated binary mask with 0 corresponding to missing values. Following , the masks Mm are generated by copying clouds from a random day not included in the triplet to maintain mask simulation realism. CRM is trained to minimize the following reconstruction error: 3LCRM=1|Mt⊙Ml|∑i=1N(xt(i)-x^t(i))2Mt(i)Ml(i), where x^t is the coarse reconstruction generated by CRM, and mask Mt has zeros at locations where ground truth measurements within the observation field xt are missing, while Ml has zeros at spatial locations belonging to land, and |Mt⊙Ml| denotes the number of ground truth measurements. The summation goes over the N pixels in each of xt, x^t, Mt, and Ml. The operator (⋅)(i) indexes the ith element of a matrix. The consecutive observations used as the model input and the masks Mt, Ml, and Mm used in the training process are visualized in Fig. .

In the second stage, the parameters of CRM are fixed and only the parameters of IRM are trained. The training samples are generated as in CRM training, but since IRM produces the mean and variance of the reconstruction, the following negative log-likelihood loss is minimized as in DINCAE : 4LIRM=1|Mt⊙Ml|∑i=1N(xt(i)-x̃(i))2σ(i)2+log⁡(σ(i)2)Mt(i)Ml(i), where x̃ and σ2 are the reconstruction and variance estimated after the last iteration in IRM, and the summation goes over the N pixels in each of x̃, σ2, and xt. This loss thus trains the model to assign higher variance to areas with greater than expected reconstruction error. We validate the variance prediction quality in Sect. , by demonstrating its correlation with empirical errors.

CRM (Sect. ) consists of 12 encoder and decoder transformer blocks, with 3 multi-head attention (MHA) heads, a token dimension of Dt=192, and a patch size of 3×8×8, where 3 denotes the number of channels, while 8×8 represents the width and height, respectively. IRM (Sect. ) consists of a CNN-based encoder with 3 double conv blocks, each followed by a 2×2 max pooling operation. The double conv block is composed of two 3×3 convolutional layers, each followed by a batch normalization layer and a ReLU activation function. The number of convolutional kernels in each block is 32,64, and 128, respectively. This is followed by another double conv block, with 256 kernels, at the bottleneck of the network, a Feature Fusion Module (FFM), and a decoder with 3 transpose convolution layers, each followed by a concatenation based skip connection and a double conv block. The number of kernels in each block is 128,64, and 32, respectively. IRM utilizes NIRM=3 refinement iterations – this value is selected based on the results of the ablation study in Sect. . Hyperparameters θ1 and θ2 are set as θ̃1=ln(NIRM)+θ1 and θ̃2=NIRMθ2 to ensure that the variance σ2 is bounded between 1/exp(θ1) and 1/θ2 for an arbitrary number of refinement iterations NIRM≥1.

CRITER is implemented using the PyTorch library and trained on an NVIDIA Tesla V100 GPU. The CRM block is trained with a batch size of 8 using the AdamW optimizer with a learning rate α=3×10-4, β1=0.9, and β2=0.95 for 60 epochs (warm-up period) and then with a cosine decay scheduler with step size 30 for another 140 epochs. In the next phase the IRM block is trained using the pre-trained CRM with fixed parameters. We train IRM using the Adam optimizer, with α=3×10-4, β1=0.9, and β2=0.999 for 300 epochs, using a step learning rate scheduler with step size 50 and multiplicative factor γ=0.5.

The performance of CRITER is assessed on an independent test set. Reconstruction quality is computed in terms of root-mean-squared error (RMSE) between the ground truth xt and the reconstruction x̃. In particular, the overall reconstruction error RMSE_all is defined as 5RMSEall=∑i=1N(xt(i)-x̃(i))2Mt(i)Ml(i)|Mt⊙Ml|.

For additional insights we compute the RMSE separately for (i) deleted regions, corresponding to observations artificially removed by simulated clouds in the L3 SST product and thus withheld during the training, and (ii) visible regions, corresponding to remaining observations post-deletion in xt.

The reconstruction error of deleted regions is defined as 6RMSEmis=∑i=1N(xt(i)-x̃(i))2Mt(i)Ml(i)(1-Mm(i))|Mt⊙Ml⊙(1-Mm)|, Mm is the mask of deleted regions, and |Mt⊙Ml⊙(1-Mm)| denotes the number of deleted ground truth measurements. The reconstruction error of visible regions is defined as 7RMSEvis=∑i=1N(xt(i)-x̃(i))2Mt(i)Ml(i)Mm(i)|Mt⊙Ml⊙Mm|, where |Mt⊙Ml⊙Mm| is the number of visible ground truth measurements. To enhance the metric stability, we sample 10 distinct cloud masks for each test SST field, simulating realistic observational variability. We thus evaluate the performance on 2560, 3900, and 1720 masked SST fields for the respective regions, ensuring robust statistical validation.

We compare CRITER with DINCAE2 , a well-known and highly competitive SST reconstruction method, serving as a widely recognized benchmark in recent studies , and with the recently presented MAESSTRO on the three datasets from Sect. . We reimplemented both DINCAE2 (originally in Julia) following  and MAESSTRO (public implementation unavailable) following  in Pytorch. To ensure a fair evaluation, both methods were trained using the same dataset splits, with tuned hyperparameters, and employed the same loss function computed over identical regions to CRITER. For MAESSTRO, architectural modifications were necessary to ensure comparability. Please refer to Appendix  for the implementation details of baseline models.

Results in Table  demonstrate CRITER's consistent superior performance across all datasets. Compared to the current state of the art, DINCAE2, CRITER achieves error reductions in deleted and visible regions of 20 % and 89 % for the Mediterranean, 44 % and 80 % for the Adriatic, and 1 % and 88 % for the Atlantic dataset, respectively. MAESSTRO's significantly lower performance is attributed to its single time step reconstruction approach. This hypothesis is confirmed by our ablation study, detailed in Sect. , which examines the importance of modeling spatio-temporal data dependencies.

The relative improvements of CRITER compared to the related methods vary across the datasets. This can be attributed to the differing amounts of information available for reconstruction, which is inversely proportional with the extent of missing values. Our analysis of missing values (Appendix ) reveals that the datasets can be ranked by the average amount of information available in each observation triplet, from highest to lowest: Mediterranean, Adriatic, and Atlantic. Notably, the Adriatic dataset shows the greatest decrease in reconstruction error, suggesting that CRITER achieves optimal improvement when the available information is moderate. In contrast, the Atlantic dataset, with the lowest amount of available information, likely requires additional data to be effectively reconstructed. To address this, we propose increasing the temporal horizon Δt and incorporating supplementary or proxy variables, such as chlorophyll a and surface winds. We leave the exploration of this approach to future work.

CRITER and DINCAE2 estimate both the reconstruction of missing values and the associated uncertainty (i.e., the standard deviation) for each pixel. To assess the reliability of the estimated standard deviation, we employ the scaled error metric 8ϵ(i)=x(i)-x̃(i)σ(i), as proposed by . This metric quantifies the difference between the ground truth observation x(i) and the reconstruction x̃(i), normalized by the estimated standard deviation σ(i), where i is the pixel index. We calculate the mean, μϵ, and standard deviation, σϵ, of the scaled error over the entire test set. Furthermore, we compute the bias, defined as the (non-normalized) mean difference between the ground truth observations and reconstructions. An ideal reconstruction method would thus have the bias equal to zero (i.e., predicted values are not globally under or over estimated). and standard deviation of the scaled error σϵ equal to 1 (i.e., per-pixel disparities match the predicted uncertainties). Standard deviation of the scaled error σϵ<1 indicates that the predicted standard deviation σ is overestimated, while σϵ>1 indicates that σ is underestimated.

Figure  displays the histogram of the scaled error metric ϵ(i) for each test set, along with the corresponding Gaussian distribution, characterized by the estimated mean μϵ and standard deviation σϵ. The mean (μϵ), standard deviation (σϵ), and the bias for each dataset are provided in Table . Notably, CRITER moderately underestimates the standard deviation, with standard deviation of the scaled error σϵ values of 1.116, 1.082, and 1.156 on the Mediterranean, Adriatic, and Atlantic datasets, respectively, ranging from 8 % to 16 %. In contrast, on average, DINCAE2 significantly overestimates the standard deviation, with σϵ values of 0.334,0.996, and 0.801 across the three datasets. The over-estimation thus ranges from as little as 0.4% to substantial over-estimates of 66 %. CRITER consistently exhibits a very low bias (of the order of 0.01 °C or lower) over all datasets. Furthermore, CRITER exhibits a significantly smaller bias on the Mediterranean and Adriatic datasets than DINCAE2, whereas DINCAE2 achieves a smaller bias on the Atlantic dataset. Note that, on the Adriatic dataset, DINCAE2 exhibits 18× larger bias than CRITER.

We analyze the proposed CRITER architecture by ablating or replacing individual parts. All model variants are trained for a total of 500 epochs (CRM and IRM are trained for 200 and 300 epochs, respectively) using the hyper-parameters described in the “Implementation details” section (Sect. ). The variants are evaluated on the Mediterranean dataset.