The FV3 dynamical core was implemented in GFS, replacing the spectral dynamical core for an operational upgrade in June 2019. The FV3 is a fully compressible, nonhydrostatic core featuring a Lagrangian vertical coordinate and cubed-sphere grid . The Lagrangian vertical coordinate allows for a unique, straightforward representation of vertical motions directly through the relative deformation of the vertical layers. This is in contrast with the Eulerian framework presently featured in operational nonhydrostatic dynamical cores in use at the convective scale .

The FV3, originally a global model, features three types of local refinement capabilities: stretching of the global grid using the Schmidt refinement technique , one- and two-way nesting within the global grid , and, recently, a LAM capability . The LAM capability eliminates the need to run a concurrent global model and instead relies upon lateral boundary conditions (LBCs) provided at prespecified intervals from an external source. A more complete description of the FV3 LAM and additional justification for limited-area modeling in the context of operational, convection-allowing NWP can be found in . In this study, the focus is on the LAM capability, as it underpins the future RRFS and requires fewer computing resources to achieve similar forecast performance compared with a two-way nesting method at lead times of less than 24 h . For the FV3 LAM, initial conditions (ICs) must also be provided at least once to initiate the forecast sequence for subsequent data assimilation cycling.

Preprocessing is performed by the utilities (UFS_UTILS) developed by NCEP's Environmental Modeling Center (EMC; https://github.com/NOAA-EMC/UFS_UTILS, last access: 12 July 2021) and other collaborators. UFS_UTILS can be used to generate the model grid, orography, and surface climatology (e.g., maximum snow albedo, soil, vegetation type, and vegetation greenness). UFS_UTILS can also read from external models and prepare ICs and LBCs for an FV3 LAM model run.

The Common Community Physics Package (CCPP; https://dtcenter.org/community-code/common-community-physics-package-ccpp, last access: 18 August 2021) is a collaborative effort between scientists at NOAA and the National Center for Atmospheric Research (NCAR). The goal is to assemble parameterizations developed by different groups into a common framework to be used interchangeably for numerical prediction at any scale . Hence, the CCPP contains a set of physical schemes and a common framework that facilitates the interaction between the physics parameterizations and the dynamical core . The current common framework was developed by the Developmental Testbed Center (DTC). A number of physics suites are available, allowing great flexibility for a wide range of users. A single-column model (CCPP SCM) option has also been developed, which is available in the latest CCPP release (version 5.0, CCPPv5). The CCPPv5 supports the RRFS_PHYv1a and GFS version 16 physics suites for the SRW. The RRFS_PHYv1a suite is based on physical schemes implemented in the operational RAP, HRRR, and GFS systems and is used in all simulations in this study. Table  presents the RRFS_PHYv1a physics parameterizations and associated studies that describe each scheme, based on .

GSI is a variational data assimilation system featuring the 3DVar e.g.,, hybrid 3DEnVar e.g.,, and hybrid 4DEnVar e.g., methods. It also includes an optional non-variational, complex cloud analysis capability that executes after the variational analysis as a method to specify cloud and hydrometeor variables e.g.,. GSI features the following standard control (analysis) variables: streamfunction, velocity potential, temperature, surface pressure, and normalized relative humidity following . However, the choice of control variable is flexible, and one may extend or modify the standard set to include other fields, such as hydrometeors or radar reflectivity e.g.,. In 3DVar and the associated hybrid variants, the static BEC is approximated through the application of a recursive filter which models the autocorrelations , while cross-covariances are handled in the standard context through statistical balance relationships obtained via regression e.g.,. The analysis is obtained by minimizing the incremental form cost function through the preconditioned conjugate gradient method .

The extension of GSI from traditional 3DVar to hybrid 3DEnVar and to hybrid 4DEnVar is accomplished through the extended control variable approach e.g.,. In this configuration, one is able to incorporate flow-dependent covariance information obtained from a complementary suite of ensemble forecasts. Typically this ensemble is obtained from a companion ensemble-based data assimilation system, such as the EnKF; however, one may use any suitably available ensemble. In fact, the regional operational data assimilation systems at NCEP have used the ensemble members from the GFS Data Assimilation System directly in the hybrid 3DEnVar framework . Although the use of lower-resolution global ensemble members may not be ideal for the representation of the error characteristics at finer scales, showed that considerable forecast improvement can be obtained even if the ensemble provided is from a different system, which is consistent with findings in other studies such as . The present study focuses on the 3DVar and hybrid 3DEnVar frameworks and uses the global ensembles as described in . Future work on RRFS involves the extension to a convective-scale ensemble in the EnKF, which will improve the representativeness associated with the forecast error covariance at finer scales. However, such a change is not a panacea. Aside from increased computational expense, the problem of rank deficiency of the ensemble-derived error covariance becomes more apparent with the expanded degrees of freedom associated with the finer spatial resolution. While localization helps somewhat, a computationally affordable ensemble is one that is often insufficiently sized. Therefore, future work also includes efforts to introduce multiscale data assimilation capabilities, such as scale-dependent localization e.g.,.

GSI is capable of assimilating a large suite of observations. This includes (but is not limited to) satellite radiances e.g.,, derived Global Navigation Satellite System Radio Occultation (GNSS-RO) observations, radar radial velocity and reflectivity e.g.,, Geostationary Lighting Mapper (GLM) lightning flash rates, web-camera-derived estimates of horizontal visibility , and conventional observations . After 2014, GSI became a community system, maintained and supported by the EMC and the DTC . Recently, it has been added as the analysis component to improve initial conditions for the RRFS .

Presently, GSI is the data assimilation system used at NCEP for all operational atmospheric data assimilation applications e.g.,. It was initially developed by the EMC and implemented as the analysis component in the operational GFS in May 2007 and in the operational RAP in May 2012 .

UPP is used at NCEP in all operational models. A community version is currently supported and maintained by the DTC. UPP takes native output from the model grid points/cells and creates post-processed outputs including numerous diagnostic quantities in the same model output grid and model-native or isobaric vertical coordinate . Post-processed outputs include diagnostic fields that are not part of the model computation and have been developed for different applications. These include, for example, precipitation type, composite reflectivity, simulated satellite brightness temperature, updraft helicity, storm motion, ceiling or cloud-base height, vertically integrated liquid, and lightning, among several others. More details on the diagnostic fields developed for hourly updated NOAA weather models such as RAP and HRRR, as well as how they are calculated, can be found in . These products are critical for users in their forecast processes. UPP was selected as the unified post-processing system for UFS, and modifications have been made to work with FV3-based models. Currently, it can be used in the UFS Medium-Range Weather and SRW applications.

The workflow ties all RRFS components together and handles all system interdependencies. It is based on the UFS SRW application v1.0.0 community workflow which uses the Rocoto Workflow Management System (https://github.com/christopherwharrop/rocoto/wiki/Documentation, last access: 17 May 2021). In essence, it manages the cycling configuration, taking each task dependency and specification into account. It oversees that the tasks to generate ICs and LBCs only start if all of the required information is obtained from the previous step. It manages how the data assimilation cycle advances, i.e., by running the forecast to generate the first guess and running the analysis once the first guess is completed. It handles the model execution by supervising the availability of ICs and LBCs for the specific hour, and it controls that model outputs only be post-processed if they exist in the model run directory. It also manages crucial information on the computational resource requirements to run each task.

A schematic diagram of major tasks and the general pipeline of the RRFS system is provided in Fig. . The “Make fixed files” task generates the model grid, orography, and climatological information needed for the model execution. The “Make ICs” and “Make LBCs” tasks read data from external models (such as GFS and HRRR); perform the necessary calculation, interpolation, and conversion; and then generate the appropriate ICs and LBCs for an FV3 LAM model run. The “Run analysis” task (the gray shaded area in Fig. ) executes the data assimilation system for an FV3 LAM run. It ingests various types of observations and combines them with a first guess (or background) to generate a best possible atmospheric analysis for the initialization of the FV3 LAM model integration. The first guess can be either an IC from an external model (after the “Make ICs” task) or a short-term forecast (1–6 h forecast, configurable to users) from a previous FV3 LAM model run. The first scenario is referred to as a “cold start” (the blue box in Fig. ), whereas the latter is called a “warm start” (the red box in Fig. ). In practice, for an FV3 LAM “warm start,” the first guess comes from “restart” forecast files generated by the FV3 LAM model. The “Run model” task is to run the FV3 LAM model with ICs and LBCs prepared from the previous steps. It is worth mentioning that, besides the “cold start” and “warm start,” an FV3 LAM model run can also start from an IC made directly from an external model without the data assimilation step. This is also referred as a “cold start”. The “Run post” task is to post-process the FV3 LAM forecasts and generate all target model fields for downstream plotting and/or examination.

The cycling configuration of the RRFS v0.1 is similar to the one used in RAP (i.e., cold starts are performed every 12 h and warm starts are performed at all other cycles using the 1 h forecast from the previous cycle as background for the analysis). RAP performs hourly updated continuous cycles with cold starts at 09:00 and 21:00 UTC using the 1 h forecast from cycles initialized at 08:00 and 20:00 UTC in 6 h parallel hourly spin-up cycles. The parallel spin-up cycles are cold started from GFS atmosphere analyses and RAP surface fields at 03:00 and 15:00 UTC. Cold starts in RAP introduce the atmospheric conditions while RAP land surface fields are fully cycled in the continuous cycle . Periodic updates of the large-scale atmospheric conditions are needed in regional modeling systems in order to account for corrections made by global observations over land and ocean and to avoid model drift from those conditions . At the time of execution of this research, not many RAP functionalities were available for use in the RRFS data assimilation framework; therefore, a simplified configuration with partial cycling is used. Development currently underway includes establishing a partial cycling capability for the inaugural operational implementation, RRFS version 1, with subsequent plans to consider a fully cycled version in later implementations leveraging recent advances discussed in . Figure  illustrates the RRFS cycling configuration from cycles initialized between 00:00 and 12:00 UTC. In each cycle, an 18 h free forecast is launched following the analysis, with hourly outputs. A cold start is performed at 00:00 and 12:00 UTC, and warm starts are performed between 01:00 and 11:00 UTC using the FV3 LAM 1 h forecast from the previous cycle as background for the analysis.

A line of convective storms developed over northeastern Oklahoma ahead of a southward moving cold front during the afternoon of 4 May 2020. At 18:00 UTC on 4 May 2020, a surface low-pressure system was observed across western Oklahoma with a dry line extending over western Texas, favoring an environment with low-level convergence, high temperatures, and humidity over these areas. Between 19:00 and 20:00 UTC, high values of mixed-layer convective available potential energy (MLCAPE) (3694 Jkg-1) and effective bulk shear (48 kt for the surface to 3 km layers and 36 kt for the surface to 6 km shear) were observed over northeastern Oklahoma. The instability parameters are based on the observed sounding at 19:00 UTC over Norman, Oklahoma (KOUN; University of Oklahoma Westheimer Airport). This environment provided favorable conditions for severe convective storms with potential for strong updrafts and the development of supercells e.g.,. At 20:00 UTC, convective cells were first seen in the radar reflectivity observations over that region (Fig. a); by around 22:00 UTC, a line of storms extended across central Oklahoma along the pre-frontal wind shift (Fig. c). The system evolved while slowly moving southeastward. A supercell developed over far southwestern Missouri at 00:00 UTC on 5 May (Fig. e), producing hail of 1.25 and 1.5 in. (3.175 and 3.81 cm, respectively) in diameter according to hail reports from the Storm Prediction Center (SPC). Clusters of severe storms developed across south central Oklahoma along the intersection of the cold front with the dry line. The convection associated with the squall line evolution resulted in several instances of large hail and high winds, mostly over northeastern and south central Oklahoma, southeastern Kansas, southwestern Missouri, and northwestern Arkansas.

For the simulation of this case, a domain is configured consisting of 460 grid cells ×460 grid cells centered on Fort Smith, Arkansas, with a 3 km horizontal grid spacing and 65 vertical layers. All simulations start at 00:00 UTC on 4 May 2020 and run hourly cycles until 06:00 UTC on 5 May 2020. Hourly 3 km HRRR analyses and forecasts are used to generate the ICs and LBCs for the FV3 LAM. The observation data assimilated in each experiment are the same as those used in the operational RAP system and include upper-air observations from rawinsondes, dropsondes, pilot balloons, aircraft, and wind profilers; rawinsondes data from synoptic stations, METeorological Aerodrome Reports (METAR), and the Mesoscale Network (MESONET); radar radial velocity and the vertical azimuth display derived from radar radial velocity; Atmospheric Motion Vectors (AMV) wind derived from satellite observations; and the Global Positioning System (GPS) Integrated Precipitable Water (GPS-IPW). The time window used is 1 h, allowing for observations to be assimilated within 30 min before to 30 min after the central analysis time.

Experiments are conducted testing the GSI 3DVar and 3DEnVar systems. For the hybrid 3DEnVar analysis, the Global Data Assimilation System (GDAS) 80-member ensemble forecasts (9 h forecasts) are used to provide the ensemble BEC e.g.,. These forecasts have a horizontal resolution of approximately 25 km and are available four times per day; therefore, the same 9 h GDAS ensemble forecasts are used for the 2 h before and 3 h after its valid hour. For example, the 9 h GDAS ensemble forecasts initialized at 00:00 UTC (valid at 09:00 UTC) are used for the cycles from 07:00 to 12:00 UTC. Similarly, the 9 h forecast GDAS ensemble initialized at 06:00 UTC (valid at 15:00 UTC) is used for the cycles from 13:00 to 18:00 UTC. This follows the same strategy as in the RAP system . As shown in , using off-time global and fixed ensemble-based BEC still produces better results than only using the static BEC. In all experiments with data assimilation, two outer loops with 50 iterations per loop are performed to minimize the cost function and find each analysis. In each outer loop a re-linearization is performed e.g.,. The increment is zero for the first outer loop; for the second outer loop, in contrast, it is updated with the solution found after the 50 iterations of the first outer loop. The spatial resolution of the analysis is 3 km, as in the forecast model.

GSI provides many functionalities and parameters, enabling users to make the best data assimilation configurations for different applications. A series of experiments are designed to examine the impact of different configurations on the analyses and forecasts. Some RAP configurations are tested in the experiments following . An experiment with no data assimilation is provided, acting as the baseline for all other experiments. This baseline experiment is called “NoDA” and uses the same cycling configuration as experiments with data assimilation, in terms of the cold and warm start ICs. The 3 km ICs from the HRRR are consistent with the 3 km grid spacing of the RRFS, such that fine-scale features found in the HRRR are present in the RRFS ICs. Table  lists all experiments in this research.

In order to examine how different weights of the ensemble BEC affect the results and what would be the best choice for the RRFS analysis, experiments with different ensemble weights are conducted. Only results from three experiments are presented here: 3DVar, 100EnBEC, and 75EnBEC. The experiment with 3DVar does not include any ensemble BEC part, 100EnBEC uses pure ensemble BEC and does not include the static part, and 75EnBEC uses a combination of 75 % ensemble BEC and 25 % static BEC. The static BEC for 3DVar is the same as currently used in RAP and HRRR .

Ensemble localization length scales play an important role in ensemble-based data assimilation algorithms, such as hybrid EnVar analyses e.g.,, as an effective way to mitigate sampling errors due to the relatively small ensemble size available for hybrid EnVar and ensemble analyses , especially at convective scales e.g.,. At this stage, it is important to determine how large the localization radius needs to be for RRFS analyses. Therefore, experiment “VLOC” was designed to examine the vertical localization radius that yields more realistic forecasts in RRFS. A separate study is underway in which the optimal horizontal localization for RRFS is also investigated; therefore, it is not examined here. The localization function in GSI is implemented as a single application of an isotropic recursive filter , and the radius is specified as a Gaussian half-width, either in scale height (ln⁡p) or in terms of the number of vertical layers. In this study, the radius is specified in terms of the number of layers. In VLOC, the vertical ensemble localization radius is changed from 3 vertical layers for the whole atmosphere (used in all other experiments) to a height-dependent localization setting: 1 vertical layer in the lowest 10 model layers and 3 layers for other model layers. A comparison experiment (not shown) was conducted reducing the vertical localization to 2 layers in the first 10 model layers, but results showed neutral impacts over VLOC.

The operational RAP system has developed a PBL pseudo-observation function in order to obtain a better representation of the PBL in the analysis. This function was initially developed to further leverage the information provided by METAR observations, extending their representativeness through the PBL depth in the Rapid Update Cycle (RUC) analyses. Improvements in the temperature, dew point, and CAPE forecasts were found when spreading the innovations from temperature, moisture, and wind in the layers above the surface and below the top of the PBL . also found a positive impact in the 3 h forecast of CAPE by using the PBL pseudo-observations, and the impact was greatly increased when additionally assimilating GPS-IPW. found a higher positive impact during the summer, when the PBL is deeper. This function has been used operationally since RAP version 3 ; therefore, it needs to be tested and tuned for its potential use in RRFS analyses. To test whether and how this function works for the RRFS v0.1, the “PSEUDO” experiment is designed, and the results of this experiment are presented in Sect. .

The study of showed that, regardless of the method used, the storm coverage was overestimated and reflectivity values were much higher than observed, which is likely linked to the physics suite used. However, it is also well known that nonphysical solutions (nonrealistic updraft/downdraft, negative humidity, supersaturation, etc.) can arise as a result of the data assimilation procedure e.g.,. In this study, the “CLIPSAT” experiment is conducted to analyze how the supersaturation removal procedure available in GSI affects the storm forecasts of RRFS. This function constitutes a simple adjustment in the background supersaturation during the cost function minimization. More details on this function are presented in Sect. .

Experiments VLOC, PSEUDO, and CLIPSAT are performed using the hybrid 3DEnVar algorithm with 75 % of the ensemble BEC and are compared against 75EnBEC results, due to the good results obtained for experiment 75EnBEC (see Sect. ) and the consideration that RAP uses 75 % of the ensemble BEC operationally .

MET version 9.0 is used for forecast verification. MET was developed at the DTC and has been widely used by the NWP community. Upper-air and surface observations are used to verify the vertical profiles of temperature, specific humidity, and wind, as well as 2 m temperature and 2 m dew point, respectively. For upper-air observations, the verification time window is 1 h and 30 min before to 1 h and 30 min after, whereas for surface observations, it is 15 min before to 15 min after the central time. The root-mean-square error (RMSE) and bias are computed and displayed with 95 % confidence intervals in Figs. , , and . The confidence intervals are derived at each forecast lead hour in every cycle using a bootstrap resampling technique of 1000 replications with replacement, and with bias-corrected percentiles e.g.,. Upper-air statistics are further analyzed at 00:00 and 12:00 UTC valid times.

Precipitation forecasts are verified against the hourly Stage IV precipitation product in terms of the ETS and frequency bias (FBIAS) for different thresholds, but only >0.01 in.h-1 (0.254 mmh-1) for lighter precipitation and >0.25 in.h-1 (6.35 mmh-1) for heavier precipitation are presented here. The grid-to-grid approach in MET is used.

Hourly MRMS composite reflectivity mosaics (optimal method) observations are used to verify the composite reflectivity forecasts using the Method for Object-Based Diagnostic Evaluation (MODE) in MET. In order to quantitatively identify the experiment configuration that yielded better forecasts, the median of maximum interest (MMI (F+O)) is analyzed. This metric results from the median between the maximum interest from each observed object with all predicted objects (MIF) and the maximum interest from each predicted object with all observed objects (MIO). It takes all attributes used in the total interest calculation into account, summarizing them into a single value. The forecast in greatest agreement with the observations will give MMI (F+O) values closer to one; otherwise, the values will be closer to zero.

Observation availability and coverage play an important role in the data assimilation process. Therefore, the number and type of observations available for this squall line case are examined. Figure  shows the spatial distribution of assimilated temperature observations at the 19:00 UTC cycle on 4 May 2020 for experiment 3DVar (other cycles and experiments have similar distributions and are not shown here). The analysis residuals (OmA) are also depicted in Fig.  using red and blue color depth for positive and negative values, respectively. In this analysis, assimilated temperature observations include those from aircraft, surface marine synoptic stations, METAR, and MESONET observations. There are a total of 3307 observations, which are well distributed across the limited model domain. Among these observations, 1545 are from aircraft, which concentrate around a few major airports as flights descend or ascend, and spread along flight paths. Moreover, a substantial amount of MESONET surface observations are also assimilated. There are far fewer METAR observations, but they are distributed evenly in the domain. A very limited number of surface marine synoptic stations are found near the coast on the Gulf of Mexico. The analysis residuals for temperature are generally small for aircraft and METAR observations, mostly less than ±1 ∘K in magnitude, whereas some MESONET observations have large analysis residuals. As pointed out in , while some MESONET stations are well maintained, the majority do not meet siting standards and maintenance protocols; therefore, these sites are assigned a higher observation error via a station blacklist. As expected, larger residuals are found from these observations when compared with other observation networks.

In order to check how results of the RRFS analysis behave at different cycles and whether the RRFS analysis executes correctly, Fig.  presents time series of the RMSE and bias for the backgrounds (1 h forecasts) as well as the analyses, verified against temperature observations (including all surface and upper-air data, as mentioned in Sect. ). Results presented are for experiments 75EnBEC (Fig. a) and 3DVar (Fig. b). Verification is conducted by utilizing the observation innovations (OmB) and the analysis residuals (OmA) generated by GSI for assimilated observations. Based on these OmB and OmA data, the RMSE and bias for the background and analysis are computed. It can be seen from Fig.  that the analyses have smaller RMSEs and biases compared with the background in both experiments. This means the analyses fit the observations more closely, although (owing to observation error) not perfectly, which is expected from a correctly executed data assimilation procedure. There is a noticeable jump in the RMSE values of the OmB from 00:00 UTC (12:00 UTC) to 01:00 UTC (13:00 UTC) on 4 May 2020. This is because 00:00 and 12:00 UTC are cold started from HRRR analyses. On the contrary, at 01:00 UTC (13:00 UTC) on 4 May, the background used is from the FV3 LAM 1 h forecast. Therefore, forecasts used to initialize cycles at 01:00 and 13:00 UTC undergo a spin-up process. The FV3 LAM 1 h forecasts are still in this spin-up process and, hence, yield larger RMSEs. In Fig. a, the background RMSE increases steadily from 14:00 to 23:00 UTC, compared with the relatively gentle increase between 02:00 and 11:00 UTC on 4 May. This may be due to the fact that there is active convection during the afternoon hours; hence, it is harder to obtain good forecast skill. Figure b has a much larger increase in the RMSE of OmB than that in Fig. a during the same time period from 14:00 to 23:00 UTC, indicating that 75EnBEC performs better than 3DVar. Results from 100EnBEC are similar to 75EnBEC and are not shown here.

The impact of adding PBL pseudo-observations to the analysis based on surface temperature and moisture observations is evaluated in experiment PSEUDO. This function first identifies the PBL height using the background (FV3 LAM 1 h forecast). Using METAR observations, it then computes the 2 m temperature and 2 m moisture observation innovations (OmB) such that they are inserted at multiple vertical levels, from the surface to the level corresponding to 75 % of the PBL height and spaced every 20 hPa . This technique works as if additional PBL observations are available at those levels and, thus, more observation innovations can be computed. Therefore, they are called “PBL pseudo-observations”. This function is tested with the PBL pseudo-observation configuration used in the operational RAP system.

The 2, 4, and 6 h composite reflectivity forecasts from experiments PSEUDO and 75EnBEC are presented in Fig. . PSEUDO clearly predicted more convection than 75EnBEC in the 2 h forecast, with a smaller MMI (F+O) value of 0.533 in comparison with 0.540 in the experiment 75EnBEC (Fig. a). However, noticeable improvements in the coverage and positioning of the storm are found in 4 and 6 h forecasts, with a corresponding increase in the MMI (F+O) values when compared with 75EnBEC (Fig. b, c). Especially at 4 h, the representation of the squall line over Oklahoma is greatly improved after adding PBL pseudo-observations with a better coverage of the squall line, although an increase in the intensity of the convective cores is also noted (Fig. b). Spurious convection also appeared over northwest Oklahoma and Texas in the 4 h forecast and over Texas in the 6 h forecast. These results indicate the potential of using PBL pseudo-observations in RRFS to improve the representation of convection.

The RMSE and bias vertical profiles for the 2 h forecast of temperature, specific humidity, and wind against sounding observations at the 00:00 and 12:00 UTC valid hours are presented in Fig. . The use of PBL pseudo-observations gives subtle positive impacts at both valid hours and most vertical levels for the RMSE and bias of temperature and specific humidity (Fig. a, b, d, e). Improvements in the RMSE and bias of temperature are observed below 900 hPa at valid hour 00:00 UTC. The positive impact in the bias extends to 800 hPa, indicating the better representation of the lower atmosphere in the experiment PSEUDO (Fig. a). A slight degradation is observed in the middle levels at the same valid hour. For wind, the RMSE shows more promising results with a positive and statistically significant impact between 500 and 550 hPa at 00:00 and 12:00 UTC valid hours. This positive impact is also significant at 300 hPa in the RMSE and bias results at 00:00 UTC (Fig. c). At 12:00 UTC, the bias shows more subtle improvements in 750 and 300 hPa (Fig. f).

Similar to the upper-air verification, the RMSE and bias of 2 m temperature and 2 m dew point temperature for the 2 h forecast in PSEUDO show an overall neutral impact. A degradation in the RMSE of 2 m temperature is observed between cycles initialized at 21:00 and 00:00 UTC (Fig. a) and in the bias between cycles initialized at 17:00 and 21:00 UTC. A subtle improvement is seen in the bias of 2 m temperature between cycles initialized at 21:00 and 23:00 UTC. Slight improvements are observed in the RMSE and bias of 2 m dew point temperature between cycles initialized at 19:00 and 23:00 UTC. Adding PBL pseudo-observations helps to mitigate near-surface dry bias during afternoon hours, makes upper-air forecasts better in some levels of the middle and upper atmosphere, and clearly improves the storm forecast in the 4 and 6 h forecasts. Nevertheless, more tuning and testing of this function are needed before applying this technique in the RRFS.

GSI has a function to remove supersaturation in the background by capping specific humidity to its saturation value in each outer loop during the minimization of the cost function, as calculated using the background fields . Figure  shows the difference in the specific humidity (gkg-1) analyses between the 75EnBEC analysis and the 75EnBEC analysis with the supersaturation clipping function activated (75EnBEC vs. CLIPSAT) for the 19:00 UTC cycle on 4 May 2020. As more moisture is present in the lower atmosphere, model hybrid level 15 (located in the lower atmosphere at around 850 hPa) is selected to show this result. Positive (negative) differences in Fig.  indicate that more (less) specific humidity is found in the 75EnBEC analysis than in CLIPSAT. The figure suggests that supersaturation is removed in the CLIPSAT analysis mostly over southwestern and northwestern Missouri, southeastern Kansas, northern Arkansas, and Oklahoma. It is worth mentioning that the computational runtime of the analyses in CLIPSAT is quite similar to 75EnBEC (not shown).

The 2, 4, and 6 h composite reflectivity forecasts are shown in Fig.  for experiments CLIPSAT and 75EnBEC. When the supersaturation removal function is activated in the analyses, a better evolution of the squall line is observed in the 4 and 6 h forecasts (Fig. b, c). The displacement errors are reduced, and less spurious convection is seen over southern Missouri and northern Arkansas at these forecast hours (Fig. b, c, e, f). As seen in Fig. , the CLIPSAT analysis shows less specific humidity content over these areas than in 75EnBEC. However, less spatial coverage of the convection is forecast over eastern Missouri, and the spurious convection for values lower than 35 dBZ is increased over southwestern Missouri at the 2 h forecast in CLIPSAT compared with 75EnBEC (Fig. a, d). Nevertheless, both experiments overforecast the reflectivity values larger than 35 dBZ over that area. The MMI (F+O) values show more skillful forecast of reflectivity larger than 35 dBZ for all forecast lengths in the experiment CLIPSAT. These values are greatly increased at 2 h forecast, from 0.540 in 75EnBEC to 0.811 in CLIPSAT, due to a better positioning and coverage of the reflectivity above 35 dBZ in areas over southeastern Missouri. Overall, more spurious convection over northwestern Missouri is shown in 75EnBEC which led to the lower MMI (F+O) at this forecast hour (see the blue and red circles over this area in Fig. a and d, highlighting improvement and degradation, respectively, for each experiment). At 4 h forecast, MMI (F+O) results show an increase from 0.698 in 75EnBEC to 0.793 in CLIPSAT, with a reduction of the spurious convection between north central Arkansas and south central Missouri in CLIPSAT. Nevertheless, most of the spurious convection shown in 75EnBEC over other regions is also observed in CLIPSAT, which may have penalized the MMI (F+O) values in the last experiment. At 6 h forecast, more similar MMI (F+O) values are found, but still less spurious convection is observed for lower reflectivity thresholds in CLIPSAT. Results from CLIPSAT indicate the presence of longer-term bias that is being corrected to some extent in this experiment. However, because the atmospheric state is periodically refreshed with the large-scale conditions as part of the partial cycling procedure, the model bias cannot be fully examined. Further investigation involves adapting the approach employed by in which forecast tendencies are used to investigate systematic model biases in a continuously cycled experiment.

To further evaluate the experiments conducted, the FV3 LAM 1 h accumulated precipitation is also analyzed. Precipitation forecasts remain a challenge for NWP models at various spatial and temporal scales. Because of their complexity, precipitation forecasts are frequently used to evaluate model performance.

As mentioned in Sect. , precipitation forecasts are verified against Stage IV precipitation observations at various thresholds. Figure  shows the ETS and FBIAS for 1 h accumulated precipitation greater than 0.01 in.h-1 (0.254 mm h−1) (Fig. a, c) and 0.25 in.h-1 (6.35 mm h−1) (Fig. b, d) for all experiments at each forecast lead hour. These verification measures are based on the two-by-two contingency table used for categorical (dichotomous) variables e.g.,. ETS is based on the threat score or critical success index and is commonly used to examine the performance of precipitation forecasts. Perfect forecasts have ETS values close to one, while forecasts without skill have ETS values close to zero. Meanwhile, FBIAS indicates when an event is forecast more or less often than it is observed. FBIAS greater than one indicates an event is overforecast, whereas less than one suggests an event is underforecast. An FBIAS equal to one indicates that the event is predicted as frequently as it is observed e.g.,.

For this case study, ETS values decrease as the precipitation threshold increases in all of the experiments assessed (Fig. a, b), indicating the difficulty in predicting heavier precipitation events. Most of the experiments with data assimilation have higher ETS scores for precipitation greater than 0.01 in.h-1 (0.254 mm h−1) compared with NoDA during almost the entire 18 h forecast (Fig. a). This shows the positive impact of data assimilation in the analyses and subsequent lighter precipitation forecasts. Experiments 100EnBEC, CLIPSAT, 75EnBEC, and PSEUDO show higher ETS values in the first 4 h of the forecast. Between 4 and 16 h forecast, experiment CLIPSAT shows the best performance among all experiments, followed by 100EnBEC, 75EnBEC, and PSEUDO, which shows very close results to 75EnBEC. In terms of FBIAS, 100EnBEC shows better scores until the 11 h forecast lead (Fig. c). Between 2 and 8 h forecast, experiment VLOC shows the greatest underforecast among all experiments. The verification of 1 h accumulated precipitation greater than 0.25 in. (6.35 mm) consistently shows that using hybrid and pure ensemble BEC in data assimilation improves the precipitation forecasts in the first 13 h forecast, with 75EnBEC outperforming 100EnBEC within the first 4 h (Fig. b). After the 13 h forecast, experiment NoDA performs better, which shows that data assimilation mainly improves the short-term forecast and that the major factor for a good long-term forecast is the quality of the background from the outside model as well as the FV3 LAM model itself. For the 0.25 in.h-1 (6.35 mm h−1) threshold, precipitation is overforecast in experiments 100EnBEC and NoDA in the first 3 h and underforecast in experiments 3DVar, 75EnBEC, PSEUDO, VLOC, and CLIPSAT. All experiments underforecast accumulated precipitation greater than 0.25 in.h-1 (6.35 mm h−1) after the 9 h forecast (Fig. d).