Flex_extract v7.1 – A software to retrieve and prepare ECMWF data for use in FLEXPART

Flex_extract is an open-source software package to efficiently retrieve and prepare meteorological data from the European Centre for Medium-Range Weather Forecasts (ECMWF) as input for the widely-used Lagrangian particle dispersion model FLEXPART and the related trajectory model FLEXTRA. ECMWF provides a variety of data sets which differ in a number of parameters (available fields, spatial and temporal resolution, forecast start times, level types etc.). Therefore, the selection of the right data for a specific application and the settings needed to obtain them are not trivial. Therefore, the data 5 sets which can be retrieved through flex_extract by both authorised member state users and public users and their properties are explained. Flex_extract 7.1 is a substantially revised version with completely restructured software, mainly written in Python3, which is introduced with all input and output files and for the four different application modes. Software dependencies and the methods for calculating the native vertical velocity η̇, the handling of flux data and the preparation of the final FLEXPART input files are documented. Considerations for applications give guidance with respect to the selection of data sets, caveats 10 related to the land-sea mask and orography, etc. Formal software quality assurance methods have been applied to flex_extract. It comes with a set of unit and regression tests as well as code metric data. A short description of the installation and usage of flex_extract as well as information about available detailed documentation is also provided.


Introduction
The widely used off-line Lagrangian particle dispersion model (LPDM) FLEXPART Stohl et al., 2005; The retrievable data sets are identified by the key meta data listed in the "Identification" section of Table 1. The relevant data period for each data set is also listed. Furthermore, the table presents the available temporal and spatial resolution as well as the number of ensemble members (may change in the future for the operational data). The availability ofη is important for the mode of preparing the vertical velocity fields (see Sect. 3.7) and is therefore marked for accessibility as well. With the current operational data, a temporal resolution of 1 h can be established with a well-selected mix of analysis and forecast fields (see Sect. 4). The horizontal grid type refers to the way how it fields are archived in MARS. Table 4 provides the relationship 155 between corresponding spectral, Gaussian and latitude / longitude resolutions.
In this paper, we collect the essential changes in forecast steps and spatial resolution since the first IFS release, as they need to be known for using flex_extract. Table 2 lists the evolution of horizontal and vertical resolutions for all operational data sets. The evolution of the forecast steps and the introduction of additional forecast times in "DET-FC" and "ENS-CF" are summarised in Table 3. 160 The reanalysis data sets are naturally more homogeneous. Nevertheless, they all have their individual characteristics, making the selection process with flex_extract complex. Table 1 provides an overview of the main differences in the reanalysis meta data. ERA-Interim has a 3-hourly resolution with an analysis and forecast field mix in the full access mode but only a 6hourly resolution for public users. It lacks theη fields which makes the retrieval of ERA-Interim computationally demanding (Sect. 3.7). The ERA5 and CERA-20C reanalyses can be retrieved with 1 h resolution and include ensembles; however, ERA5 165 ensemble members are not yet retrievable with flex_extract and therefore omitted in the tables. Even though the availability of 1-hourly analysis fields means that forecast fields are not required for most of the variables, accumulated fluxes are only available as forecasts. One should also pay attention to differen forecast start times in both data sets and the complication inplied by forecasts starting from 18 UTC as the date will change until the subsequent start time; see also Sect. 3.6.
With the establishment of the Copernicus Climate Change Service (C3S) in March 2019, a new channel for accessing 170 ECMWF reanalysis data, most prominently ERA5 (Hersbach et al., 2020), has been opened. At the same time, access to this data set via the ECMWF Web API was cancelled. While access directly from ECMWF servers is not affected, in local retrieval modes now one has to submit requests to the Copernicus Climate Data Store (CDS), which uses another Web API called CDS API; in the background, this API retrieves the public data from dedicated web servers for faster and easier access.
Unfortunately, only surface and pressure level data are available in CDS at the moment; this might change in the future. 175 In the case of member-state users, it is possible to pass the request to the MARS archive even through the CDS interface.
Flex_extract is already modified to use this API so a member user can already retrieve ERA5 data. However, experience shows that the performance of this access mode is not good, thus currently it cannot be recommended.

Software description and methods
The flex_extract software package allows to retrieve and prepare the meteorological input files from ECMWF for FLEXPART 180 (and FLEXTRA) easily and in an automated fashion. The ecessary meteorological parameters to be retrieved are predefined Gateway (member) This mode is recommended in the case a local Member State Gateway server is in place (ECMWF,215 2019j) and the user has a member-state account. Job scripts would then be prepared locally (on the gateway server) and submitted to the ECMWF Linux Member State server via the ECMWF web access toolkit ECaccess. The actual data extraction and post-processing is then done at the ECMWF servers and the final data are, if selected, transferred back to the local gateway server. The installation script of flex_extract must be executed at the local gateway server. However, this will install flex_extract in the users $HOME directory on the ECMWF server and some extra setup is done in the 220 local gateway version. For instructions about establishing a gateway server, please consult ECMWF (2019j) directly. The necessary software environment has to be established before installing flex_extract.
Local member Member-state users work on their local machines which require a similar software environment as the one on ECMWF servers plus the provided Web API's as the interface for the MARS archive.
Local public Public users can work on their local machines by not only preparing the general software dependencies but 225 also adding the ECMWF Web API as the interface to the public MARS archive. In this case, a direct registration at the ECMWF is necessary, and all users have to accept a specific license agreement for each data set which is intended to be retrieved.

Software dependencies
The software required to run flex_extract depends on the application mode. Basic requirements for all application modes are 230 listed in Table 7. The local mode requires in addition Python packages ecmwf-api-client and / or cdsapi, depending on the data set to be retrieved, to connect to the MARS archive as Table 6 shows. Users should make sure that all dependencies are satisfied before starting the installation. The software is tested only in a GNU / Linux environment, although it might be possible to use it also under other operating systems.

Program structure 235
The work of flex_extract cam be decomposed into the following three separate tasks: 1. Setting the parameters controlling the retrieval and the data set: Reading of the CONTROL file, command line arguments, and ECMWF user credential file (in the case of remote or gateway mode). Depending on the application mode, flex_extract prepares a job script which is sent to the ECMWF batch queue, or proceeds with the tasks 2 and 3. 240 2. Retrieve data from MARS: MARS requests are created in an optimised way (jobs split with respect to time and parameters) and submitted. Retrieved data are arranged in separate GRIB files. If the parameter REQUEST was set to 1, the request is not submitted and only a file mars_requests.csv is created. If it is set to 2, this file is created in addition to retrieving the data.
3. Post-process retrieved data to create final FLEXPART input files: 245 After all data are retrieved, flux fields are disaggregated, and vertical velocity fields are calculated by the Fortran program calc_etadot. Finally, the GRIB fields are merged into a single GRIB file per time step with all the fields FLEXPART expects. Depending on the parameter settings, file transfers are initiated and temporary files deleted.
In task 1, the software differentiates depending on the application mode. In the case of remote or gateway mode (see also Fig. 2), the job script for the ECMWF batch system is prepared and submitted to the batch queue. The program finishes with 250 a message to standard output. In the case of the local application mode, the work continues locally with tasks 2 and 3, as illustrated in Figure 3.
Each application mode has its unique process steps and its own connection to the MARS archive. Figure 4 demonstrates the involved input files, execution scripts and connection methods as well as the locations where each step takes place. In the gateway mode, the setup task will be done on the gateway server and the created job script is sent via the ECaccess command 255 to the batch queue before flex_extract terminates. As soon as the job script is processed, the job environment is created and flex_extract is started in local mode, reading the prepared CONTROL file, extraction of data, and post-processing tasks. In this mode, the extraction is done with a MARS command. If it was selected, the final output files are sent to the local member-state gateway server. The remote mode works completely on ECMWF servers but has the same process sequence as the gateway mode. In the local mode, all work is done on the local host, except the data extraction which is done by an HTTP request via 260 the Web APIs. The data are sent back to the local host instantly. The job script will then start on the ECMWF server and apply flex_extract in local mode. In the case of queue = None, the local mode is selected and Figure 3 applies as is indicated by the branch do local work. Trapezoidal boxes mark standard output, simple rectangles mark the execution of sequential instructions, and the rectangles with a side border mark the execution of subroutines. The boxes in diamond form indicate decisions.  Figure 3. Flow diagram for the local application mode. If queue = None, flex_extract assumes that a job script has to be sent to an ECMWF server and selects another branch shown in Fig. 2. This is marked by the submit block. In the case of request == 1, flex_extract is forced to skip the retrieval and post-processing steps and just writes the mars_request file. Within the pure local mode the retrieval and postprocessing parts are conducted. Symbols as in Fig. 2 11 https://doi.org/10.5194/gmd-2019-358 Preprint. Regarding the file content, the first string in each line is the parameter name, the following string(s) (separated by spaces) are the parameter values. The parameters may appear in any order, with one parameter per line. Comments can be added as lines beginning with #-sign, or after the parameter value. Some of these parameters can be overruled by command line parameters provided at program call. In earlier versions, each parameter name contained the leading string M_; this was removed for 280 version 7.1 but is still possible for compatibility. The grid resolution had to be provided in 1/1000 of a degree before, while now it can be also provided as a decimal number with unit degree. Flex_extract is able to check for correct setting of the GRID parameter with the domain-specific settings.
It is now also possible to reduce the number of data values for the combination of TYPE, TIME and STEP parameter combination to the actual temporal resolution. Previous versions expected to have 24 values per parameter, one for each hour 285 of the day, even though a 3-hourly temporal resolution was selected as shown in the following example: The more intuitive solution of providing just the data for the time steps to be retrieved leads, for example, to eight data values per parameter for a 3-hourly retrieval as shown in the next example: DTIME 3 TYPE AN AN AN AN AN AN AN AN STEP 00 00 00 00 00 00 00 00 or four values for a 6-hourly retrieval.

DTIME 6
TYPE AN AN AN AN TIME 00 06 12 18 300 STEP 00 00 00 00 The only necessity is the consistent setting of the DTIME parameter to define the temporal resolution. For backward compatibility, this means that DTIME can be coarser than the number of temporal points provided in TYPE, TIME and STEP, but not finer.

305
In the remote and gateway mode, the software sends job scripts to the batch system of an ECMWF server, thus it is necessary to provide the user and group name which are given in file ECMWF_ENV. Additionally, this file provides the name of the local member-state gateway server and the destination so that unattended file transfer 4 (ectrans) between ECMWF and member gateway servers can be used. The destination is the name of the so-called ectrans association; it has to exist on the local gateway server.

Template files
Some files are highly variable depending on the setting in the other input files. They are created during run time by using template files. The templates are listed in Table 11. Flex_extract uses the Python package genshi to read the templates and substitute placeholder by values. These placeholders are marked by a leading $ sign. In the case of the korn shell job scripts, where (environment) variables are used, the $ sign needs to be escaped by an additional $ sign. Usually, users do not have to 315 change these files.

Installation
The installation of flex_extract is done by the shell script setup.sh, which is located in the root directory of flex_extract.
This script sets all available command line arguments for the installation, does some plausibility checks and finally calls the 320 Python script install.py. Users are supposed to provide these arguments according to their environment and application mode. Parameters which must be set for all application modes are defined in (settings of related parameters to be done in setup.sh, see Table 10), the job script template is prepared, and the korn shell script for compiling the Fortran source code compilejob.ksh is created. After these preparations, a tar ball with the core 325 content is created and copied to the target location (ECMWF server or local installation path). Next, the compilejob.ksh is submitted to the batch system of ECMWF servers via ECaccess commands, or just untar-ed at the target location. It compiles the Fortran code and, in the case of remote / gateway mode, a log file is sent to the user's email address.

Execution
In earlier versions of flex_extract, an extraction was initiated by calling the Python script submit.py with suitable command 330 line arguments. The Python script constitutes the main entry point and controls the program flow including the call of the Fortran program. Now, it is still possible to work in that way, but a wrapper shell script run.sh is provided in addition. This shell script contains a user section where the Python command line arguments are to be set. This mode of operation is simpler for beginners, and it is useful for repetitive tasks, as no arguments can be forgotten. The available command line arguments are listed in Table 13. Some of the arguments occur only with the program call, while others are also defined in the CONTROL file.

335
In this case, the values in submit.sh take precedence over those from the CONTROL file.
The submit.py script interprets the command line arguments (overview available through ./submit.py --help from the Python source directory) and, based on the input parameter QUEUE, it decides which application mode is active.
In local mode, data are fully extracted and post-processed, while in the remote and gateway mode, a korn shell script called job.ksh is created from the template and submitted to the ECMWF batch system via the gateway server.

340
The underlying template is job.temp, stored in the Templates directory. This template was generated in the installation process from job.template where some basic settings were done, which are the same for each flex_extract execution.
The job script sets necessary directives for the batch system, creates the run directory and the CONTROL file, sets some environment variables (such as the CONTROL file name) and executes flex_extract. The standard output is collected in a log file which will be sent to the users' email address in the end. The batch system settings are fixed and they differentiate between 345 the ecgate and the cca/ccb server systems to load the necessary modules for the environment when submitted to the batch queue. The ecgate server has directives marked with SBATCH 5 for the SLURM workload manager, the high performance computers cca and ccb have PBS 6 comments for PBSpro. The software environment dependencies mentioned in Section 3.2 are fulfilled by loading the corresponding modules. It should not be changed without further testing.
Just for completeness, there are two more entry points in the software which are provided for debugging. The Python scripts 350 getMARSdata.py and prepare_flexpart.py are normally used as modules, but can also be used as an executable program. The getMARSdata.py script controls the complete extraction of ECMWF data, while the prepare_flexpart.py  Stohl et al. (2005). This method requires point values in the discrete input fields. However, flux data (as listed in Table 14) from ECMWF represent cell averages or integrals and are accumulated over a time interval, which depends on the data set. Hence, to conserve the integral quantity with FLEXPART's linear interpolation, a pre-processing scheme has to be applied.
The first step is to de-accumulate the fields in time so that each value represents an integral in (x, y, t)-space. Afterwards, a 360 disaggregation scheme is applied. While the horizontal cell values are simply ascribed to the cell centre, with respect to time, a more complex procedure is needed because the final values should correspond to the same time as the other variables. In order to be able to carry out the disaggregation procedure of Paul James, additional flux data are retrieved automatically for one day before and one day after the period specified. Note that these additional data are temporary and used only for disaggregation within flex_extract. They are not contained in the final FLEXPART input files. The flux disaggregation produces files named 365 fluxYYYYMMDDHH, where YYYYMMDDHH is the date. Note that the first and last two flux files do not contain any data. Note that for operational retrievals which use the BASETIME parameter, forecast fluxes are only available until BASETIME, so that interpolation is not possible in the last two time intervals. This is the reason why setting BASETIME is not recommended for regular on-demand retrievals.

370
In versions 7.0.x and before, a relatively simple method was applied to process the precipitation fields, consistent with the linear temporal interpolation applied in FLEXPART for all variables. At first, the accumulated values are divided by the number of hours (i.e., 3 or 6). For the disaggregation, precipitation sums of four adjacent time intervals (p a , p b , p c , p d ) are used to generate the new instantaneous precipitation (disaggregated) value p which is valid at the boundary between time intervals 1 and 2 as follows: Horizontally, precipitation is given as cell averages. The cell midpoints coincide with the grid points at which other variables are given, which is an important difference to the temporal dimension. FLEXPART uses bilinear interpolation horizontally.
3.6.2 Disaggregation for precipitation in version 7.1 385 Due to the shortcomings described above, a new algorithm was developed by Hittmeir et al. (2018). In order to achieve the desired properties (Hittmeir et al., 2018(Hittmeir et al., , p. 2513, a linear formulation with two additional supporting points within each interval is used. The current version of flex_extract implements this algorithm for the temporal dimension. Figure 6 shows how these requirements are fulfilled in the new algorithm for the simple case presented in Figure 5.
Flex_extract allows to choose between the old and the new disaggregation method for precipitation. In the latter case, the two 390 additional sub-grid points are added in the output files. They are identified by the parameter "step" which is 0 for the original time at the left boundary of the interval, and, respectively, 1 or 2 for the two new sub-grid points. Filenames do not change.

Disaggregation for the other flux fields
The accumulated values for the other variables are first divided by the number of hours and then interpolated to the exact times using a bicubic interpolation which conserves the integrals of the fluxes within each timespan. Disaggregation uses integrated 395 values F during four adjacent time-spans (F 0 , F 1 , F 2 , F 3 ) to generate a new, disaggregated point value F valid at the boundary between intervals 1 and 2 as follows:

Preparation of vertical velocity
An accurate representation of the vertical velocity is a key component for atmospheric transport models. One of the considerations for the design of FLEXTRA was to work entirely in the native coordinate system of ECMWF's IFS model to minimise 405 interpolation errors. This meant that the same hybrid η coordinate (terrain-following near ground, approaching pressure levels towards the model top) would be used, which implied to use the corresponding native vertical velocity ("etadot") rather than the more commonly used ordinary vertical velocity in a simple z-sytem (units of m s −1 ) or the vertical motion ω of pressure-based systems (unit Pa s −1 ). For reasons that we can't reconstruct, however, FLEXTRA did not useη strictly, but rather a quantitẏ which obviously has units of Pa s −1 . The code calls this quanitity etapoint, not to be confused with etadot. Even though in FLEXPART this concept had to be abandoned in favour of a terrain-following z-system to allow a correct implementation of the Langevin equation for turbulent motion, FLEXTRA and FLEXPART share the same requirement for the vertical motion 415 with respect to their input. Over many years, ECMWF would store only the post-processed perssure vertical velocity ω=dp/dt.
Transforming this back toη, with approximations and interpolations involved in both operations, leads to vertical velocities that do not fulfill continuity. Therefore,η was reconstructed from the fields of divergence using the continuity equation, integrated from the model top downward as described in Simmons and Burridge (1981). In the IFS model, dynamical variables are horizontally discretised by spherical harmonics. It is best to do this on the reduced Gaussian grid that is used in IFS when a 420 grid-point representation is required.
In September 2008, ECMWF started to archive the model's native vertical velocity fields (η) for the operational analyses and forecasts. This allowed flex_extract to skip the cumbersome reconstruction and directly use this parameter. The amount of data that needs to be extracted from MARS, the CPU time and the memory requirements are all reduced substantially. The ERA5 and CERA-20C reanalyses also provideη. Thus, even though it is possible to use the old method on new data sets, there 425 is no reason to do so and it would be a waste of resources. It is, however, still kept in flex_extract to allow extraction of data from the older data sets, in particular ERA-Interim. In the following, the two methods are briefly characterised.

Reconstruction of the vertical velocity using the continuity equation
The most accurate algorithm for the reconstruction of the native vertical velocity requires the extraction of the horizontal divergence fields and the logarithm of the surface pressure in spectral representation (and thus always global, regardless of the 430 final domain), their transformation to the reduced Gaussian grid (introduced by Ritchie et al. (1995)), on which the continuity equation is solved, a transformation back to the spectral space, and finally the evaluation on the latitude-longitude grid desired by users. Especially for high spectral resolution, this is a compute-and memory-intensive process that also takes time, even when making use of OpenMP parallelisation. Larger data sets can only be treated on the supercomputer (cca) but not on ecgate. The code for these calculations is written in Fortran90.

435
Alternatively, data can be extracted from MARS immediately on the latitude-longitude grid for the domain desired, and the continuity equation is then solved on this grid, but this method is not as accurate as the calculations on the Gaussian grid, particularly for higher spatial resolutions.

Preparation of the vertical velocity using archivedη
If the vertical velocity is available in MARS, it only needs to be multiplied with ∂p/∂η. In the flex_extract version discussed here, this is done by the Fortran program whose functionality is described below.

Short description of the functionality of the calc_etadot code
A dedicated working directory is used where all input and output files are kept. Currently, the files have names of the form fort.xx where xx is some number.
The control file steering the code is fort.4 and has the form of a Fortran namelist. An overview of the options set by this 445 namelist is contained in Table 15. The control file is prepared automatically by the Python code, but some of these parameters appear also as input to the Python part. Note that the selection of the method for obtainingη follows the logic laid out in Table   16.
All other input files are data in GRIB format that were retrieved from MARS. The code is using dynamic memory allocation and thus does not need to be recompiled for different data sets.

450
The code is provided with a set of makefiles. The standard version assumes a typical GNU/Linux environment with a gfortran compiler and the required libraries: openmp for parallelisation -it comes with the gcc compiler package, libgrib_api or libeccodes for handling GRIB files, libemos for transformation between the various representations of fields. Note that the latter two typically require also so-called developer packages containing the Fortran module files. It is assumed that these libraries have been installed as a package from the distribution and thus are at their standard locations and 455 compatible with the gfortran compiler. There is one makefile called makefile_fast with optimisation that is used for production. In addition, there is makefile_debug which is optimised for debugging. There are also makefiles for cca and ecgate as well as a makefile for the Intel Fortran compiler to be used locally. The latter one may require adaptations by users with respect to the library and include paths.
If the program finishes successfully, the last line written to standard output is SUCCESSFULLY FINISHED calc_etadot:

460
CONGRATULATIONS which is useful for automated checking the success of the run. The output file into which the fields ofη p and the other three-dimensional variables (temperature, specific humidity, u and v components of the wind -not the recently introduced cloud water variable) are combined is fort.15; it is a GRIB file.
The code also foresees options for certain checks where different forms of the vertical velocity are obtained, statistically compared, and also written out (see Table 15). These options were used for quality control in the development process and 465 should not normally be activated by users.
Currently, the code also unifies the three-dimensionsal fields extracted from MARS and stored in separate GRIB files with the calculated vertical velocity by writing out all fields into a single GRIB file; later this is unified with the 2D fields and the new 3D parameters such as cloud water and written out into a final single GRIB file as required by FLEXTRA and FLEXPART.

470
These temporary output files are usually deleted after a successful data extraction. They are only kept in debugging mode, which is the case if the DEBUG parameter is set to true.

MARS GRIB files
All extracted meteorological fields from MARS are in GRIB format and stored in files ending with .grb. MARS requests are split in an optimised way to reduce idle times and considering the limit of data transfer per request. The output from each 475 request is stored in one GRIB file whose name is defined as <field_type><grid_type><temporal_property> <level_type>.<date>.<ppid>.<pid>.grb. The field type can be analysis (AN), forecast (FC), 4d variational analysis (4V), validation forecast (CV), control forecast (CF) and perturbed forecast (PF). The grid type can be spherical harmonics (SH), Gaussian grid (GG), output grid (OG) (typically lat/lon) or orography (_OROLSM) while the temporal property distinguishes between an instantaneous field (__) or an accumulated field (_acc). Level types can be model (ML) or surface level 480 (SL) and the date is specified in the format YYYYMMDDHH. The last two placeholders are the process number of the parent process of submitted script (ppid) and the process number of the submitted script (pid). The process IDs are incorporated to avoid mixing of fields if several flex_extract jobs are performed in parallel (which is, however, not recommended).

MARS request file
This file contains a list of the MARS requests from one flex_extract run, with one request per line. This is an optional file users 485 are able to create in addition to full extraction; it can also be created without actually extracting the data which is useful for test purposes. Each request consist of the following parameters whose meaning is explained in Tables 8 and 9, and in more detail in the supplemental material or are self-explanatory: request number, accuracy, area, dataset, date, expver, Gaussian, grid, levelist, levtype, marsclass (alias class), number, param, repres, resol, step, stream, target, time and type. The parameters Gaussian (defines whether the field is regular or a reduced Gaussian grid), levtype (distinguishes between model levels and 490 surface level) and repres (defines the grid type -SH, GG, OG) are internal parameters not defined as any available input parameter.

Vertical discretization constants
The file VERTICAL.EC is created by the Fortran program as a temporary storage for internal usage and contains the A and B constants to calculate the model level heights in m.

Index file
The index file is called date_time_stepRange.idx. It contains indices pointing to specific GRIB messages from one or more GRIB files so Python can easily loop over these messages. The messages are selected with a predefined composition of GRIB keywords.  Note that, in the case of the new dis-aggregation method for precipitation, two new sub-intervals are added in between each 505 original time interval. They are identified by the forecast step parameter STEP which is 0 for the original time interval and 1 or 2 for the two new intervals respectively.

fort. * files
There are a number of input files for the calc_etadot Fortran program named fort.xx, where xx is the number which defines the meteorological fields stored in these files. They are generated by the Python part of flex_extract by just splitting 510 the meteorological fields for a unique time step from the * .grb files. Table 17 explains the numbers and the corresponding content. Some of the fields are optional and are retrieved only with specific settings, for example the divergence is retrieved only ifη is not available in MARS, and the total cloud water content is an optional field for FLEXPART v10 and newer. The output of calc_etadot file fort.15.

515
The final output files are the FLEXPART input files containing the meteorological information. FLEXPART expects one file with all relevant fields per time step. Table 18 and 19 list all of the meteorological fields that flex_extract retrieves and FLEXPART expects. The naming of these files depends on the extracted data. In the following sections we describe the differences and how the filenames are built.
3.9.1 Standard output files 520 The standard file names have the format <prefix>YYMMDDHH, where the <prefix> is by default defined as EN and can be re-defined in the CONTROL file. Each file contains all meteorological fields on all selected model levels on a latitude-longitude grid needed by FLEXPART for the specified time step in the filename. If not otherwise stated, model level fields are in GRIB2 format and surface levels in GRIB1. Forecast times and steps are summed up to the corresponding analysis hour.

Output files for pure forecasts 525
For the selection of forecasts longer than 23 h, a different naming scheme has to be applied to avoid collisions of time steps for forecasts of more than one day. This case is defined as pure forecast mode and file names are defined as <prefix>YYMMDD.HH.
<FORECAST_STEP>. The <prefix> is, as in the standard output files, EN by default and can be re-defined in the CONTROL file. In this case, the date format YYMMDD does not include the hour directly and the hour represents the starting forecast time and is placed separately by .HH. The FORECAST_STEP is a 3-digit number which represents the forecast step in hours. The new dis-aggregation method for precipitation fields produces two additional fields for each time step and precipitation type. They contain the sub-grid points in the correspoinding original time intervals as described above in Sect. 3.6.2. The two additional fields are marked with the step parameter in the GRIB messages, are set to "1" and "2", respectively. The output filenames do not change in this case.

4 Considerations for application
First of all, users should be aware of the different natures of operational and reanalysis data sets (see Table 1). Operational data are available since the start of ECMWF's operational forecasts, and are influencend by frequent changes in IFS model physics and resolution. Reanalysis data sets were created using a single IFS model version throughout the whole period covered. More precisely, the CERA-20C data set (with 91 vertical levels, 1.25 • horizontal and 3 h temporal resolution) has a lower resolution 545 but covers a very long period (from 1901 to 2010) and will thus be suitable for certain climate applications. ERA-Interim data set (with 60 vertical levels, a medium resolution of 0.75 • horizontally and 3 h temporally) was the standard ECMWF reanalysis until recently, but has noη stored in the MARS archive which make retrievals computationally demanding. The new ERA5 data set has the highest resolution (0.25 • horizontally and 1 h temporally, 137 vertical model levels). Users are encouraged to use ERA5 data rather than the ERA-Interim data set (production ended in August 2019). In addition to its better resolution, ERA5 550 covers a longer period than ERA-Interim, provides uncertainty estimates with a 10-member ensemble of data assimilation and uses a newer IFS model version (ECMWF, 2019l).
With respect to the relation between temporal and spatial resolution, it is important to consider the use in FLEXPART and their influence on numerical errors. It is not useful to apply high horizontal resolution in combination with, for example, 6hourly temporal resolution as in such a case, small fast-moving structures are resolved in space, but their movement will not 555 be properly represented. Interpolation will not let the structures move, but rather jump from their position at time t to that at time t + 6 h if the displacement between two subsequent times where fields are available is comparable to or larger than their characteristic width normal to the phase speed. Users can orient themselves with the spatial and temporal resolutions at which ECMWF provides reanalysis data, and the sample CONTROL files.
global scales, a horizontal resolution of 1 • could be a reasonable choice, whereas tracking point releases in complex terrain would call for the best available resolution.
Attention should also be paid to the model topography and the land-sea mask. Due to limited resolution, a coastal site with a given geographical coordinate could be over water in the model. Then it might be better to shift the coordinates of a release or receptor point in FLEXPART slightly. Another aspect is that the smoothed representation of the topography could mean 565 that the model topography is above or below the real height of a site. It is therefore important to select the proper kind of z coordinate in the FLEXPART RELEASES file. As a compromise, one can place a release location at a hight between real and model topography (for mountain sites which are usually lower in the model topography than in reality). In such cases, it is strongly recommended to retrieve the model topography and land-sea mask and investigate them carefully before deciding on the FLEXPART set-up details, or even before retrieving the full meteorological data set, as one might come to the conclusion 570 that one with better resolution should be used.
The vertical levels used in FLEXPART follow a hybrid η coordinate system. This is much more efficient than pure pressure levels since hybrid η coordinates are terrain following near the ground and approach pressure levels close to the model top. This has the advantage of better resolution of strong gradients in the boundary layer irrespective of the terrain height, and allows to easily fulfill the lower boundary condition of a flow parallel to the surface whereas pressure levels do not follow the terrain 575 (Stohl et al., 2001). ECMWF data sets either directly provide theη variable or the data needed to reconstruct it accurately. This is a big advantage of ECMWF data compared to other data sources, most notably the NCEP model data, which are available only on pressure levels.
Attention should be paid to the number of vertical model levels to be extracted and used in FLEXPART, as the computational cost of the FLEXPART verttransform subroutine (reading and preparing meteorological input) increases with the 580 third power of the number of vertical levels. Thus, only data that are really needed for the application (e.g., troposphere, or troposphere and lower stratosphere) should extracted.
Operational data sets and ERA-Interim have analysis fields on a 6-or 12-hourly basis (0, (6), 12 and (18) UTC) only. The gaps inbetween can be filled with forecast fields. Mixing analysis and forecast fields should be done by considering at which time steps the differences between two IFS run segments will be the lowest. For example, using all four analysis fields, but the 585 forecasts starting from 00 and 12 UTC only would lead to unnecessary rate of changes between 05, 06, and 07 UTC and 17, 18, and 19 UTC. This should be avoided by using only 00 and 12 UTC analysis fields and the forecast fields for +1 to +11 hours for the forecasts starting at times 00 and 12 UTC, respectively.

Quality assurance
Nowadays software development is mainly dominated by adding new features as well as maintaining and adjusting specifica-590 tions rather than developing from scratch (Beizer, 1990 that the software works under specified conditions. As a consequence, tests from the previous software version can be used to show that there are no undesired changes in the unchanged part of the software. This is called regression testing (Beizer, 1990;Spillner, 2012 Unit tests are used to test the smallest pieces of code (single code blocks) independently to identify a lack of functional specification (Beizer, 1990). Applying unit tests does not guarantee error-free software rather than limiting the chance of occurrence. Once the tests were written they are also a kind of documentation and serve as a protection to not altering a functional behavior after applying code changes (Wolff, 2014). Hence, they are also kind of a regression test.
As a first step, we launched unit tests for functions which were designed or partly refactored to be testable code blocks. Since 610 unit tests are for the verification of small and independent code blocks, functions which are too complex or too long are badly testable most of the times. In the future, our intention is to increase the number of unit tests and refactor the still too complex functions into smaller ones (see also Sect. 5.6 for identifying complex functions).
We used the pytest package which is a part of standard Python as well as the mock package which simulates external dependencies or results for the tests solely. This gives the opportunity to test the good and bad pathes in a function and usually 615 a function holds as many unit tests as there are different branches. It is a matter of defining all possible results depending on the input states and verify the expected results. The first set of unit tests were applied for functions from the install and tools modules as well as for the UIOFiles and EcFlexpart class. The details for each test are not described here; their functionality is obvious from the code. which should not be different. Future improvments may also test for value differences considering a significance threshold.

Regression testing for MARS requests
A regression test was created which compares the GRIB files produced by two versions with respect to the number of files 640 produced, the file names, the number of GRIB messages per file, the content of the GRIB messages header, and statistical parameters for the data themselves. If differences are reported, the developer has to judge whether they are expected or an indication of problems. The current release version 7.1 includes a minimal set of reference data from version 7.0.4, one for each type of data set (see Sect. 2.2). There will be more test data in the future which can then be downloaded from the community website to limit the size of the distributed release tarball. The corresponding reference CONTROL files are also 645 distributed with the tarball to enable the retrieval of the data with the new version. This has to be done manually followed by placing the resulting GRIB files in a specific path as described in the README.md file.

Functionality and performance tests for the Fortran code
Regression tests were set up to reflect the three possibilities for obtaining the vertical velocityη listed in

655
The code package contains a set of reference outputs, and scripts to create the reference output and to run the actual regression tests. It checks for bitwise identity of the output files (data files and standard output written to a log file). A quantitative comparison of the resultingη p which would be useful for modifications that affect the results is not yet implemented. The scripts run each test with both the fast and the debug version of the executable. The script for creating the reference also ensures that both yield identical results. In addition, the runtimes are saved to a csv file.

Generic test using predefined CONTROL files
Flex_extract comes with a set of CONTROL files listed in Table 20; executing it with each of them constitutes a generic test ensuring that the data extraction works without problems for all typical applications. This has been verified for version 7.1 by manually executing the software with all these files, and inspecting the results produced. Note that public users can only use files ending with .public; they were tested in local public mode. All other cases were tested both in local and in gateway 665 mode. Since the remote mode does not differ much from the gateway mode, only a subset of files were also tested in this mode. Results were evaluated by inspecting the log files for "success" messages and, where possible, with the regression test for GRIB file comparison (Sect. 5.3). Regarding new features, the files were inspected manually for the expected result.

Code Metrics
Metrics for the maintainability and complexity of code as well for the documentation are a useful tool for developers who of blank lines (blank), with the following relation between these numbers: The comparison shown in Table 21 indicates a significant increase not only in the logical lines of code, but even more in of connected components (which are sub-graphs from functions independent of the supergraph) (Lacchia, 2019;Beizer, 1990;Sneed et al., 2010). The nodes represent the conditional branch instructions and program junctions, and edges are the segments between such points. Regarding code testing, CC provides a lower-bound number of how many test cases (unit tests) are necessary to provide complete path coverage (Beizer, 1990). This metric was also calculated with the radon package (Lacchia, 2019). It provides the CC rank for each function, class and class method. Table 22 gives an overview of the interpretation of 695 these ranks. In general its said that the score should not be above 10, corresponding to rank C to F. From the statistical point of view, Table 23 shows that only 10.3 % of flex_extract version 7.1 code blocks have higher complexity, while in version 7.0.4 this was the case for 30.8 %.
The mean cyclomatic complexity of all code blocks in the new Python code is 5.74 (B); for those blocks with C to F (Table   24), it is 21 (C). In version 7.0.4, the corresponding numbers are 13 (C) and 31.86 (E), indicating a substantial improvement.
700 Table 24 lists all the code blocks with their ranks and scores. For example, the class ControlFile was improved significantly, as well as the class renamed from EIFlexpart to EcFlexpart. On the other hand, the class method deacc_fluxes became more complex in version 7.1. This is mainly due to two new features, of ensemble retrieval and the new disaggregation.
Nevertheless, the overall code complexity was reduced.
Another software metric is the maintainability index (MI), ranking between 0 and 100; it is a function of SLOC, CC and the where C is the fraction of comment lines (converted to radians) (Lacchia, 2019). The Halstead volume is defined as with η 1 being the number of distinct operators, η 2 being the number of distinct operands, N 1 the total number of operators and 710 N 2 the total number of operands. Table 26 defines the classification ranks. The index is calculated for a complete Python file and Table 26 shows the ranks of version 7.0.4 and 7.1 respectively.
Additionally, we used a source code quality checker program called pylint (Thénault, 2001) to support following the Python style guide PEP8 (van Rossum et al., 2001). This tool provides an overall rating index with a maximum number of 10. Applying this tool, flex_extract version 7.0.4 has a rating of -8.77 and version 7.1 a rating of 9.09. This shows a massive 715 improvement in following the official style guides.
6 Final remarks and outlook

Conclusions
This paper describes the software package flex_extract v7.1, which retrieves meteorological fields from the ECMWF IFS model and prepares them for the use in the Lag rangian particle dispersion model FLEXPART. The software was initially developed 720 in the 1990ies and underwent various developments to adapt to the ECMWF environment and the data set characteristics.
In the past two years, the ECMWF introduced considerable changes in its software environment to retrieve, read and access data as well as the preparation of new data sets. This necessitated a substantial upgrade of flex_extract to adapt to these changes. Additionally, the user community had new requirements for data retrievals which were considered in this version.
In the development process, substantial refactoring was carried out, the number of retrievable data sets was increased, user-725 friendliness was improved, current ECMWF software packages considered, an online documentation was built, and a first set of test cases for future regression testing was created. Furthermore, a recently developed and improved disaggregation method for precipitation fields was implemented as an option.
The number of groups using FLEXPART grew substantially over the past decade and with the new opportunity of publicly available reanalysis data sets there will likely be even more users who will try out and apply FLEXPART for their research.

730
Alongside this reference paper, the newly established git repository on the FLEXPART community website https://flexpart.eu and the online documentation should assist all these users with up-to-date information about changes, releases of new versions, installation and usage including a documentation useful for future developers.

Support
FLEXPART has a community website http://flexpart.eu, where flex_extract as a pre-processor has its own sub page 7 . The 735 website features a ticket system to report issues or feature requests. The tickets can be viewed by anyone, to create a ticket, registration 8 is necessary. There is also a mailing list for discussion among FLEXPART / FLEXTRA users and with developers, where questions may be asked or experiences be shared, including pre-and post-processing issues. Announcements for all FLEXPART users, such as new releases, are distributed through the list as well.

Future work 740
In its current status, the on-line documentation provides a basic reference. In the future, more examples should be provided, including answers to typical user questions and workarounds for known problems. Information about updates and new releases will also be implemented in this documentation.
It is also intended to optionally retrieve meteorological fields from ECMWF needed as input for the WRF model to support the FLEXPART-WRF community. The unification of the three-dimensional fields into a single file shall be removed from the Fortran code as this is a simple task that can be fulfilled more efficiently and transparently with ECcodes command-line tools.
The ERA5 reanalysis has ensemble members stored in the enda stream, but the flux data have a different accumulation period and therefore are not yet retrievable. It is planned to allow the retrieval of these ensemble members in the future. Up to now, it is possible to set flex_extract to retrieve reduced Gaussian fields. This should be extended to include the octahedral 750 reduced Gaussian grid.
The hybrid vertical velocityη is now stored not only for the operational forecasts but also for the new reanalyses, thus the need to calculate it is diminishing. Therefore, the option to calculate it on the native Gaussian grid might be removed in the future. This would allow to do all the remaining calculations in Python3 without resorting to Fortran code.
The flex_extract software is currently provided as a compressed tar file. In the future, a package shall be made available to 755 be installed as a system software for all users. Then, only user-specific data need to reside in a user directories. The software comes with a number of test cases which should be used by developers in the future. Some tests need additional reference data which have to be downloaded from the project website in addition.

765
The

Appendix A: Installation instructions
First of all, download the release version from the FLEXPART community website. Alternatively, if git is installed, you may clone the latest version from our git repository master branch.
git clone --single-branch --branch master Currently, the software was only tested for a GNU/Linux environment. The installation process depends on the user group (see Sect. 2.1) and the application mode (see Sect. 3.1). One should first decide for the modes and then follow the compact instructions in the corresponding subsections. For more details see the instructions in the online documentation.

A1 Registration and licenses
780 Table 6 summarizes which registration is required. Follow the given links from the literature to the registration websites (or footnotes).
A separate license has to be accepted for each ECMWF public data set, regardless of the user group. For the ERA-Interim and CERA-20C datasets this can be done at the website for "Available ECMWF Public Datasets" 10 . Log in and follow the license links on the right side for each data set and accept it. For the ERA5 data set this has to be done at the "Climate Data 785 Store (CDS) website" 11 . Log in and select, on the left panel, product type "Reanalysis" for finding ERA5 data sets. Then follow any link with ERA5 to the full data set record, click on tab "Download data" and scroll down. There is a section "Terms of use" where the "Accept terms" button has to be clicked. The licenses for member state users are accepted by the user when receiving a so-called "Token", which generates new passwords for each log in.

Remote mode
ECMWF servers provide all required libraries (see Table 7) via a module system. Flex_extract takes care of loading the right modules at runtime except the Python3 module which needs to be loaded prior to its execution by module load python3.
This is due to the fact that flex_extract is first started to prepare the job script with the correct settings before submitting the job to the batch queue.

Gateway mode
In this mode, access to the ECMWF computing and archiving facilities is enabled through an ECaccess gateway server on a local member state server. The ECaccess framework is necessary to interactively submit jobs to the ECMWF batch system and to transfer files between ECMWF and local gateway server. As a consequence, a member state gateway server has to be established 12 and a so-called association 13 has to be created to use the ECaccess file transfer service ectrans.

800
The next step is to create an ECaccess certificate to authorize the file transfers and job submissions. This certificate has to be renewed periodically (every 7 days). The certificate is created by executing the command ecaccess-certificatecreate on the command line of the local gateway server and the user is prompted for the ECMWF member state user name and a password (generated by a token).

905
An example for retrieving ERA5 data from MARS is shown below and can be tested if the code from above worked. Eventually, adapt the parameters (described in Table 12 and 10) in the setup.sh script and execute it.
Flex_extract uses the email address connected to the user account to notify the user about successful or failed installation.

Gateway mode
The actual execution of flex_extract with retrieval and preparation of the data will be run on an ECMWF servers. The only 940 difference is the preparation of the job script, which is done on the local gateway server and sent to ECMWF servers by the ECaccess services. Unpack the release tarball and change into its directory. Substitute X.X with the actual release version number.

945
Afterwards, prepare the setup.sh script by configuring its parameters (described in Table 12 and 10) and execute it. The makefile has to be selected according to the selection of the target, e.g. ecgate or cca/ccb servers. In this mode the DESTINATION and GATEWAY parameters have to be set to be able to use the ectrans service. A confguration job script is then sent to the ECMWF batch queue and flex_extract uses the email address connected to the user account to notify the user about successful or failed installation.

Local mode
Since flex_extract compiles the Fortran program preconvert during the installation process, a corresponding makefile has to be provided. Flex_extract comes with two makefiles for the local mode prepared for the gfortran (https://gcc.gnu.org/fortran/) and the ifort (https://software.intel.com/en-us/fortran-compilers)compiler. The gfortran version assumes that eccodes and emoslib are installed as distribution packages. It is necessary to adapt the two parameters ECCODES_INCLUDE_DIR and ECCODES_LIB 955 in these makefiles if other than standard paths are used.
Hence, if needed, prepare the Fortran makefile for your environment by starting from one of the two provided makefiles makefile.local.gfortran or makefile.local.ifort. They can be found at flex_extract_vX.X/ Source/Fortran, where vX.X should be substituted with the current version number. Edit the paths to the eccodes library on your local machine.

960
Eventually, adapt the command line parameters (described in Table 12 and 10) in the setup.sh script in the root directory of flex_extract and execute it.

A4 Installation test
The most common errors in applying flex_extract arise from wrong installation and settings regarding the libraries for the Fortran program. Therefore it is useful to do a simple test with a prepared minimal data set. The following instructions have to 965 be executed on the local system for the local mode and on the ECMWF servers in the remote and gateway mode. From the flex_extract root directory change into the Testing/Installation/Convert/ directory and execute the Fortran program by Note that on ECMWF servers the flex_extract root directory is placed in the $HOME directory.

980
Appendix B: How to use flex_extract Flex_extract is a command-line tool which can be started by executing the submit.py script in the Python source directory or more preferably with an upstream shell script run.sh which calls the submit.py script with its corresponding commandline arguments. Therefore, the user should navigate to the Run directory, where the shell script is located. For the first data retrieval it is recommended to use one of the example CONTROL files stored in the Control directory to avoid unnecessary problems. We recommend to extract CERA-20C data since they are usually not highly demanded and guarantee quick processing for the best testing experience.

Remote and gateway modes
For member state users it is recommended to use the remote or gateway mode, especially for more demanding tasks, to retrieve and convert data on ECMWF machines and to transfer only the final output files to the local host.
The only difference between both modes is the location where flex_extract will be started from. In the remote mode we work directly on the ECMWF server, therefore login to the ECMWF server of your choice and change to the Run directory as shown 1000 above. Remember, at ECMWF servers flex_extract is always installed in the $HOME directory. To be able to start the program, please load the Python3 environment with the module system first.
module unload python module load python3 Within the gateway mode, only a change into the Run directory of flex_extract on the gateway server is necessary.

1005
Otherwise, the rest of the working steps are the same in both modes. Now, open the run.sh script and modify the parameter block marked in the file as shown below. The parameters are described in Table 13. over Europe on the ECMWF server ecgate. For the ECMWF cca/ccb servers, the parameter QUEUE has to be adapted. Since the ectrans parameter in the CONTROL_CERA file is set to 1 the resulting output files will be transferred to the local gateway into the path stored in the destination, provided that the destination was correctly setup. The parameters listed in the run.sh script would overwrite existing settings from the CONTROL file.
Starting the retrieval process will be done by executing the script by ./run.sh.

1035
Flex_extract will print some information about the job on standard output. If there is no error in the submission to the ECMWF server a message like this will be shown: In case the job failed, the subject will contain the keyword ERROR! and the job name. Then, the user can check the email or 1045 on ECMWF servers in the $SCRATCH directory for debugging information.
In the $SCRATCH directory on ecgate it is recommended to list the content with ls -rthl to list the most recent logs and temporary retrieval directories (usually pythonXXXXX, where XXXXX is the process id). Under pythonXXXXX a copy of the CONTROL file is stored under the name CONTROL, the protocol is stored in the file prot and the temporary files as well as the resulting files are stored in a directory work. The original name of the CONTROL file can be found within this new file 1050 under parameter controlfile.
If the job was submitted to the High Performance Computer (HPC) (QUEUE is cca or ccb) you may login to the HPC and look into the directory /scratch/ms/ECGID/ECUID/.ecaccess_do_not_remove for job logs. The working directories are deleted after job failure and thus normally cannot be accessed.
If the resulting files can not be found in the destination path of the local gateway server, it can be checked if the files are still 1055 to be transferred to the local gateway server by using the command ecaccess-ectrans-list.
After this test retrieval was successful, feel free to try changing the CONTROL file parameters described in Tables 8 and 9 and by selecting other CONTROL files. Please mind the considerations of application in Section 4.

Local mode
Since this mode can be used by member and public users, we show an example for both user groups. Open the run_local.sh 1060 file and adapt the parameter block marked in the file as shown for the corresponding user group. The parameters are described in Table 13. Take this setting as member-state user: This would retrieve a one day (08.09.2000) CERA-20C dataset with 3 hourly temporal resolution and a small 1 • domain over Europe. The destination location for this retrieval is set by the INPUTDIR parameter and will be the Workspace/CERA * directory within the current Run directory. This can be changed to whatever path is preferred. The parameters listed in run_local.sh would overwrite existing settings in the CONTROL file.
Starting the retrieval process will be done by executing the script by ./run\_local.sh.

1115
While a job submission on the local host is convenient and easy to monitor (on standard output), there are a few caveats with this option.
There is a maximum size of 20 GB for single retrievals via ECMWF Web API. Normally this is not a problem but for global fields with T1279 resolution and hourly time steps the limit may already apply. If the retrieved MARS files are large but the resulting files are relativly small (small local domain, but large time period) then the retrieval to the local host may be inefficient 1120 since all data must be transferred via the Internet. This scenario applies most notably if ETADOT has to be calculated via the Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Munoz-Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., Simmons,

1170
A., Soci, C., Abdalla, S., Abellan, X., Balsamo, G., Bechtold, P., Biavati, G., Bidlot, J., Bonavita, M., Chiara, G. D., Dahlgren, P., Dee, D., Diamantakis, M., Dragani, R., Flemming, J., Forbes, R., Fuentes, M., Geer, A., Haimberger, L., Healy, S., Hogan, R. J., Hólm, E., Janisková, M., Keeley, S., Laloyaux, P., Lopez, P., Radnoti, G., de Rosnay, P., Rozum, I., Vamborg, F., Villaume, S., and Thépaut, J.-N.: The ERA5 Global Reanalysis, Quart. J. Roy. Met. Soc., submitted, 2020. Table 1. Overview of ECWMF data sets with associated parameters required in MARS requests (Berrisford et al., 2011;Laloyaux et al., 2016;ECMWF, 2019e, h). DET-FC stands for "Deterministic forecast", ENS-DA for "Ensemble data assimilation", ENS-CF for "Ensemble control forecast", ENS-CV for "Ensemble validation forecast" and ENS-PF for "Ensemble perturbed forecast". All times are in UTC, all steps in hours. Dates are written as DD/MM/YYYY (day optional). Steps and members are written in the format of Start/End/Step. The information for operational data sets is valid at the time of publication (except ENS-CV -deprecated since August 8th, 2016), and may change in the future. The grid type (Oxxx) for the operational data refers to the octahedral reduced Gaussian grid. The identification parameter "Data set" is to be used by public users only. Note that there is also the ERA40 reanalysis; however, as it has been superseded by ERA-Interim and ERA5 and thus rarely used nowadays, it is not included here (but flex_extract should still be applicable).      Laloyaux et al. (2016). For the spectral grid the truncation number is denoted by "T" or "TL" where the latter means linear spectral truncation. The corresponding reduced Gaussian grids are denoted by "N" followed by the number of lines between the pole and the equator. Only linearly truncated grids can be selected with flex_extract. The new octahedral grid is denoted by "TCO", meaning "spectral cubic octahedral"; they correspond to a octahedral reduced Gaussian grid denotes with an "O".              Table 17. List of fort files generated by the Python part to serve as input for the Fortran program, and the output file of calc_etadot.

Operational data Reanalyses
If the optional fields were not extracted, the corresponding files are empty.

Number Content
Input to the Fortran program calc_etadot