The primary focus of the framework is to support common commercial ALCs. Ceilometers are considered the most basic type of lidar intended as commercial products designed for unattended operation. They are used routinely to measure CBH, but most instruments also provide the full vertical profiles of the attenuated volume backscattering coefficient. Therefore, they are suitable for model evaluation by comparing not only CBH, but also cloud occurrence as a function of height. Their compact size and low cost make it possible to deploy a large number of these instruments in different locations or use them in unusual settings such as mounted on ships . Common off-the-shelf ceilometers are the Lufft CHM 15k and the Vaisala CL31 and CL51. Some lidars offer higher power and therefore higher SNR, as well as capabilities not present in ceilometers such as dual polarisation, multiple wavelengths, Doppler shift measurement and Raman scattering. Below we describe ALCs supported by the framework and used in our case studies: Lufft CHM 15k, Vaisala CL31 and CL51 and Droplet Measurement Technologies MiniMPL. Table  lists selected parameters of the supported ALCs.

The Lufft CHM 15k (previously Jenoptik CHM 15k) is a ceilometer operating at a wavelength of 1064 nm (near IR). The maximum range of the instrument is 15.4 km, with a vertical sampling resolution of 5 m in the first 150 and 15 m above as well as sampling rate of 2 s. The total number of vertical levels is 1024. The wavelength in the near-IR spectrum ensures low molecular backscattering. The instrument produces NetCDF files containing uncalibrated attenuated volume backscattering coefficient profiles and various derived variables, although the calibration coefficient is relatively consistent for different instruments of the model Fig. 13.

The Vaisala CL31 and CL51 are ceilometers operating at a wavelength of 910 nm (near IR). The maximum range of the CL31 and CL51 is 7.7 and 15.4 km, and the sampling rate is 2 and 6 s, respectively. The vertical resolution is 10 m. The total number of vertical levels is 770 and 1540, respectively. The wavelength is characterised by relatively low molecular backscattering (but higher than 1064 nm) and is affected by water vapour absorption , which can cause additional absorption of about 20 % in the mid-latitudes and 50 % in the tropics (see also Sect. ). The instruments produce data files containing uncalibrated attenuated volume backscattering coefficients which can be converted to NetCDF (see cl2nc in the “Code and data availability” section). The firmware configuration option “noise_h2 off” results in a backscatter range correction being selectively applied under a certain critical range and above this range only if cloud is present Sect. 3.2. This was the case with our case study dataset (Sect. ). We apply a range correction to the uncorrected range gates during lidar data processing. The critical range in CL51 is not documented but was determined as 6000 m based on an observed discontinuity.

The Droplet Measurement Technologies Mini Micro Pulse Lidar (MiniMPL) (previously Sigma Space MiniMPL and Hexagon MiniMPL) is a dual-polarisation micro-pulse lidar (meaning that it uses a high pulse repetition rate (PRF) and low pulse power) operating at a wavelength of 532 nm (green in the visible spectrum). The maximum range of the instrument is 30 km. The vertical resolution is 5–75 m and the sampling rate is 1 s. The shorter wavelength is affected by stronger molecular backscattering than 910 and 1064 nm. The instrument can be housed in an enclosure with a scanning head to provide configurable scanning by elevation angle and azimuth. The instrument produces data files containing raw attenuated volume backscattering coefficients which can be converted to NetCDF containing normalised relative backscatter (NRB) with the vendor-provided tool SigmaMPL (see also mpl2nc in the “Code and data availability” section).

Below we briefly describe the reanalyses and models

We use the term “reanalysis” when referring to ERA5, JRA-55 and MERRA-2 even though the reanalyses are based on atmospheric models. We use the term “model” when referring to AMPS and the UM, which are atmospheric models.

The lidar ratio S is the extinction-to-backscattering ratio of atmospheric constituents at the lidar wavelength. It is an important quantity in lidar observations and the lidar simulator because it determines the amount of attenuation and backscattering. LR is not explicitly known from the observed attenuated volume backscattering coefficient. For liquid cloud droplets at near-IR wavelengths it is relatively constant at S≈19 sr (Sect. ), while for ice crystals (Sect. ) and aerosol it is highly variable. When the lidar signal is fully attenuated, and under the assumption that cloud LR is constant and scattering from clouds is much stronger than molecular and aerosol scattering, LR can be determined from the observed attenuated volume backscattering coefficient by integrating it vertically : 1S′=ηS=12∫0∞β′dz, where S′ is effective (apparent) LR, a quantity which does not depend on the multiple-scattering coefficient.

The Rayleigh volume backscattering coefficient βmol (m-1sr-1) in ACTSIM is parameterised by the following equation (Eq. 8 in ): 2βmol=pkBT(5.45×10-32)λ550nm-4.09=pkBTCmol, where for lidar wavelength λ=532 nm, Cmol=6.2446×10-32; kB is the Boltzmann constant kB≈1.38×10-23 JK-1, p is the atmospheric pressure and T is the atmospheric temperature. We multiply this equation by exp⁡(4.09(log⁡(532)-log⁡(λ))) (where the value of λ is in nanometres) to get molecular backscattering for wavelengths other than 532 nm, which allows us to support multiple commercially available instruments. The strength of molecular backscattering is usually lower than backscattering from clouds for the relevant wavelengths.

The lidar signal at visible or near-IR wavelengths is scattered by cloud droplets in the Mie scattering regime . In the most simple approximation, one can assume spherical dielectric particles. The scattering from these particles depends on the relative size of the wavelength and the (spherical) particle radius r, expressed by the dimensionless size parameter x: 3x=2πrλ.

While the wavelength is approximately constant during the operation of the lidar

The actual lidar wavelength is not constant and is characterised by a central wavelength and width. The central wavelength may fluctuate with temperature .

Simulation of backscattering from ice crystals is relatively complex compared to backscattering from liquid droplets due to the very high variability of ice crystal microphysical properties such as habit, size, orientation and surface roughness, all of which affect LR, extinction cross section, single-scattering albedo and the multiple-scattering coefficient. Common habits include hexagonal plates, hexagonal columns, hollow hexagonal columns, droxtals, bullet rosettes, hollow bullet rosettes and aggregates . Size can be highly variable and bimodal with a dependence on temperature and relative humidity. Orientation is commonly random or horizontally oriented (often reported with hexagonal ice plates). The surface can vary between smooth and rough depending on supersaturation and crystal age. In general, the Mie theory cannot be used to simulate backscattering from ice crystals because of their irregular shape . While large crystals allow the use of the geometric optics approximation to estimate the optical properties, smaller crystals and diffraction by large crystals necessitate the use of more advanced techniques such as the T-matrix method, finite-difference time domain (FDTD), discrete dipole approximation (DDA) and others, which are generally computationally expensive. Current global atmospheric models do not normally explicitly parameterise the microphysical properties of cloud ice and provide only very limited information such as ice mass concentration and in some cases the effective radius of ice crystals in the model output. Radiative transfer schemes of atmospheric models do not explicitly evaluate backscattering (the phase function at 180∘) and therefore cannot provide this information to the simulator. Instead the phase function is parameterised by the asymmetry factor, which is likely insufficient to give an accurate estimate of backscattering.

Because the model ice crystal microphysical and optical properties are not known, they have to be parameterised. A first option is to parameterise the microphysical properties such as habit and size and theoretically calculate optical properties. A second option is to directly parameterise the optical properties. This appears to be a more practical choice because of the broad availability of global remote sensing measurements of optical properties from satellites and ground-based lidars compared to relatively scarce in situ measurements of ice crystals. analysed CALIPSO lidar and co-located passive infrared data from the Imaging Infrared Radiometer (IIR) and determined a global relationship between temperature, LR and the multiple-scattering coefficient at the lidar wavelength of 532 nm. The multiple-scattering coefficient is taken as a constant of 0.6 in the standard CALIPSO data processing, but they determined that it is in fact variable between about 0.4 and 0.8. Here, we parameterise LR based on their findings. LR varies with the lidar wavelength, a larger part of which is due to the change in the diffraction peak and a smaller part is due to the variation of the refractive index . We use the colour ratio to estimate LR at lidar wavelengths other than 532 nm. A colour ratio of 1064 nm relative to 532 nm is commonly estimated for dual-wavelength lidars such as CALIOP. Here, we use a value of 0.8, approximately consistent with the results of and . The effective radius is defined for non-spherical particles as reff=32IWCσ, where IWC is the ice water content, and σ is the volume extinction coefficient of ice. summarised the ice crystal effective radius (related to IWC / σ by a factor of 1.64) parameterised as a function of temperature based on a number of field studies. We use this relationship for determination of the effective radius. Figure c shows the true and effective LR based on and the effective radius based on , parameterised by the following equations: 13S=20+(34-20)1/T-1/2001/230-1/200sr,14η=0.8+(0.5-0.8)1/T-1/2001/240-1/200,15reff=exp⁡log⁡(16.4)+(log⁡(49.2)-log⁡(16.4))1/T-1/213.151/253.15-1/213.15µm, where T is atmospheric temperature in Kelvin (K). S follows Fig. 12b, η follows Fig. 9a and reff follows Fig. 2, where the concave and convex shape (respectively) is approximated by using 1/T as an argument of the linear approximation, and we use a logarithmic scale of reff in the expression for reff to avoid negative values at low temperature. Figure c also shows LR when calculated with the assumption of η=0.6 (SCALIPSO,η=0.6) as in the standard processing of CALIPSO data. This corresponds to the empirically found relationship in Fig. 12a and Fig. 9 with a local maximum at 225 K. LR at wavelengths other than 532 nm is approximated by 0.8λ-532532, where λ is lidar wavelength in micrometres (µm) and 0.8 is the approximate value of the 1064 nm / 532 nm colour ratio. The parameterisation of LR (S in Fig. c) spans about the same range of values as reported by Fig. 5.6 (20 to 60 sr) and (10 to 60 sr). Based on CALIPSO observations, determined that while the effective LR of global ice clouds at a lidar wavelength of 532 nm is mostly clustered around 17 sr, horizontally oriented plates produce a much lower effective LR below 10 sr caused by specular reflection. These results are close to our parameterisation of effective LR (SCALIPSO′). In the current version of the lidar simulator we do not parameterise horizontally oriented plates, but in a future version they could be taken into account by parameterising their concentration based on temperature . For the ALCs we use the same constant value of the multiple-scattering coefficient η=0.7 as for liquid cloud droplets (Sect. ).

Model cloud is defined by the liquid and ice mass mixing ratio as well as the cloud fraction in each atmospheric layer. The lidar simulator simulates radiation passing vertically at a random location within the grid cell. Therefore, it is necessary to generate a random vertical cloud overlap based on the cloud fraction in each layer, as the overlap is not explicitly defined in the model output. Two common methods of generating overlap are the random and maximum–random overlap methods . In the random overlap method, each layer is either cloudy or clear with a probability given by CF, independent of other layers. The maximum–random overlap method assumes that adjacent layers with non-zero CF are maximally overlapped, whereas layers separated by zero CF layers are randomly overlapped. COSP implements cloud overlap generation in the Subgrid Cloud Overlap Profile Sampler (SCOPS) . The ALC simulator uses SCOPS to generate 10 random subcolumns for each profile using the maximum–random overlap assumption as the default setting of a user-configurable option. The attenuated volume backscattering coefficient profile and cloud occurrence can be plotted for any subcolumn. Due to the random nature of the overlap, the attenuated volume backscattering coefficient profile may differ from the observed profile even if the model is correct in its cloud simulation. The random overlap generation should, however, result in unbiased cloud statistics.

Due to a finite FOV of the lidar receiver, a fraction of the laser radiation scattered forward will remain in the FOV. Therefore, the effective attenuation is smaller than calculated with the assumption that all but the backscattered radiation is removed from the FOV and cannot reach the receiver. The forward scattering can be repeated multiple times before a fraction of the radiation is backscattered, eventually reaching the receiver. To account for this multiple-scattering effect, the COSP lidar simulator uses a multiple-scattering correction coefficient η, by which the volume scattering coefficient is multiplied before calculating the layer optical thickness . The theoretical value of η is between 0 and 1 and depends on the receiver FOV and optical properties of the cloud. For CALIOP at λ = 532 nm a value of 0.7 is used in the COSP lidar simulator. implemented a fast approximate multiple-scattering code. This code has recently been used by in their ceilometer calibration method. They noted that η is usually between 0.7 and 0.85 for wavelengths between 905 and 1064 nm. The ALC simulator presented here does not use an explicit calculation of η but retains the value of η=0.7 for cloud droplets. The code of , “Multiscatter”, is publicly available (http://www.met.reading.ac.uk/clouds/multiscatter/, last access: 1 January 2021) and could be used in a later version of the framework to improve the accuracy of simulated attenuation and calibration.

ALC signal reception is affected by a number of sources of noise such as sunlight and electronic noise . Range-independent noise can be removed by assuming that the attenuated volume backscattering coefficient at the highest range gate is dominated by noise. This is true if the highest range is not affected by clouds or aerosol and if contributions from molecular scattering are negligible. The supported instruments have a range of approximately 8 (CL31), 15 (CL51, CHM 15k) and 30 km (MiniMPL). By assuming that the distribution of noise at the highest level is approximately normal, the mean and standard deviation can be calculated from a sample over a period of time such as 5 min, which is short enough to assume the noise is constant over this period and long enough to achieve accurate estimates of the standard deviation. The mean and standard deviation can then be scaled by the square of the range to estimate the distribution of range-independent noise at each range bin. By subtracting the noise mean from the measured attenuated volume backscattering coefficient we get the expected attenuated volume backscattering coefficient. The result of the noise removal algorithm is the expected attenuated volume backscattering coefficient and its standard deviation at each range bin.

ALCs often report the attenuated volume backscattering coefficient in arbitrary units (a.u.) or as NRB (MiniMPL). If they report it in units of m-1sr-1, these values are often not calibrated to represent the true absolute attenuated volume backscattering coefficient. Assuming that range-dependent corrections (overlap, dead time and after pulse) have been applied to the attenuated volume backscattering coefficient in a.u., the reported attenuated volume backscattering coefficient is proportional to the true attenuated volume backscattering coefficient (inclusive of noise backscattering). In order to have a comparable quantity to the lidar simulator and consistent input to the subsequent processing (e.g. cloud detection), calibration by multiplying by a calibration coefficient is required. Formally, the units of the calibration coefficient depend on the units of backscattering recorded by the instrument, which are m-1sr-1 in CL31 and CL51, unitless in CHM 15k, and µs-1µJ-1km2 in MiniMPL; i.e. the units of the calibration coefficient are m-1sr-1 / (instrument units). In the following discussion, we leave out the units. Several methods of calibration have been previously described: calibration based on LR in fully attenuating liquid stratocumulus clouds , calibration based on molecular backscattering and calibration based on a high-spectral-resolution lidar reference . In addition, calibration can be assisted by sun-photometer or radiosonde measurements .

Relatively large variability in the calibration coefficient has been determined for instruments of the same model . However, past studies can be useful for determining an approximate value of the coefficient before applying one of the calibration methods. For the CL51, reported a value of 1.2 ± 0.1 based on a multi-wavelength lidar reference. reported mean values of 1.4–1.5 for a number of CL31 instruments (software version 202). For CHM 15k, reported mean values between 0.3 and 0.8 for a majority of the instruments examined. The ALCF provides per-instrument default values of the calibration coefficient (Table ), but a unit-specific coefficient should be determined for an analysed instrument during the lidar data processing step.

Calibration based on LR in fully opaque liquid stratocumulus clouds has been successfully applied to large networks of ALCs. It utilises the fact that given suitable conditions the vertically integrated attenuated volume backscattering coefficient is proportional to LR of the cloud, which can be theoretically derived if the cloud droplet effective radius can be assumed. The theoretically derived value is about 18.8 sr for common ALC wavelengths and a relatively large range of effective radii . Another factor which needs to be known or assumed is the multiple-scattering coefficient, which tends to be about 0.7–1.0 in common ALCs. Due to its relatively simple requirements, this method is possibly the easiest ALC calibration method. The ALCF implements this calibration method by letting the user identify time periods with fully opaque liquid stratocumulus cloud, for which the mean LR is calculated. The ratio of the observed LR and the theoretical LR is equivalent to the calibration coefficient. This implementation, while very easy to perform, has multiple limitations, some of which are highlighted by .

Aerosol can cause additional attenuation and scattering, which results in LR that is different from the theoretical value by an unknown factor. Therefore, a frequent re-calibration may be necessary.

These limitations could be addressed in the future by (1) using sun-photometer observations as an optional input to determine the aerosol optical depth (AOD), (2) calculating the multiple-scattering coefficient more accurately (such as with the Multiscatter package of ), (3) calculating the water vapour absorption explicitly based on water vapour, temperature and pressure fields from a reanalysis or radiosonde profile data, (4) correcting the near-range backscatter based on the integrated attenuated volume backscattering coefficient distribution as a function of the height of the maximum backscatter Sect. 5.1, or (5) combining the attenuated volume backscattering coefficient profile with the temperature field from a reanalysis to exclude cold clouds.

Molecular (Rayleigh) backscattering can be accurately calculated if the temperature and pressure of the atmospheric profile are known (Sect. ). This can be employed for absolute calibration of ALCs. Given the low SNR of low-power ALCs, several hours of integration are required to identify the molecular backscattering . The molecular backscattering is attenuated by an unknown amount of aerosol with unknown LR, and the near-range backscattering is affected by a potentially inaccurate overlap correction. Therefore, this method alone produces calibration coefficients which depend on the atmospheric conditions. We found that all studied ALCs except for the CL31 are capable of observing the molecular backscattering (Sect. ). Therefore, this method may be used in addition to the liquid stratocumulus LR method for cross-validation of the calibration.

Cloud is the most strongly attenuating feature in ALC attenuated volume backscattering coefficient measurements. Due to this attenuation, the lidar signal is quickly attenuated in thick cloud and can fall below the noise level before reaching the top of the cloud. This means that the first cloud base can be detected reliably (unless the cloud is too thin or too high and obscured by noise), while the cloud top or multi-layer cloud cannot be observed reliably under all conditions. The opposite is true for spaceborne lidars, which can detect the cloud top reliably but cannot always detect the cloud base. Therefore, ALC observations can be regarded as complementary to spaceborne lidar observations. By applying a suitable algorithm, one can detect CBH and CTH as well as identifying cloud layers. Instrument firmware often determines CBH and sometimes cloud layers as part of its internal processing, often using an undisclosed algorithm which is not comparable between different instruments and potentially not even different versions of the instrument firmware . compared a large number of ALCs and found differences of up to 70 m between the reported CBH, and others found relatively large differences as well . Alternatively to instrument-reported CBH and cloud layers, it is possible to detect cloud based on the attenuated volume backscattering coefficient profile. A relatively large number of cloud detection algorithms have been proposed . We use a simple algorithm based on an attenuated volume backscattering coefficient threshold applied to the denoised backscatter, assuming that the noise can be represented by a normal distribution at the highest range, which is unlikely to contain cloud or aerosol if the instrument is pointing vertically (this may not be true, however, for CL31, which has a maximum range of just 7.7 km). This assumption neglects the range-dependent molecular backscattering, which is relatively small at the ceilometer wavelengths examined (910 and 1064 nm). A cloud mask is determined to be positive where the attenuated volume backscattering coefficient is greater than a chosen threshold plus 5 standard deviations of noise at the given range. In addition, the observed attenuated volume backscattering coefficient can optionally be coupled with a simulated attenuated molecular volume backscattering coefficient and molecular backscattering removed from the observed backscattering prior to cloud detection. This improves the results in the boundary layer, especially with instruments which operate in the visible range and are therefore affected by large molecular backscattering (MiniMPL). A threshold of 2×10-6m-1sr-1 was found to be a good compromise between false detection and misses in our Southern Hemisphere data relatively unaffected by anthropogenic aerosol. Our observed and simulated results show that cloud backscatter is generally higher than 1×10-6 m-1sr-1, and a threshold below 2×10-6 m-1sr-1 results in excessive false detection due to aerosol, molecular backscattering and noise from sunlight. The threshold is an adjustable option of the ALCF. Users are encouraged to change this value if, for example, the data are affected by a large amount of aerosol. This value is above the maximum molecular backscattering, which is approximately 1.54×10-6 m-1sr-1 at the surface in the case of the MiniMPL (wavelength 532 nm). Noise is not simulated by the lidar simulator, but the cloud detection algorithm allows for coupling of simulated and observed profiles, whereby the noise standard deviation is taken from the corresponding location in the observed profile. With 5 min averaging, when the standard deviation of noise is relatively low, we found that the coupling does not make substantial differences in the detected cloud (not shown). While the threshold-based algorithm is less sophisticated than other methods of cloud detection, the vertical resolution of the simulated attenuated volume backscattering coefficient is likely too low and the vertical derivatives of the simulated attenuated volume backscattering coefficient too crudely represented (Table ) to apply any algorithm based on the vertical derivatives of the attenuated volume backscattering coefficient. Using the same cloud detection algorithm on the observed and simulated attenuated volume backscattering coefficient is essential for an unbiased one-to-one comparison of cloud.

Previous studies have noted that ceilometers which utilise the wavelength of 910 nm, such as the Vaisala CL31 and CL51, are affected by additional absorption of laser radiation by water vapour . The wavelength coincides with water vapour absorption bands between 900 and 930 nm, while the other common ceilometer wavelength of 1064 nm is not affected. reported that it can cause absorption of the order of 20 % in the extratropics and 50 % in the tropics. The lidar simulator does not currently account for this. However, as the water vapour concentration is available from the reanalyses and models, it should be possible to use a line-by-line model to calculate the water vapour volume absorption coefficient for each vertical layer during the integration process. Water vapour also affects calibration of the observed attenuated volume backscattering coefficient. In order to use the liquid stratocumulus LR calibration method, the attenuated volume backscattering coefficient has to be corrected for water vapour absorption to achieve high-accuracy calibration. used a simplified approach based on a parameterised curve and reported a difference from explicit radiative transfer calculations of 2 % in the United Kingdom atmosphere (Middle Wallop). In the future either approach should be used to include water vapour absorption in the simulator or remove the effect of water vapour absorption from the observed lidar attenuated volume backscattering coefficient to achieve an improved one-to-one comparison between the observations, reanalyses and models.

We analysed 13 d of CL51 observations from the Cass field station in late winter. Due to the location of the station at a relatively high altitude in a varied terrain of the Southern Alps, the models, with their relatively coarse horizontal grid resolution, do not represent the terrain and position accurately. The orography representation of the models meant that the virtual altitude of the station was 1115 m (AMPS), 1051 m (ERA5), 401 m (JRA-55), 914 m (MERRA-2) and 428 m (UM). The virtual position, which is the centre of the nearest model grid cell to the site location, ranged from relatively close in the Southern Alps (AMPS, ERA5, MERRA-2, UM) to relatively far on the west coast of NZ (JRA-55) depending on the horizontal resolution of the grid. The time period examined was characterised by diverse cloud occurrence with periods of low cloud and precipitation, mid-level cloud, fog, high cloud, and clear skies. Precipitation, currently not simulated by the lidar simulator, was present in about 18 % of the observed attenuated volume backscattering coefficient profiles, as determined by visual inspection. Figure b shows that predominantly low cloud and precipitation between the ground and 3 km a.s.l. in 25 % of profiles was observed. Cloud between 3 and 12 km a.s.l. was observed about evenly in 2 % of profiles. While the reanalyses and models were able to partially reproduce the peak of cloud occurrence near 1 km a.s.l., the peak they displayed is less vertically broad than observed, and in the UM the peak was much weaker than observed. The lack of precipitation simulation might have also contributed to this apparent difference between observed and simulated cloud. Above 3 km a.s.l., the reanalyses and models tended to overestimate cloud, with only ERA5 and JRA-55 simulating close to the observed cloud occurrence. The observed total CF was 61 %. AMPS overestimated this value by 5 percentage points (pp), and ERA5 and the UM reproduced almost the exact value (within 1 pp), while the other reanalyses (JRA-55 and MERRA-2) underestimated CF by about 15 pp.

We also analysed 13 d of CL31 and MiniMPL observations from the Lauder station in summer. During the time period relatively diverse cloud was observed, with periods of low, middle and high cloud, clear sky, and a small fraction of profiles with precipitation (about 3 %). The altitude of the station of 370 m a.s.l. generally had a much higher equivalent in the reanalyses and models at 565 m (AMPS), 642 m (ERA5), 681 m (JRA-55) and 786 m (MERRA-2) due to the presence of hills in the surrounding region (the station is in a high valley), with the exception of the UM wherein the altitude was 385 m. The virtual station position in the reanalyses and models ranged from relatively close to the station in the same geographical region (AMPS, ERA5), to a nearby location in a more hilly region (JRA-55), a relatively distant location in the adjacent Dunstan Mountains (MERRA-2) and a relatively distant location in central Otago (UM). Figure c shows that the CL31 observed relatively even cloud occurrence between the ground and 3 km a.s.l. at 8 %, falling off to about 3 % between 4 and 8 km a.s.l. (the maximum lidar range of CL31 is 7.7 km). The MiniMPL observed a much weaker attenuated volume backscattering coefficient than CL31 below 3 km a.s.l., which was identified as an overlap calibration issue in the MiniMPL. The MiniMPL observed substantial amounts of cloud above 8 km not present in the CL31 observations due to its range limitation. Overall, the observed cloud occurrence had two peaks at the ground to 3 km a.s.l. and at about 9 km a.s.l. The simulated cloud occurrence was generally underestimated between the ground and 5 km a.s.l., with the exception of the UM which reproduced the lower half of the peak accurately and ERA5 which reproduced the upper half of the peak accurately. Above 5 km a.s.l., the cloud occurrence was well reproduced in ERA5 and JRA-55 but strongly overestimated in AMPS, MERRA-2 and the UM. The reanalyses and models also tended to have two peaks at about 2 and 11 km a.s.l., but these were quite different from the observed peaks, with the lower peak underestimated by about 5 pp in the reanalyses and models and the higher peak overestimated by about 5–10 pp. The total CF was observed as 45 % and 60 % by CL31 and MiniMPL, respectively. CF observed by the MiniMPL was likely higher due to its higher maximum lidar range (CL31 missed substantial amounts of high cloud due to this limitation). The total CF was strongly underestimated by the reanalyses and models by up to 31 pp (CL31) and 28 pp (MiniMPL), with the exception of the UM which simulated the correct CF within 3 pp.

The Christchurch observations were made during a total of 23 d in middle to late winter. The cloud situations were characterised by the frequent occurrence of low cloud and fog, with relatively diverse mid-level and high-level cloud and periods of clear sky also present (not shown). Precipitation was present in about 9 % of profiles and fog in about 11 % of profiles. As the site location is relatively flat (Canterbury Plains), the models did not have any difficulty in reproducing the altitude of the site, which was 32 m (AMPS), 72 m (ERA5), 143 m (JRA-55) and 76 m (MERRA-2). The virtual location was within the boundaries of the city (AMPS), on the Canterbury Plains close to the city boundaries (ERA5, MERRA-2) and over Lake Ellesmere about 20 km from the city (JRA-55). Figure d shows that the co-located CHM 15k and MiniMPL observed a strong peak of cloud occurrence of 26 % (CHM 15k) at about 500 m a.s.l. This was likely due to the combined precipitation and fog as well as false detection of aerosol as cloud. The observed cloud occurrence had a local minimum of 2 % at about 5 km a.s.l., a secondary peak of 5 % at 7 km a.s.l. and fell off 0 % at 11 km a.s.l. The CHM 15k and MiniMPL observations showed inconsistencies of up to 4 pp. The reanalyses and models underestimated low cloud by 5–10 pp. With the exception of AMPS, they underestimated mid-level cloud by about 5 pp and represented high cloud relatively accurately. The total CF observed was 68 %, while the reanalyses and models strongly underestimated CF by up to 34 pp (JRA-55), with common underestimates of around 20 pp.

Figures , and show images of the attenuated volume backscattering coefficient for three separate days taken from the three case studies. The selected days represent some of the best-matching profiles and demonstrate how well the reanalyses and models can simulate cloud under favourable conditions. As can be seen in the figures, ERA5 and the UM perform the best in terms of the temporal and height accuracy of the simulated cloud (Figs. c, f, c, f, c). This is likely due to the high output temporal resolution of the UM and ERA5 of 20 min and 1 h, respectively. The UM and ERA5 were able to represent the relatively fine structure of cloud and to a lesser extent the optical thickness (inferred from the strength of backscattering) of the cloud. Deficiencies, however, are readily identifiable. The low cloud in the UM (Fig. f) covers too large of an area relative to observations (Fig. a) and the high cloud has a greater vertical extent in the UM. Likewise, the altocumulus cloud observed in Fig. a is shifted by several hours in the UM (Fig. f). The stratocumulus and nimbostratus cloud, visually identified based on the attenuated volume backscattering coefficient profiles, in ERA5 (Fig. c) is markedly lower than observed (Fig. a), as well as optically thicker than in reality. The mid-level cloud in ERA5 (Fig. c) was located about 2 km higher than observed (Fig. a). Precipitation observed in Fig. a towards the end of the analysed period was not present in the ERA5 simulated profile (Fig. c) due to lack of precipitation simulation in the current lidar simulator (even though rain- and snow-specific content is available from the reanalysis). AMPS and MERRA-2 had lower cloud representation accuracy. They managed to capture the overall structure of clouds (Figs. b, d, b, d, b, d), but substantial discrepancies were present, some of which were likely due to the relatively low temporal resolution of 3 h. AMPS, however, has a relatively high horizontal grid resolution of 21 km. This demonstrates that factors in the model other than resolution have a stronger influence on the quality of cloud simulation. JRA-55 was identified as the last in terms of cloud representation accuracy. JRA-55 has the lowest temporal resolution of the studied reanalyses and models of just 6 h, as well as the lowest horizontal grid resolution of 139 km. Therefore, it cannot be expected to capture any fine details of cloud. In the presented profiles (Figs. e, e, e) one can see that the cloud is only crudely represented. JRA-55 was able to represent the stratocumulus cloud in Fig. a, although its temporal extent and optical thickness were overestimated. The mid-level clouds in Figs. a and a were relatively well represented in terms of height and optical thickness given the low temporal resolution of the reanalysis. We stress that a direct attenuated volume backscattering coefficient profile intercomparison is highly dependent on the temporal resolution of the model output. The statistical intercomparison, however, should still give unbiased results if the cloud physics are accurately simulated by the atmospheric model.

Figures a, a and a also show the effective LR of observations calculated by integrating the vertically attenuated volume backscattering coefficient (Sect. ). If the attenuated volume backscattering coefficient is properly calibrated, under fully attenuating cloud conditions effective LR converges to the theoretical value of the LR of liquid cloud droplets (approximately 18.8 sr at near-IR wavelengths) multiplied by the multiple-scattering coefficient (approximately 0.7; Sect. ).

Figure shows attenuated volume backscattering coefficient histograms as a function of height for small values of the coefficient (up to 2×10-6 m-1sr-1) observed and simulated at the sites of the case studies, calculated for the entire time period of each case study. The scale of values is below cloud backscattering and therefore shows backscattering which results from molecular and aerosol scattering and noise. Molecular backscattering depends on the atmospheric pressure and temperature as well as the lidar wavelength. It causes the main “streak” (a local maximum) visible in each of the histograms. The observed molecular attenuated volume backscattering coefficient at the surface approximately corresponds to the theoretically calculated value at each wavelength: 0.0906×10-6 m-1sr-1 (λ=1064 nm), 0.172×10-6 m-1sr-1 (λ=910 nm) and 1.54×10-6 m-1sr-1 (λ=532 nm) at 1000 hPa and 20 ∘C (Table ). The molecular backscattering in the boundary layer is, however, superimposed on backscattering by aerosol and cloud. In the case of the MiniMPL observations at the Christchurch site (Fig. i), the molecular attenuated volume backscattering coefficient streak has multiple secondary streaks. These are caused by different levels of attenuation by cloud and aerosol during the period of the observations. These secondary streaks were also partially reproduced by the simulator (Fig. j). A smaller portion of the width of the streak is also caused by fluctuations of atmospheric temperature and pressure. Under suitable conditions, the molecular attenuated volume backscattering coefficient can be used for absolute calibration of an instrument. With the exception of CL31 (Fig. c), the molecular backscattering can be identified in the observed attenuated volume backscattering coefficient in each case. Therefore, it is possible to choose a calibration coefficient such that the observed and simulated molecular attenuated volume backscattering coefficients overlap. This can be considered a viable alternative to the liquid stratocumulus LR calibration method or as a means of cross-validating the instrument calibration. However, it should be noted that the accuracy of this method is affected by an unknown amount of aerosol attenuation. Cloudy profiles can be filtered when calculating the histogram, and therefore the effect of cloud attenuation can be minimised. In addition to the molecular attenuated volume backscattering coefficient streak, there is a zero-centred streak visible in the histograms. This is caused by noise when the signal is fully attenuated by cloud. Lastly, a zero-centred “cone” of noise is visible in the observed attenuated volume backscattering coefficient, increasing with the square of range. The size of this cone is particularly large in the case of the CL31 (Fig. c), which is most likely the result of its low receiver sensitivity and low power compared to the other instruments. The standard deviation of the cone at the furthest range is used to determine the noise standard deviation used by the cloud detection algorithm (Sect. ).

Figure shows the same information as Fig.  but for clear-sky profiles only. Here, it can be seen that the zero-centred peak caused by the complete attenuation by cloud is no longer present. There is a clear overlap between the centre of the noise cone and the simulated molecular attenuated volume backscattering coefficient; i.e. the noise cone is centred at the observed molecular attenuated volume backscattering coefficient. This is visible with all instruments including CL31 (Fig. c), for which the overlap between the observed and simulated molecular attenuated volume backscattering coefficient is most clearly visible at about 1 km a.s.l. Below 1 km a.s.l., the effect of boundary layer aerosol distorts the molecular attenuated volume backscattering coefficient by an unknown quantity. The clear-sky histograms as shown in Fig.  may therefore be preferable to the all-sky histograms in Fig.  for calibration by fitting the molecular attenuated volume backscattering coefficient. The dead time, after-pulse and overlap MiniMPL calibration supplied by the vendor appears to be deficient and causes range-dependent bias in the attenuated volume backscattering coefficient profile.

We now examine the noise in each instrument using the ALCF. Figure  shows the distribution of the standard deviation of backscatter noise determined at the highest observable range of each instrument and range-scaled to 8 km. It can be seen that the CL31 is affected by the greatest amount of noise, peaking at about 2×10-6 m-1sr-1. This is at the threshold of cloud detection of 2×10-6 m-1sr-1. Therefore, thin cloud may be obscured by noise at higher ranges with this instrument. The MiniMPL, operating in the visible spectral range, shows a strongly bimodal distribution of the attenuated volume backscattering coefficient noise depending on sunlight. During daytime, it peaks at about 0.7×10-6 m-1sr-1, which is the second highest of the analysed instruments. During nighttime, it peaks at about 0.02×10-6 m-1sr-1, which is the lowest of the analysed instruments. The CHM 15k and CL51 peak between the nighttime and daytime MiniMPL at about 0.05×10-6 m-1sr-1. The CL31, CL51 and CHM 15k show a slight reduction of noise during nighttime, presumably because of a small amount of incoming solar radiation at near-IR wavelengths. The difference between the nighttime and daytime attenuated volume backscattering coefficient noise in the MiniMPL has been previously analysed by (Fig. S3), and these results confirm their findings.