Aerosols in the atmospheric column are a significant source of uncertainty for passive remote-sensing (e.g., from a satellite) retrievals of cloud optical and microphysical properties. Thick aerosol layers can be wrongly identified as clouds, and aerosols above clouds will lead to biases in cloud retrievals (Meyer et al., 2013). Biases in cloud detection and retrievals of cloud microphysics will lead to uncertainties in properties important for quantifying Earth's radiative budget. On the other hand, clouds wrongly identified and retrieved as aerosol may have similar impacts on estimates of aerosol radiative forcing and effects on climate and clouds. The Moderate-resolution Imaging Spectroradiometer (MODIS; Barnes et al., 1998) has been flying on the polar-orbiting (at 705 km altitude) satellites known as Terra (since 2000) and Aqua (since 2002). Viewing a 2300 km swath, split into 5 min granules, MODIS measures radiance (or reflectance) in 36 spectral channels, of which 19 are in reflective solar bands, with the other 17 being terrestrial infrared emission. All bands are in at least 1 km spatial resolution. Based on MODIS observations, separate teams have created high-quality retrievals of both cloud (e.g., the M{O/Y}D06_L2 (MxD06); Platnick et al., 2003) and aerosol (M{O/Y}D04_L2 (MxD04; Levy et al., 2013) properties. Current operational cloud retrieval includes methods for clearing the aerosols misidentified as clouds from retrieval attempts (Zhang and Platnick, 2011; Pincus et al., 2012). Similarly for aerosol retrievals, much effort has been made to reclassify scenes that are, in fact, heavy dust or smoke as “not cloudy”. Therefore, for both teams, uncertainty whether a particular sample is cloud-covered or contains primarily aerosols, and how to propagate this uncertainty into retrieval products, remains a topic of great interest. A major problem is that there is no absolute ground truth to confirm or deny these decisions in all cases. Ground-based instrumentation such as sun photometers (Holben et al., 1998) may not be able to accurately distinguish between aerosol and thin clouds due to limited spectral range, generally reaching only up to a wavelength of 1.024 µm. Newer sun photometers do provide information up to 1.64 µm, but they are not present at every ground site. The ground sites in Brazil that fall within the area we studied in this paper carry the older instrumentation. The best wavelengths for detecting cirrus clouds are located around 1.38 and 1.8 µm. There are also efforts to retrieve aerosol optical depth above cloud layers (Meyer and Platnick, 2015; Meyer et al., 2013). Validation for such algorithms is often done using lidar and radar data (Ackerman et al., 2008; Notarnicola et al., 2011). However as current spaceborne lidar and radar instruments have fixed nadir view, the amount of such data acquired in tandem with an instrument like MODIS is rather limited.

While a global meteorological model cannot be directly used to validate observations and retrievals due to the many assumptions and simplifications commonly made in the dynamic core and physics parameterizations (Rienecker et al., 2008), one can use such a model to learn about sensitivities of retrieval algorithms. As global models such as the Goddard Earth Observing System Model, Version 5 (GEOS-5; Rienecker et al., 2008; Molod et al., 2012), become increasingly realistic when simulating aerosols and clouds over complex surface terrain, we can apply detailed radiative transfer (RT) to simulate how these scenes would appear to a satellite such as MODIS, and how operational algorithms would in turn retrieve the specified conditions. Since the specified model aerosol and cloud properties of the scene are known, one can then characterize the ability (and uncertainties) of standard (e.g., MxD04 or MxD06) retrievals in these scenes. Thus, one can evaluate the current (and possibly historical) performance of cloud and aerosol properties retrievals. Application and evaluation of these simulation capabilities for known instruments is also an important step in development of Observing System Simulation Experiments for future observing missions.

The Multi-Sensor Cloud and Aerosol Retrieval Simulator (MCARS; Wind et al., 2013) is a modular, flexible tool, in which model output is coupled with a radiative transfer code in order to simulate top-of-atmosphere (TOA) radiances that may be measured by a remote-sensing instrument if it were passing over the model fields. In principle, MCARS can be applied to any model/visible-IR radiometer combination. The simulation complexity is only limited by computer power. However, in this paper, the MCARS continues to use the combination of GEOS-5 model and discrete ordinate radiative transfer (DISORT) code (Stamnes et al., 1988) to simulate MODIS radiances. In Wind et al. (2013), the MCARS simulated only clouds; here we add microphysical properties of aerosols present in scenes we examine.

The approach we take is to populate the operational MODIS Level-1B calibrated radiance files with TOA radiances simulated from GEOS-5 model output and DISORT. For a given time and location, MODIS provides a particular geometry of observation. Since GEOS-5 simulates clouds and aerosols interactively, we can replace the MODIS-observed reflectance data with the simulated radiance product derived from the model. Then we run the standard aerosol (MxD04_L2) and cloud (MxD06_L2) retrieval codes and compare retrieval result to the known GEOS-5 source data. The discrepancies diagnosed by this device can then be contrasted to discrepancies obtained by comparing the real operational retrievals to independent, trusted observations (e.g., aerosol optical depth, AOD, from AErosol RObotic NETwork – AERONET). To the extent that simulated and real statistical comparisons match, we can use capabilities of the MCARS code to examine the causes for such discrepancies, and hopefully identify opportunities for algorithm improvement. Since the aerosol retrieval is underdetermined (Levy et al., 2013) and a number of assumptions must be made, the MCARS simulation approach is highly valuable as individual assumptions can be tested in isolation. The MCARS code has sufficient flexibility to test impacts of settings of single operational retrieval code parameters without interference from other components.

Section 2 describes the GEOS-5 aerosol properties and their addition into MCARS. Section 3 describes the MODIS aerosol product. Section 4 discusses case selection for the current analysis. It shows the selected scenes simulated by MCARS and describes other special simulation settings available that provide additional analysis capabilities. This section also presents analysis of retrieved aerosol properties as compared to the specified “ground” truth that served as input to the simulations. Finally, Sect. 5 discusses next steps in the continuing MCARS development.

The MODIS “dark-target” (DT) aerosol product is described in detail in Levy et al. (2013) and references therein. In this section we will give a brief overview of the DT algorithm as applied to MODIS observations.

The standard MODIS aerosol properties retrieval algorithm is a 10 km resolution product calculated from a detailed analysis of 10×10 boxes of 1 km MODIS pixels. A necessary constraint for the algorithm is that the underlying surface is dark in visible and shortwave-IR wavelengths. There are two separate algorithm paths for ocean and land.

Pixels that are suspected to be cloudy or too bright or too noisy are discarded using conditions described in Levy et al. (2007). Once the data sample is prepared, a spectral profile of average TOA spectral reflectance is created and compared against a lookup table. If a match is found, values for AOD and fine-mode aerosol weighting (FMW) are then returned.

In this paper we will focus on the land algorithm. Full description of the ocean algorithm can be found in Levy et al. (2013). Over land, even though there is greater variability of underlying surface than over ocean and thus greater uncertainty in retrieved aerosol properties, aerosol retrieval is still achievable. Over vegetated and dark-soiled surfaces, Kaufman et al. (1997) found that surface reflectance values for red (e.g., 0.65 µm) and blue (0.47 µm) wavelengths are correlated with the surface reflectance in a short-wave infrared (SWIR) band (e.g., 2.13 µm). The land algorithm uses 0.47, 0.65, and 2.13 µm channels for the main retrieval and 0.55, 0.86, and 1.24 µm channels to give additional surface constraints.

The aerosol lookup table (LUT) is calculated for black surfaces and sea-level pressure. There are three fine-particle model types and one coarse-particle model type of aerosols used for dust based on climatology of AERONET inversion data (Dubovik et al., 2002). Each model type is multi-lognormal and is represented by size distribution, particle shape, and complex refractive indices. The three fine-dominated models are differentiated primarily by single scattering albedo (SSA) in mid-visible wavelengths: urban/industrial type (SSA ∼ 0.95), near-source biomass burning (SSA ∼ 0.85), and a moderately absorbing type (SSA ∼ 0.90) to cover all other cases. For each aerosol type, the LUT includes TOA reflectance for a variety of angles and AOD referenced to 0.55 µm.

Even with the constraints on surface reflectance, the aerosol retrieval does not have enough information to select between different aerosol types. Therefore, the relative proportion of fine-mode and coarse-mode aerosols must be prescribed so that, coupled with surface constraints, a best match can be found in the LUT for TOA spectral reflectance in the blue, red, and SWIR wavelengths. The difference between TOA and nearest LUT reflectance is the fitting error.

With Levy et al. (2013) and previous studies, the primary validation of the MODIS product is by detailed co-location with ground-based sun photometer data, especially the Aerosol Robotic Network (AERONET; Holben et al., 1998). In this way, Levy et al. (2013) have defined the expected error envelope for the 0.55 µm AOD as ±(0.05 + 15 %). While spectral surface reflectance is also retrieved, it does not tend to compare well with values obtained from the sun photometers. Note that the expected error is defined upon mutually retrieved data. This means that satellite and sun photometer both observe enough clear sky to retrieve AOD.

Also, while AERONET is well distributed about the globe, there are many situations for which MODIS retrieves aerosol, but there are no AERONET data available to compare with. Thus, there is no way to determine whether the MODIS aerosol retrieval has made reasonable choices, either for pixel selection, for cloud screening, or for aerosol model type and surface reflectance assumptions.

This motivates our use of the MCARS. Having full knowledge of underlying atmospheric, cloud, aerosol, and surface parameters MCARS allows us to see deeper than AERONET would and over a much wider spatial area.

For each granule, we ran the simulations in several modes with varied run-time option settings. For example, the cloud-only mode corresponds to a clean atmosphere with no aerosols; this mode was the only one considered in Wind et al. (2013). In the current paper we consider additional options afforded by the implementation of the aerosol effect. The cloud-free option runs atmosphere and aerosols without any clouds. When clouds are turned off, we do not alter the humidity profiles to dry the atmosphere out; because of the high relative humidity conditions where clouds are present, aerosol hygroscopic effects are pronounced there as well. The full simulation option includes atmosphere (temperature, humidity, and ozone profiles), all clouds, and all aerosols. There is also an additional option where the user can remove both clouds and aerosols and be left with just the atmosphere itself. Rayleigh scattering is always included by default, but the user also has control over whether or not to turn it off. While this no-cloud/no-aerosol mode could be useful for studies of atmospheric correction methods, we do not exercise it here, as our primary goal here is to investigate the performance of the MODIS aerosol algorithms.

The cloud-free mode of operation is convenient when complex cloud and aerosol scenes are being investigated and one wishes to quantify or remove possible impacts of cloud contamination on the retrieval. Figure 1 shows RGB images constructed from simulated MODIS L1B for the different modes of execution for the Brazil 1 case. MODIS aerosol retrievals were produced for radiance simulations including atmosphere, cloud, and aerosols (Fig. 1c) and for radiance simulations excluding clouds (Fig. 1d). Rayleigh scattering is included in these simulations.

These Brazil cases came from source MODIS Aqua granules and had been processed using the MODIS Aqua aerosol properties retrieval algorithm. Therefore in this section we will use MYD04 designation for the MODIS aerosol properties retrieval result. There are some slight differences between the MODIS Terra (MOD04) and MODIS Aqua (MYD04) algorithms due to calibration differences between the two instruments (Levy et al., 2013).

The scatter diagrams in Fig. 2 compare AOD retrieved using the MYD04 algorithm to the specified GEOS-5 AOD, which is considered the ground truth in this case. MODIS aerosol retrievals are commonly compared to co-located AERONET AOD measurements (Correia and Pires, 2006; Levy et al., 2007; Remer et al., 2005) for validation. Unlike comparisons of actual MODIS data with AERONET, the matchups in Fig. 2 did not require any temporal averaging or aggregation because for every MYD04 retrieval there is a directly corresponding input data point with all aerosol, cloud, and atmospheric properties readily available. The overall shape of resulting scatterplots turned out to be quite similar to existing MYD04–AERONET comparisons for this region such as those that appear in Correia and Pires (2006) and Fig. 3. Figure 3 shows an actual comparison of AERONET observations for months of July and August and all available Aqua MODIS-collocated observations from year 2002 through 2015. The chosen AERONET sites – Campo_Grande_SONDA, Sao_Paulo, and CUIABA-MIRANDA – fall in the general vicinity of the two Brazil cases selected for study. They of course represent a tiny sample of the geographical area covered by the MCARS data, just three points out of 2.7 million collocated samples that MCARS provides, but they display a similar shape of the relationship between ground truth and MYD04 retrieval.

MCARS is a fully configurable system where source input for all synthetic radiances can be controlled at all times so that any resulting retrieval can be examined in great detail insofar as the particular setup of model input and radiative transfer core allows. For these smoke cases we used these capabilities to investigate further the specific reasons why the MYD04 retrievals tend to underestimate AOD for smoke aerosol. The first test we carried out was to examine the performance of MYD04 cloud mask, which is an aerosol-specific product (Remer et al., 2005), different from the operational MODIS cloud mask product (Ackerman et al., 2006). The main purpose of this analysis was to ascertain whether cloud contamination could account for some of the discrepancies. Individual panels in Fig. 2 show the results of retrievals run with and without the cloud layers. Panels (a) and (b) show results for Brazil 1 and panels (c) and (d) are for Brazil 2. Brazil 1 case does not show any significant cloud contamination. The MYD04 cloud mask does a very good job of avoiding cloud. Brazil 2 does show some very minor cloud contamination as evident by a small cluster of high MYD04 AOD and low GEOS-5 AOD that disappears when clouds are removed from simulation. However the overall shape of the scatterplot when clouds are removed remains unchanged.

The aerosol models used in the MYD04 retrievals make assumptions about the smoke aerosol optical properties, which may not match the aerosol optical assumptions in GEOS-5 (Levy et al., 2007). In cases of complex aerosol mixtures or if the model selected by the MYD04 algorithm does not correspond to the aerosols provided by GEOS-5, large retrieval errors should result. Figure 4 shows the species mixture for Brazil 1 (a) and Brazil 2 (b) cases. They are both dominated by carbon, organic carbon from smoke in particular, with very little, if any, contribution from other species. Therefore these particular cases can be treated as having a single aerosol type present without significant error. MYD04 retrieval output indicates that either moderately or strongly absorbing smoke had been selected, which is very appropriate for the selected granules. Thus any discrepancy in selection of aerosol model does not explain the scatterplot shape.

Another candidate source of retrieval error is any difference between the phase functions assumed by MYD04 and GEOS-5. We ran the initial simulations simply using the Henyey–Greenstein (HG) phase function approximation and then repeated the same simulation using the phase functions provided by the OPAC database described in Sect. 2. Figure 5 shows the result for Brazil 1 and Brazil 2 cases using the cloud-free run with HG phase function versus OPAC phase function. For the smoke aerosol cases studied, the specific phase function shape does not appear to have a significant impact on the differences seen between MYD04 and GEOS-5.

An additional potential source of error for aerosol retrievals over land is the surface albedo and its variation over a 10×10 km area. We performed a simulation where we selected a single surface albedo profile from a successful MYD04 retrieval and fixed the surface albedo to that particular surface albedo profile for the entire granule. The test albedo profile used is listed in Table 1. The profile corresponds to a very dark vegetated surface, the ideal conditions for the MYD04 land algorithm. Figure 6 shows the effect of using a constant surface albedo for Brazil 1 and Brazil 2 cases. Whereas use of constant surface albedo reduces the scatterplot spread and so allows us to potentially quantify the effect of surface inhomogeneity on MYD04 land retrievals, it does not alter the overall bias characteristics of scatterplots.

With all the factors of model selection, surface parameters and cloud contamination taken into account, we now turn our attention to the aerosol scattering properties, the spectral single scattering albedo (SSA) in particular. Figures 7 and 8 show the spectral profile of aerosol SSA for Brazil 1 and Brazil 2 cases respectively for the first seven MODIS channels. This aerosol SSA is a bulk quantity, integrated over all layers and combines all 15 available aerosol species. However the cases under consideration are heavily dominated by carbon with negligible amounts of dust and sulfate. In this particular case the additional uncertainties that would arise from a mixture of aerosols with different scattering properties do not present an issue. The single scattering albedo remains quite high until we reach the 1.2 µm channel, MODIS band 5, and beyond. Then it drops precipitously. AERONET is only able to provide direct inversion retrievals of single scattering albedo for four wavelengths out to a maximum wavelength of 1.024 µm (Dubovik and King, 2000; Dubovik et al., 2002). The rapid change in single scattering albedo for smoke aerosol modeled in GEOS-5 is related to aerosol humidification effects, both dilution effects and hygroscopic growth (Colarco et al., 2010, 2013). The net effect is that when humidity decreases, so does the single scattering albedo. Figure 9 shows a plot of OPAC single scattering albedo for a variety of column relative humidity values as a function of wavelength. (Colarco et al., 2013) The operational MODIS aerosol code assumes a constant 80 % relative humidity when the lookup tables are generated (Levy et al., 2007). It is a reasonable assumption as long as one does not attempt to use channels with wavelengths that are longer than 0.8 µm. The MYD04 algorithm however does use the 2.1 µm MODIS channel in retrieval, a channel that is sensitive to humidity. MCARS is particularly well suited to test for humidity impact on the retrieval accuracy. We carried out another experiment with fixed surface albedo and OPAC aerosol phase function shape, but we used the constant single scattering albedo values from the MODIS aerosol algorithm in the reflectance calculation that serves as input to the retrieval algorithm. The result is shown in Fig. 10. When humidification effects are not taken in consideration in the SSA calculation, MYD04 retrieval results closely line up with synthetic GEOS-5 source data. The underestimate of aerosol optical depth disappears, with Brazil 2 showing the most dramatic improvement. It appears that if MYD04 were to take into account humidification effects and implement a correction for single scattering albedo value as a function of column relative humidity, the result of comparison between MODIS and AERONET could be significantly improved for biomass-burning cases in Brazil and other locations with similar synoptic conditions.

The improvement is limited however to AODs higher than about 0.5. Relative humidity does not appear to have an effect on retrieved low AOD values. MYD04 product does not provide pixel-level retrieval uncertainty estimates. It is possible that the inherent uncertainty in performing retrieval using such a small signal is so high that it drowns out other effects. More studies may be conducted as to attempt to create a pixel-level estimate of retrieval uncertainty for aerosol optical properties retrievals.

The MODIS aerosol product performs a simultaneous retrieval of land surface reflectance and aerosol optical depth. After looking at the behavior of aerosol optical depth and making a recommendation for a possible improvement in the retrieval algorithm, we examined the retrieval of land surface reflectance. The MODIS aerosol product provides retrieved land surface reflectance in the 0.47, 0.65, and 2.1 µm channels. We looked at the land surface reflectance for the simulation of Fig. 10c and d that now matched the source aerosol optical depth reasonably well. The simulation was run under constant surface albedo conditions, and we would have expected to see a result, with some degree of uncertainty of course, that would match the given constant surface albedo. However the retrieved land surface reflectance appeared to be a near-linear function of aerosol optical depth. One possible explanation for this behavior may involve the assumed fraction of coarse-mode aerosol in the aerosol model mixture. To examine this hypothesis we performed a MYD04 retrieval using an aerosol model setting so that MYD04 retrieval only used fine-mode particles. The retrieval results, depicted in Fig. 11, confirm that the near-colinearity of surface reflectance and AOD was indeed directly related to fraction of coarse-mode particles, such as dust, in the assumed aerosol mixture. Of course there is no way to know exactly what fraction of coarse-mode particles may be present in the mixture as the MODIS DT algorithm does not have enough information content to constraint the fine/coarse-mode fraction over land (Levy et al., 2007). However, it can be noted that if such colinearity is seen during a specific local aerosol study maybe during a field campaign, it may be suggested that the coarse-mode fraction assumed operationally for that particular region may be too high. An analysis of MODIS operational retrievals to identify locations and times where this colinearity exists may be useful to identify regions where the assumed coarse/fine-mode fraction might need to be adjusted. Figure 11 illustrates the impact of coarse-mode fraction selection on land surface reflectance retrievals for Brazil 1 and Brazil 2 cases. The fine-to-coarse-mode ratio does not appear to have an impact on the low bias of MYD04 AOD retrieval vs. “ground truth” comparisons presented in the earlier figures.

This paper is a continuation of work started in Wind et al. (2013). The multi-sensor cloud retrieval simulator code (MCRS) had been extended to add aerosol effects to radiance simulations. The current implementation of the MCARS code generates synthetic radiances by sending GEOS-5 model fields and MODIS sensor geometry and location information to the DISORT-5 radiative transfer core. The radiance and reflectance data are output in a standard MODIS Level-1B format that can be transparently ingested by any retrieval or analysis code that reads data from the MODIS instrument.

After the aerosol properties module had been added to the MCARS code, we used the simulator to perform detailed analysis of performance of the operational MODIS dark target aerosol properties retrieval product for the Aqua MODIS instrument (MYD04). We found the cause of known low bias in MYD04-retrieved AOD for smoke when compared to in situ measurements. We suggest that the MYD04 retrieval might consider using column relative humidity from ancillary data when performing retrievals in regions that are defined to be dominated by smoke aerosols. The mismatch between the aerosol single scattering albedo assumed by MYD04 and the given synthetic single scattering albedo is the cause of the low bias at higher AODs. The impact of surface inhomogeneity is also quantifiable. Whereas it may not be possible to make an operationally actionable item from retrieval behavior when surface is made homogeneous, it may be possible to deduce an estimate of retrieval uncertainty due to land surface effects.

This study is a good example of capabilities of the MCARS code. We are planning many more studies of retrieval algorithm performance.

The MCARS results give a relationship between aerosol single scattering albedo, bias in retrieved aerosol optical depth, and column relative humidity. One of our future directions is to examine further this relationship and possibly establish a solid parameterization that could be used by the modeling community to reduce biases in assimilated observations that might display a similar low bias when compared to in situ measurements.

The MCARS simulator is currently being extended to calculate synthetic radiances for the Meteosat Second Generation Spinning Enhanced Visible Infrared Radiometer Imager (MSG-SEVIRI).

The MCARS code and any datasets produced, including all data shown (GEOS-5 input in netCDF4 and all MODIS output in HDF4 file format) and discussed in this paper, are available to users free of charge by contacting the authors and becoming a registered user of this software package so that any updates to code or datasets can be issued directly. There may be additional, wider distribution means in the future as needed. We have not deemed it practical up to this time to release the MCARS source code into general-purpose source repositories. The data files are quite large with source input data being on the order of 20 Gb for each MODIS-like granule created. The GEOS-5 model source code is publicly available, and we may release the MCARS code under the same NASA Open Source Agreement and the same repository in the coming year.