Summary:

This paper presents a method to quantify the accuracy of the MOD06ACAERO retrieval through the combination of a forward radiative transfer model built upon the GEOS-5 model. This manuscript is aimed at evaluating the MOD0ACAERO product for inclusion in a data assimilation scheme for GEOS-5. This aerosol assimilation would be quite useful considering the lack of continuous observations of aerosol over clouds in the South East Atlantic Ocean. The use of MCARS as a closure study illustrates quite well the needed filtering processes of MOD06ACAERO in order to be used as a data source for assimilation and puts both the MCARS and MOD06ACAERO in high regards. The data and methods are clearly defined, while the codes are not currently available in a public access repository, it is mentioned that it can be accessed by contacting the author.

The manuscript is very well written and is quite pertinent to model development, particularly GEOS-5. After review of the manuscript there are only very few minor revisions to be made, however one concern, which could be elevated to a major revision, regarding the underlying aerosol model and how it is represented in both the retrieval and GEOS-5 should be addressed before publication (see below). It is suggested that this manuscript is to be published with this one major revision.

Major Comment:

The MCARS application of building a retrieval OSSE on top of GEOS-5 seems to be very powerful, but there is limited discussion of the potential pitfalls of such an approach. Of immediate concern for interpreting the retrieval accuracy of MOD06ACAERO is the aerosol microphysical properties, notably the single scattering albedo (SSA) that may not be well represented in both the retrieval and GEOS-5, which could lead to an artificial inflation of the retrieval accuracy metrics, and potentially leading to inaccuracies in model assimilated fields. For the SEAO, it has been found that the SSA is much more variable, both dependent on the measurement method, and from case study to case study (See Pistone et al., 2019). Differences in MOD06ACAERO and ORACLES data can be partially attributed to the underlying aerosol model (SSA) used in MOD06ACAERO (see LeBlanc et al., 2020). This potential problem is identified in Line 305-306, but elaboration of the impacts could be expanded. Inclusion of a figure showcasing the differences in Single Scattering Albedo output from GEOS-5 and used in MCARS as compared to the model from MOD06ACAERO (MOD04_DT), might be instructive here with an option to include the spread of SSA either from SAFARI2000 or from ORACLES as presented for few cases in Pistone et al., 2019.

General Comments:

1. Of general interests, and idea for future directions: Can MCARS be used to evaluate the emission sources (maybe as represented as MODIS fire counts)?

Specific Comments:

1. Line 62: There is evidence that neighboring clear sky AOD retrievals match the above cloud AOD: Shinozuka et al., 2020
2. Line 71: please add the reference in the parenthetical “(add reference)”
3. Line 77-78: From reading of the figure it looks closer to 70-80% and 30-50% for June and September respectively.
4. Line 96: Redemann et al., citation can be updated to the overview paper: “Redemann, J., Wood, R., Zuidema, P., Doherty, S., Luna, B., LeBlanc, S., Diamond, M., Shinozuka, Y., Chang, I., Ueyama, R., Pfister, L., Ryoo, J., Dobracki, A., da Silva, A., Longo, K., Kacenelenbogen, M., Flynn, C., Pistone, K., Knox, N., Piketh, S., Haywood, J., Formenti, P., Mallet, M., Stier, P., Ackerman, A., Bauer, S., Fridlind, A., Carmichael, G., Saide, P., Ferrada, G., Howell, S., Freitag, S., Cairns, B., Holben, B., Knobelspiesse, K., Tanelli, S., L’Ecuyer, T., Dzambo, A., Sy, O., McFarquhar, G., Poellot, M., Gupta, S., O’Brien, J., Nenes, A., Kacarab, M., Wong, J., Small-Griswold, J., Thornhill, K., Noone, D., Podolske, J., Schmidt, K. S., Pilewskie, P., Chen, H., Cochrane, S., Sedlacek, A., Lang, T., Stith, E., Segal-Rozenhaimer, M., Ferrare, R., Burton, S., Hostetler, C., Diner, D., Platnick, S., Myers, J., Meyer, K., Spangenberg, D., Maring, H. and Gao, L.: An overview of the ORACLES (ObseRvations of Aerosols above CLouds and their intEractionS) project: aerosol-cloud-radiation interactions in the Southeast Atlantic basin, Atmos. Chem. Phys., 21, 1507–1563, doi:10.5194/acp-21-1507-2021, 2021.”
5. Line 185-186, Why were the resolution of streams in DISORT increased from the somewhat standard 16 to 32or even 64? This change increases computation time, so it is likely needed to better resolve the radiances, but what was the metric used to determine this need. (especially for outside of the cloud bow region)
6. Line 265: dataset number of points at 13.5 millions seems to present an inflated sense of statistics, it would be interesting to report the number of successful MOD06ACAERO retrievals that are used in the comparison.
7. Figure 4-10: The 2 different panels seem to have differences in their representative color levels. Inclusion of a colorbar would be useful here.

References:

LeBlanc, S. E., Redemann, J., Flynn, C., Pistone, K., Kacenelenbogen, M., Segal-rosenheimer, M., Shinozuka, Y., Dunagan, S., Dahlgren, R. P., Meyer, K., Podolske, J., Howell, S. G., Freitag, S., Small-griswold, J., Holben, B., Diamond, M., Wood, R., Formenti, P., Piketh, S., Maggs-Kölling, G., Gerber, M. and Namwoonde, A.: Above-cloud aerosol optical depth from airborne observations in the southeast Atlantic, Atmos. Chem. Phys., 20, 1565–1590, doi:10.5194/acp-20-1565-2020, 2020.

Pistone, K., Redemann, J., Doherty, S., Zuidema, P., Burton, S., Cairns, B., Cochrane, S., Ferrare, R., Flynn, C., Freitag, S., Howell, S. G., Kacenelenbogen, M., LeBlanc, S., Liu, X., Schmidt, K. S., III, A. J. S., Segal-Rozenhaimer, M., Shinozuka, Y., Stamnes, S., van Diedenhoven, B., Van Harten, G. and Xu, F.: Intercomparison of biomass burning aerosol optical properties from in situ and remote-sensing instruments in ORACLES-2016, Atmos. Chem. Phys., 19, 9181–9208, doi:10.5194/acp-19-9181-2019, 2019.

Shinozuka, Y., Kacenelenbogen, M. S., Burton, S. P., Howell, S. G., Zuidema, P., Ferrare, R. A., LeBlanc, S. E., Pistone, K., Broccardo, S., Redemann, J., Schmidt, K. S., Cochrane, S. P., Fenn, M., Freitag, S., Dobracki, A., Segal-Rosenheimer, M. and Flynn, C. J.: Daytime aerosol optical depth above low-level clouds is similar to that in adjacent clear skies at the same heights: airborne observation above the southeast Atlantic, Atmos. Chem. Phys., 20(19), 11275–11285, doi:10.5194/acp-20-11275-2020, 2020.

Review comments on 'Analysis of the MODIS Above-Cloud Aerosol Retrieval Algorithm Using MCARS' by Wind et al.

This manuscript uses Multi-sensor Cloud and Aerosol Retrieval Simulator (MCARS) and GEOS5 Nature Run (G5NR) model output to simulate the MODIS radiance, run above cloud aerosol retrieval using generated radiance, and evaluate the MODIS above cloud aerosol retrieval product (MOD06ACAERO), focusing on the absorbing aerosol above low-level water cloud scene from Southeast Atlantic Ocean. The community need for such an aerosol product is urgent since it fills the aerosol data gap in regular MODIS products, which can only retrieve aerosol in the clear sky and on a larger scale (i.e., 3X3 or 10X10 km). Such a 'truth in truth out' study can validate the instrument performance, improve retrieval algorithm, and investigate the potential uncertainties in the retrieval product. This method is beneficial, especially in future satellite missions. Overall, this is a well-written paper with a clear description of the data and methods, and the results are presented and discussed in an appropriate and concise manner. This paper should be accepted essentially with one major comment and few minor revisions discussed below.

Major comment:

One potential uncertainty in aerosol retrieval comes from the aerosol model, especially the single scattering albedo (SSA). The MOD06ACAERO algorithm uses the standard MODIS dark target aerosol model. However, this SSA is higher than the ORACLES campaign reported values (Pistone et al., 2019). In a recent GRL paper by Chang et al. 2021, they compared the MOD06ACAERO product with their own MODIS above cloud AOD retrieval and many other observations (e.g., HSRL2, 4STAR) from the ORACLES campaign. And they found that the difference in SSA is a major source of uncertainty in above cloud aerosol retrieval. What is the spectral SSA in the dark target aerosol model, and how it compares to G5NR SSA and real SSA measured from the filed campaign? The correlation between MOD06ACAERO and G5NR ACAOD can be high as long as both aerosol models are similar to each other, but this doesn't mean they are accurate or close to the observations. Therefore, the authors need to address this problem in the paper. And for any algorithm development study, a validation by real observations would be helpful.

Specific comments:

1. Please add color bars to figures 4-10.
2. Line 71: Reference is missing as authors marked in the text.
3. Line 96: The reference Redemann et al. 2019 may not be appropriate. Consider citing the ORACLES overview paper by Redemann et al. 2021.
4. Line 110: In MCARS simulation, is instrument measurement uncertainty also accounted?
5. Line 118: Increase the number of streams will increase the accuracy of simulated radiance. Meanwhile, the computational time also increases, but how much more accuracy gain due to this change?
6. Line 136: It will be great to compare with the field campaign data (e.g., NASA ORACLES).
7. Line 270: 'from' is used twice, please delete one.

References:

Chang, I., Gao, L., Burton, S. P., Chen, H., Diamond, M. S., Ferrare, R. A., et al. (2021). Spatiotemporal heterogeneity of aerosol and cloud properties over the southeast Atlantic: An observational analysis. Geophysical Research Letters, 48, e2020GL091469. https://doi.org/10.1029/2020GL091469

Redemann, J., Wood, R., Zuidema, P., Doherty, S. J., Luna, B., LeBlanc, S. E., Diamond, M. S., Shinozuka, Y., Chang, I. Y., Ueyama, R., Pfister, L., Ryoo, J.-M., Dobracki, A. N., da Silva, A. M., Longo, K. M., Kacenelenbogen, M. S., Flynn, C. J., Pistone, K., Knox, N. M., Piketh, S. J., Haywood, J. M., Formenti, P., Mallet, M., Stier, P., Ackerman, A. S., Bauer, S. E., Fridlind, A. M., Carmichael, G. R., Saide, P. E., Ferrada, G. A., Howell, S. G., Freitag, S., Cairns, B., Holben, B. N., Knobelspiesse, K. D., Tanelli, S., L'Ecuyer, T. S., Dzambo, A. M., Sy, O. O., McFarquhar, G. M., Poellot, M. R., Gupta, S., O'Brien, J. R., Nenes, A., Kacarab, M., Wong, J. P. S., Small-Griswold, J. D., Thornhill, K. L., Noone, D., Podolske, J. R., Schmidt, K. S., Pilewskie, P., Chen, H., Cochrane, S. P., Sedlacek, A. J., Lang, T. J., Stith, E., Segal-Rozenhaimer, M., Ferrare, R. A., Burton, S. P., Hostetler, C. A., Diner, D. J., Seidel, F. C., Platnick, S. E., Myers, J. S., Meyer, K. G., Spangenberg, D. A., Maring, H., and Gao, L.: An overview of the ORACLES (ObseRvations of Aerosols above CLouds and their intEractionS) project: aerosol–cloud–radiation interactions in the southeast Atlantic basin, Atmos. Chem. Phys., 21, 1507–1563, https://doi.org/10.5194/acp-21-1507-2021, 2021.

Pistone, K., Redemann, J., Doherty, S., Zuidema, P., Burton, S., Cairns, B., Cochrane, S., Ferrare, R., Flynn, C., Freitag, S., Howell, S. G., Kacenelenbogen, M., LeBlanc, S., Liu, X., Schmidt, K. S., Sedlacek III, A. J., Segal-Rozenhaimer, M., Shinozuka, Y., Stamnes, S., van Diedenhoven, B., Van Harten, G., and Xu, F.: Intercomparison of biomass burning aerosol optical properties from in situ and remote-sensing instruments in ORACLES-2016, Atmos. Chem. Phys., 19, 9181–9208, https://doi.org/10.5194/acp-19-9181-2019, 2019.

This manuscript evaluates the performance of MODIS Above-Cloud AEROsol retrieval algorithm (MOD06ACAERO) in the Southeastern Atlantic for biomass burning aerosols using Multi-Sensor Cloud and Aerosol Retrieval simulator (MCARS). Lack of aerosol retrievals in cloudy conditions is a well-known problem in the aerosol community. This study helps address this issue using a closure study of MOD06ACAERO above-cloud Aerosol Optical Depth (AOD). The paper meticulously describes the observations and model and discusses the limitations and strengths of the retrieved AOD for different filtering conditions based on cloud cover, zenith angle, and pixel-level retrieval uncertainty.  The results from this study are beneficial for model evaluation studies and design of future satellite missions.

Filtering conditions and related error metrics from this study provide an opportunity to test the model performance from assimilation of MOD06ACAERO AOD retrievals. Availability of code and data will make it easier to test this approach for other satellite instruments and aerosol types. The paper is well-written and easy to read. However, there are some major revisions that need to be addressed before the paper is published. 

Major Revisions: 

• Differences in the aerosol model between MOD06ACAERO retrievals and GEOS-5 need to be discussed in the analysis section. This can be included before describing the results from the sensitivity tests (before line 290).
• What are the uncertainties/biases in the retrieved AOT (using MCARS) from this study? What are the factors influencing the uncertainties over land and ocean for the retrieved AOT using MCARS synthetic radiances? It is not clear if the algorithm is useful for clear-sky conditions or can be used only in cloudy conditions.
• Figure 2. Shows real data example of MOD06ACAERO Aerosol Optical Depth (AOD) retrieval and MODIS Dark-Target (DT) aerosol retrieval. It is clear that the data gaps are reduced in panel c). However, there are differences in AOD values between panel b) and c) even in areas where MODIS DT clear-sky retrievals are available. What are the reasons for these differences?
• How does the retrieved AOD compare against MODIS DT in clear-sky conditions?
• Figure 4-10 – it is not shown what do colors represent in the panels. Do they represent probability density values? Adding a colorbar and description of the colorbar in the caption is necessary.

Minor Revisions:

• In general, it will be interesting to assess the performance of retrieved AOD in the Atlantic for dust transport from Africa above the clouds. For this paper, it will be helpful to include comments/references on the performance of MOD06ACAERO for other aerosol types.

• Although the goal of this paper is to evaluate MOD06ACAERO, comparison of the results against ORACLES data will strengthen the paper.
• Line 71 add reference – reference is missing. “… daily mean cloud fraction greater than 50% in the MODIS Daily Level-3 gridded product (add reference)” .
• Lines 277-280 : It is understandable that G5NR is a free running model and any resemblance to real data is a coincidence. Please elaborate (or rephrase) lines 277-280 to include comments on a similar comparison of cloud amount/distribution in MODIS granules and G5NR. Spatial distribution of clouds and cloud optical properties between G5NR and MODIS granules can affect the retrieved AOD. How is this addressed? (see Le Blanc et al., 2020).
• Line 270 – “from from the simulation offers”. Remove one.
• Figure 4-10 - colorbar is missing. Since the data density between panels a and b change in these figures, it would be more meaningful to include normalized error metrics in these figures such as normalized mean bias or normalized RMSE, fractional gross error. Please add what colorbar represents in the figure captions (perhaps, probability densiity values) ? 

References

LeBlanc, S. E., Redemann, J., Flynn, C., Pistone, K., Kacenelenbogen, M., Segal-Rosenheimer, M., ... & Namwoonde, A. (2020). Above-cloud aerosol optical depth from airborne observations in the southeast Atlantic. Atmospheric Chemistry and Physics, 20(3), 1565-1590.