The performance of the CNN model is directly dependent on how well the model understands the relationship between the inputs (meteorology and ozone precursors) and output (ozone concentration). Compared with meteorological variables, emission sources from volatile organic compounds (VOCs) and NOx experience less variability in time. Thus, meteorological variables play an important role in governing the variation of the ozone at different times throughout the year (Choi, 2014; Pan et al., 2019). Temperature, wind speed, and relative humidity (RH) are among the most important meteorological parameters affecting ozone variation.

Figure 2 shows the wavelet power transform of the aforementioned meteorological variables for 2017. Since we used an hourly time series to calculate the wavelet powers, both the index and the period are in hours. The figure also locates five time periods, which indicates significant performance variations. From Fig. S6 in the Supplement, the CNN model under-performed during weeks 3–9 and 44–51, labeled the “Worst CNN results” in Fig. 2. For weeks 14–22 and 42–44, the CNN model showed the best forecasting results. Between weeks 29 and 33, the CNN model produced significant underestimations, labeled “Large under-prediction” in Fig. 2. The figure shows strong wavelet powers during a 24 h (daily) period for all variables, the results of strong diurnal variation of these parameters, which are directly or indirectly controlled by sunlight (e.g., temperature, relative humidity, etc.). While the wavelet powers for wind speed were generally larger than RH, the temperature showed lower but more consistent daily modes. This finding is important since the CNN model can more accurately detect specific “patterns” in the temperature than those in the wind speed and RH. Thus, when the daily modes are stronger in temperature, the CNN model likely performs better. In contrast, when the daily modes of the meteorological variables are relatively weak, the CNN model performs poorly (see Fig. 2).

The large coarse modes in the wind speed and RH lead to significant over and underestimation of the CNN model. Figure S8 shows the polar frequency (influenced by the wind speed) of the CNN modeling bias in various months. As the figure shows, while southwesterly winds in August 2017 were associated with relatively large underpredictions boosted by pollution transport from the Incheon area, north–northwesterly winds with air coming from less urbanized regions were allied with notable overpredictions.

Figure S9 in the Supplement compares the CNN model predictions with observational data for the seasons with respect to levels of RH. The figure showed the largest differences in the CNN model predictions (both overpredictions and underpredictions) when the level of RH was close to the extreme (very high and very low). This finding was particularly evident for the summer months when the model showed poor performance at capturing high-ozone episodes. This finding underscores the importance of coarse models from the wavelet analysis during the warm months. Directly indicating the overpredictions or underpredictions by the model through these modes, however, is challenging. For instance, Fig. S10 in the Supplement shows one high-ozone episode in July 2017, when the daily ozone peak exceeded 90 ppb on 2 continuous days at most stations. Here, the overprediction of the CNN model was associated with high RH, while the underprediction was linked to low RH, indicating more complexity among the relationships between meteorological factors and ozone formation or depletion.

Another reason for the poor performance of the CNN model during the selected time period was the relatively large coarse modes (period >24 h). The CNN model received information about only the last day; hence, it was unable to address the bi-daily and weekly trends with the input data. For instance, for time periods with large underpredictions, coarse modes in the wind speed were even larger than the daily modes. Thus, employing a longer history would adequately explain the relationship between wind speed and ozone. In the comparison of the average wavelet powers in various periods (from daily to weekly modes) of CNN predictions and observational data, Fig. 3 shows that the powers for both time series match periods of approximately 24 h. After 32 h, however, the wavelet power of the CNN model shrinks to a relatively constant power while that for the observation reaches local extrema at around 3, 5, and 7 d.

Although wavelet analysis indicates that modes coarser than 24 h are important components of the ozone time series, their relationship to CNN model accuracy can be complicated. Figure 4 compares wavelet powers for both fine and coarse modes with a correlation coefficient (r) in 25 ozone stations in Seoul. For stations closer to the downtown area (i.e., those with station numbers under 11), the fine modes had fewer wavelet powers than those for stations in less urbanized areas, indicating that the relationship between ozone concentrations with local emissions was evident in the less urbanized areas than it was in the other areas. The coarse modes, however, varied from station-to-station with relatively high coarse wavelet power for those in less urbanized areas. Nonetheless, no evidence points to a clear relationship between either coarse or fine wavelet modes and the accuracy of the model. Figure 4 shows that the CNN model generally performed better for stations close to downtown Seoul. Because Seoul has only one meteorological station, these stations had access to more realistic weather parameters in their training and prediction process.

In their discussion of several air quality forecasting models that incorporated machine learning algorithms, including CNN, deep neural networks, and decision trees, Eslami et al. (2020a) and Eslami et al. (2020b) claimed that the algorithms encounter a significant modeling bias while estimating air quality concentrations during the nighttime. This bias reduced the prediction accuracy of nighttime ozone concentrations, compared to daytime concentrations, by more than 20 %. A similar issue is also encountered by CTMs, even those with complex physical and chemical equations that explain the diurnal variation of ozone concentrations.

One reason for this modeling bias was likely the result of variation among the meteorological inputs during the nighttime. Although their absolute values were generally higher they were during the daytime, the relative frequency of variation was more pronounced during the nighttime, causing a discontinuity in the learning process of the CNN model. Since both daytime and nighttime hours were inputs, the CNN model minimized the cost function that contained “normalized” errors during both daytime and nighttime hours (the cost function was the mean-squared errors or 24 h ozone predictions at each step). Generally, there are more daytime hours than nighttime hours (see Fig. S11 in the Supplement). Also, the accumulation of NO2 concentrations for these extreme cases was mainly due to stagnant atmospheric conditions with wind speeds close to their yearly minimum values (see Fig. S12a in the Supplement for scatterplots with levels of wind speeds). As a result, the CNN model was vulnerable to characteristic bias in nighttime ozone estimations. As a customized cost function could be a potential solution to this limitation, it requires further investigation.

The performance of the CNN model in predicting nighttime ozone concentrations also suffered because of the misinterpretation of extreme conditions of the input parameters. Figure 5 shows scatterplots that compare CNN predictions and observations by the levels of two important ozone precursors (NO2 concentrations) and meteorological variables (RH %) separated into daytime and nighttime. The NO2 concentration was generally higher during the nighttime when the ozone concentration was near zero for extreme NO2 values because of conditions amenable to ozone depletion with the absence of sunlight. Unable to capture this relationship, however, the CNN model overestimated these cases (See Fig. 5a).

In contrast to the above-mentioned overestimated events, Fig. 5b shows an underestimation of nighttime ozone when the level of RH % was generally high, primarily during warm days. A similar pattern occurred when the surface pressure was accounted for (Fig. S12b in the Supplement). Such underestimated events occurred for two reasons. One is that high (or low) levels of RH % and surface pressure generally occur at about the same time during the early morning (or late afternoon) when the planetary boundary layer (PBL) is at its lowest (or highest) level during the day. In these extreme conditions, the earlier sunrise (or later sunset) during the summer months established a condition that elevated ozone concentrations. As these events normally occurred only during short periods of time, the CNN model was not sufficiently trained to capture these relationships.

Figure 6 shows the time-series of CMAQ, CMAQ-CNN, and observed daily ozone concentrations at three EPA stations. These stations were selected because the IOA accuracy of the CMAQ-CNN model was either more than 0.9 (Fig. 6a and b) or 20 % more than that of CMAQ (Fig. 6c). Figure 7 compares the DTW distance analysis of CMAQ and CMAQ-CNN for the same stations. These are three typical cases of satisfactory improvement by the CMAQ-CNN post-processing approach:

Figures 6a–7a show that observed ozone concentrations in this California location were higher at the beginning of the ozone season, followed by relatively steady values ranging between 20–40 ppb. After May, however, CMAQ significantly overestimated daily ozone concentrations. The overestimation was more pronounced at the end of the ozone season, resulting in an overall IOA accuracy of 0.73. The DTW distance analysis showed a consistent distance between CMAQ predictions and observed values. Because of this consistency, the CMAQ-CNN model recognized the bias trends in CMAQ, boosting its prediction accuracy by 0.17, even though the large distance from the CMAQ predictions (mean distance = 0.52) mirrored a relatively significant overestimation in the CMAQ-CNN post-processed results.

Figures 6b–7b show that here the trend in ozone concentrations followed a U-shaped curve in the ozone season because of strong summer winds coming from the large bodies of water near Florida (the North Atlantic Ocean and the Gulf of Mexico). For this station, CMAQ accurately predicted this trend throughout the ozone season with a relatively constant bias from July to September. As a result, the overall accuracy of the IOA was 0.84 for the CMAQ prediction. The CMAQ was also consistent with the DTW analysis, with two distance gaps in July and September (at the beginning and the end of the CMAQ overestimation period). The CMAQ-CNN model, recognizing the adequate performance of the base model in its post-processing algorithm, further improved the IOA accuracy of CMAQ by around 10 %.

Figures 6c–7c show that the trend of observed ozone had a steady decrease in this northeastern state because of the significantly cooler summer and fall months. This trend, along with the fewer ozone emission sources surrounding this station, resulted in the formation of less ozone during the ozone season. The CMAQ model overestimated ozone concentrations by more than 50 % during most of the season with a relatively large mean DTW distance (0.62). The CMAQ-CNN model was able to address this issue because of the consistency of the bias trend in CMAQ predictions (see left panel in Fig. 7c for DTW distance). Thus, overall, the accuracy of IOA improved by 0.2.

The satisfactory post-processing results using the CMAQ-CNN model were mainly characterized by the regularity of the bias trend in CMAQ as the base model for training the CNN model. As shown by the DTW distance analysis, when the DTW distance of CMAQ predictions from observed values was consistent throughout the ozone season, the CNN model was able to improve the CMAQ results to a reliable level (IOA >0.8). To test this hypothesis, we used the CMAQ-CNN post-processing approach in typical unsatisfactory scenarios.

Figure 8 compares the time series of ozone observations with the CMAQ and CMAQ-CNN models at three selected EPA stations. For all of these stations, the CMAQ-CNN model failed to reach a reliable IOA accuracy level of 0.8, while the accuracy of the CMAQ model improved. Figure 9 represents the DTW distance analysis of the two models and the ozone observation for the same stations. Unsatisfactory improvement by the CMAQ-CNN model occurred in the following three cases.

Figures 8a–9a show that the ozone trend in this station fluctuated throughout the ozone season with frequent spikes in May, July, and October, primarily the result of biomass burning (Choi et al., 2016). While the CMAQ model predicted ozone concentrations with a relatively small bias (IOA = 0.7), the bias trend varied from time to time, i.e., trends of underpredictions and overpredictions changed frequently. A footprint of these trends, i.e., changes in the path of the distance trend, is evident in the DTW analysis. This inconsistency was mirrored in the equivalent DTW analysis for the CMAQ-CNN model by a consistent distance trend, resulting in an unsatisfactory IOA accuracy level (IOA = 0.78) with an increased mean DTW distance (0.89 compared to 0.74 for the CMAQ time series).

Figures 8b–9b show that the trend in this California location was a relatively constant concentration of ozone generally ranging between 10–30 ppb. The CMAQ model significantly overpredicted ozone concentrations throughout the entire time period, mostly as a result of the proximity of this station to the Pacific Ocean (San Diego County), which controls the variation in the daily ozone concentration (Pan et al., 2017). The DTW distance analysis shows a significant yet steady spike in the distance between CMAQ and the observation. Thus, even though the CMAQ-CNN significantly improved the accuracy of the CMAQ model (IOA = 0.63 compared to CMAQ IOA = 0.44), the large distance accounted for the underperformance of the post-processing approach. This also mirrored the consistent distance in the CMAQ-CNN distance trend (see the right panel in Fig. 9c).

Figures 8c–9c show that, in this station, the ozone concentration followed an infrequent trend with lows and highs spread indiscriminately across the ozone season, the result of several factors affecting air pollution in this region, including biomass burning, a strong frontal system, and other conditions. As a result, the CMAQ model underperformed, with substantial overestimation during most of the time period (IOA = 0.55). In addition, the bias of the CMAQ model did not follow as clear a trend as the DTW distance analysis. The CMAQ-CNN model improved the prediction results by more than 10 % with a reduced DTW distance (0.27m versus 0.35 for the CMAQ time series). Nevertheless, the varying ozone trend accompanying the inconsistency in the prediction bias trend resulted in the low overall accuracy of the IOA of the CMAQ-CNN for this station (IOA = 0.67).

Unlike the satisfactory cases, the unsatisfactory post-processing results using the CMAQ-CNN model stemmed from the inconsistency in the bias trend found by the DTW distance analysis. Another influential factor was the variability of observed ozone concentrations. Because of the frequent variation in the observational data, it was more complicated to train the CMAQ-CNN model so that it addressed the bias in the CMAQ model. The geographical location of a station was also an important factor in the improvement level of the post-processing approach. Proximity to the large body of water and/or sources from biomass burning during the ozone season were among the influential geographical features. Also, as Figs. 8 and 9 show, the DTW distances of the CMAQ-CNN predictions from the observed ones followed a consistent trend. Therefore, the information in Figs. 6 and 7 indicate that a secondary post-processing model might be a possible solution to boosting prediction accuracy.